This is an automated email from the ASF dual-hosted git repository. zhaocong pushed a commit to branch add_195_3.0_note in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
commit b8ac535749cfef5d1f1449bf9eb4b14bf9b70a74 Author: coderzc <[email protected]> AuthorDate: Mon May 8 15:28:07 2023 +0800 Add PIP 195 to 3.0 release note --- release-notes/versioned/pulsar-3.0.0.md | 48 +++++++++++++++++---------------- 1 file changed, 25 insertions(+), 23 deletions(-) diff --git a/release-notes/versioned/pulsar-3.0.0.md b/release-notes/versioned/pulsar-3.0.0.md index b02ba2a8739..01a60b2e588 100644 --- a/release-notes/versioned/pulsar-3.0.0.md +++ b/release-notes/versioned/pulsar-3.0.0.md @@ -35,6 +35,7 @@ sidebar_label: Apache Pulsar 3.0.0 - PIP-188: Cluster migration or Blue-Green cluster deployment support in Pulsar [#16551](https://github.com/apache/pulsar/pull/16551) - PIP-192: New Pulsar Broker Load Balancer [#16691](https://github.com/apache/pulsar/pull/16691) - PIP-193: Sink preprocessing Function [#16739](https://github.com/apache/pulsar/issues/16739) +- PIP-195: New bucket based delayed message tracker [#16763](https://github.com/apache/pulsar/issues/16763) - PIP-214: Add broker level metrics statistics and expose to prometheus [#18056](https://github.com/apache/pulsar/pull/18056) - PIP-218: Consumer batchReceive() single partition every receive [#18182](https://github.com/apache/pulsar/pull/18182) - PIP-229: Add a common interface to get fields of the MessageIdData [#18950](https://github.com/apache/pulsar/pull/18950) @@ -68,15 +69,34 @@ sidebar_label: Apache Pulsar 3.0.0 - Implement load data store by @Demogorgon314 in [#18777](https://github.com/apache/pulsar/pull/18777) - Implement broker registry for new load manager by @Demogorgon314 in [#18810](https://github.com/apache/pulsar/pull/18810) - Added ServiceUnitStateChannelImpl by @heesung-sn in [#18489](https://github.com/apache/pulsar/pull/18489) +- [Broker] PIP-195: New bucket based delayed message tracker [#16763](https://github.com/apache/pulsar/issues/16763) + - New bucket based delayed message tracker - interface&config&proto -part 1 by @coderzc in [#17344](https://github.com/apache/pulsar/pull/17344) + - Implement delayed message index bucket snapshot (create/load) - part2 by @coderzc in [#17611](https://github.com/apache/pulsar/pull/17611) + - Implement BookkeeperBucketSnapshotStorage - part3 by @coderzc in [#17677](https://github.com/apache/pulsar/pull/17677) + - Support internal cursor properties - part4 by @coderzc in [#17712](https://github.com/apache/pulsar/pull/17712) + - Implement BucketDelayedDeliveryTrackerFactory and load BucketDelayedDeliveryTracker - part6 by @coderzc in [#17756](https://github.com/apache/pulsar/pull/17756) + - Implement delayed message bucket snapshot recover - part5 by @coderzc in [#18420](https://github.com/apache/pulsar/pull/18420) + - Implement Filter out all delayed messages and skip them when reading messages from bookies - part7 by @coderzc in [#19035](https://github.com/apache/pulsar/pull/19035) + - Fix cursor skip read by @coderzc in [#19124](https://github.com/apache/pulsar/pull/19124) + - Implement delayed message index bucket snapshot(merge/delete) - part8 by @coderzc in [#19138](https://github.com/apache/pulsar/pull/19138) + - Make BucketDelayedDeliveryTracker can retry snapshot operation & improve logs by @coderzc in [#19577](https://github.com/apache/pulsar/pull/19577) + - Fix BucketDelayedDeliveryTracker merge issues by @coderzc in [#19615](https://github.com/apache/pulsar/pull/19615) + - Cut off snapshot segment according to maxIndexesPerBucketSnapshotSegment by @coderzc in [#19706](https://github.com/apache/pulsar/pull/19706) + - Add metrics for bucket delayed message tracker by @coderzc in [#19716](https://github.com/apache/pulsar/pull/19716) + - Don't clean up BucketDelayedDeliveryTracker when all consumer disconnect by @coderzc in [#19801](https://github.com/apache/pulsar/pull/19801) + - Add topicName and cursorName for ledger metadata of bucket snapshot by @coderzc in [#19802](https://github.com/apache/pulsar/pull/19802) + - Make bucket merge operation asynchronous by @coderzc in [#19873](https://github.com/apache/pulsar/pull/19873) + - Clear delayed message when unsubscribe & Make clear operation asynchronous by @coderzc in [#19901](https://github.com/apache/pulsar/pull/19901) + - Merge multiple buckets at once by @coderzc in [#19927](https://github.com/apache/pulsar/pull/19927) + - Ensure previous delayed index be removed from snapshotSegmentLastIndexTable & Make load operate asynchronous by @coderzc in [#20086](https://github.com/apache/pulsar/pull/20086) + - Fix avoid future of clear delayed message can't complete by @coderzc in [#20075](https://github.com/apache/pulsar/pull/20075) + - Support disabling delayed bucket merging. by @mattisonchao in [#20155](https://github.com/apache/pulsar/pull/20155) - [Broker] Pip-257: Add AuthenticationProviderOpenID by @michaeljmarshall in [#19849](https://github.com/apache/pulsar/pull/19849) - [Broker] Make `LeastResourceUsageWithWeight` thread safe by @lifepuzzlefun in [#20159](https://github.com/apache/pulsar/pull/20159) -- [Broker] Support disabling delayed bucket merging. by @mattisonchao in [#20155](https://github.com/apache/pulsar/pull/20155) - [Broker] Supply download auth params when provided for k8s runtime by @michaeljmarshall in [#20144](https://github.com/apache/pulsar/pull/20144) - [Broker] AuthenticationProviderOpenID k8s error logs by @michaeljmarshall in [#20135](https://github.com/apache/pulsar/pull/20135) - [Broker] Implement authenticateAsync for AuthenticationProviderList by @michaeljmarshall in [#20132](https://github.com/apache/pulsar/pull/20132) - [Broker] Fix broker restart logic by @wolfstudy in [#20113](https://github.com/apache/pulsar/pull/20113) -- [Broker] Ensure previous delayed index be removed from snapshotSegmentLastIndexTable & Make load operate asynchronous by @coderzc in [#20086](https://github.com/apache/pulsar/pull/20086) -- [Broker] Fix avoid future of clear delayed message can't complete by @coderzc in [#20075](https://github.com/apache/pulsar/pull/20075) - [Broker] Refresh auth data if ProxyLookupRequests by @michaeljmarshall in [#20067](https://github.com/apache/pulsar/pull/20067) - [Broker] Fix flaky testCreateTopicWithZombieReplicatorCursor by @BewareMyPower in [#20037](https://github.com/apache/pulsar/pull/20037) - [Broker] Prevent range conflicts with Key Shared sticky consumers when TCP/IP connections get orphaned by @lhotari in [#20026](https://github.com/apache/pulsar/pull/20026) @@ -84,33 +104,24 @@ sidebar_label: Apache Pulsar 3.0.0 - [Broker] Make LedgerOffloaderFactory can load the old nar. by @horizonzy in [#19913](https://github.com/apache/pulsar/pull/19913) - [Broker] Ignore and remove the replicator cursor when the remote cluster is absent by @BewareMyPower in [#19972](https://github.com/apache/pulsar/pull/19972) - [Broker] Fix DeadLetterProducer creation callback blocking client io thread. by @lifepuzzlefun in [#19930](https://github.com/apache/pulsar/pull/19930) -- [Broker] Merge multiple buckets at once by @coderzc in [#19927](https://github.com/apache/pulsar/pull/19927) - [Broker] Fix the loss of bundle stats data reported to zookeeper, when the updateStats method is executed by @lordcheng10 in [#19887](https://github.com/apache/pulsar/pull/19887) -- [Broker] Clear delayed message when unsubscribe & Make clear operation asynchronous by @coderzc in [#19901](https://github.com/apache/pulsar/pull/19901) - [Broker] Fix can't send ErrorCommand when message is null value by @coderzc in [#19899](https://github.com/apache/pulsar/pull/19899) - [Broker] PIP-240: new public method unloadSubscription in PersistentTopic by @poorbarcode in [#19737](https://github.com/apache/pulsar/pull/19737) - [Broker] PIP-250: Add proxyVersion to CommandConnect by @michaeljmarshall in [#19618](https://github.com/apache/pulsar/pull/19618) -- [Broker] Make bucket merge operation asynchronous by @coderzc in [#19873](https://github.com/apache/pulsar/pull/19873) - [Broker] Suppress error logging when message expiration fails by @massakam in [#19778](https://github.com/apache/pulsar/pull/19778) - [Broker] Allow proxy to pass same role for authRole and originalRole by @michaeljmarshall in [#19557](https://github.com/apache/pulsar/pull/19557) - [Broker] Make authentication refresh threadsafe by @michaeljmarshall in [#19506](https://github.com/apache/pulsar/pull/19506) - [Broker] Fix RetentionPolicies constructor by @nodece in [#19777](https://github.com/apache/pulsar/pull/19777) -- [Broker] Add metrics for bucket delayed message tracker by @coderzc in [#19716](https://github.com/apache/pulsar/pull/19716) -- [Broker] Add topicName and cursorName for ledger metadata of bucket snapshot by @coderzc in [#19802](https://github.com/apache/pulsar/pull/19802) -- [Broker] Don't clean up BucketDelayedDeliveryTracker when all consumer disconnect by @coderzc in [#19801](https://github.com/apache/pulsar/pull/19801) - [Broker] Fix create cluster with empty url by @nodece in [#19762](https://github.com/apache/pulsar/pull/19762) - [Broker] Fix potential exception cause the policy service init fail. by @Technoboy- in [#19746](https://github.com/apache/pulsar/pull/19746) - [Broker] Validate originalPrincipal earlier in ServerCnx by @michaeljmarshall in [#19270](https://github.com/apache/pulsar/pull/19270) - [Broker] Fix issue where msgRateExpired may not refresh forever by @massakam in [#19759](https://github.com/apache/pulsar/pull/19759) - [Broker] Fix index generator is not rollback after entries are failed added. by @gaozhangmin in [#19727](https://github.com/apache/pulsar/pull/19727) -- [Broker] Cut off snapshot segment according to maxIndexesPerBucketSnapshotSegment by @coderzc in [#19706](https://github.com/apache/pulsar/pull/19706) - [Broker] Change type of `allowAutoTopicCreationType` to TopicType by @yuruguo in [#18814](https://github.com/apache/pulsar/pull/18814) - [Broker] Store the original authentication data by @nodece in [#19519](https://github.com/apache/pulsar/pull/19519) - [Broker] GetLastMessageId returns a wrong batch index of last message if enabled read compacted by @poorbarcode in [#18877](https://github.com/apache/pulsar/pull/18877) - [Broker] PIP-188 support blue-green cluster migration [part-1] by @rdhabalia in [#17962](https://github.com/apache/pulsar/pull/17962) - [Broker] Embed stringified exception in placeholder in log message by @massakam in [#19633](https://github.com/apache/pulsar/pull/19633) -- [Broker] Fix BucketDelayedDeliveryTracker merge issues by @coderzc in [#19615](https://github.com/apache/pulsar/pull/19615) -- [Broker] Make BucketDelayedDeliveryTracker can retry snapshot operation & improve logs by @coderzc in [#19577](https://github.com/apache/pulsar/pull/19577) - [Broker] Transaction buffer recover blocked by readNext by @poorbarcode in [#18833](https://github.com/apache/pulsar/pull/18833) - [Broker] Fix geo-replication admin by @nodece in [#19548](https://github.com/apache/pulsar/pull/19548) - [Broker] Counter of pending send messages in Replicator incorrect if schema future not complete by @poorbarcode in [#19242](https://github.com/apache/pulsar/pull/19242) @@ -136,7 +147,6 @@ sidebar_label: Apache Pulsar 3.0.0 - [Broker] Expect msgs after server initiated CloseProducer by @michaeljmarshall in [#19446](https://github.com/apache/pulsar/pull/19446) - [Broker] Close transactionBuffer after MessageDeduplication#checkStatus failed by @tjiuming in [#19157](https://github.com/apache/pulsar/pull/19157) - [Broker] Topic could be in fenced state forever if deletion fails by @nicoloboschi in [#19129](https://github.com/apache/pulsar/pull/19129) -- [Broker] Implement BucketDelayedDeliveryTrackerFactory and load BucketDelayedDeliveryTracker - part6 by @coderzc in [#17756](https://github.com/apache/pulsar/pull/17756) - [Broker] Fix the incorrect total size if use ML interceptor by @poorbarcode in [#19404](https://github.com/apache/pulsar/pull/19404) - [Broker] Fix currency bug in BucketDelayedDeliveryTracker#recoverBucketSnapshot by @lhotari in [#19394](https://github.com/apache/pulsar/pull/19394) - [Broker] Added isActive in ManagedCursorImpl by @heesung-sn in [#19341](https://github.com/apache/pulsar/pull/19341) @@ -174,7 +184,6 @@ sidebar_label: Apache Pulsar 3.0.0 - [Broker] Reduce GetReplicatedSubscriptionStatus local REST call by @AnonHxy in [#16946](https://github.com/apache/pulsar/pull/16946) - [Broker] Improve tls config on replication client and cluster cli by @nodece in [#17295](https://github.com/apache/pulsar/pull/17295) - [Broker] Persist correct markDeletePosition to prevent message loss by @michaeljmarshall in [#18237](https://github.com/apache/pulsar/pull/18237) -- [Broker] Implement delayed message index bucket snapshot(merge/delete) - part8 by @coderzc in [#19138](https://github.com/apache/pulsar/pull/19138) - [Broker] Deprecate blocking AuthorizationService, AuthorizationProvider methods by @michaeljmarshall in [#19180](https://github.com/apache/pulsar/pull/19180) - [Broker] Allow user lookup topic name with `-partition-` but no metadata by @mattisonchao in [#19171](https://github.com/apache/pulsar/pull/19171) - [Broker] Add ref count for sticky hash to optimize the performance of Key_Shared subscription by @codelipenghui in [#19167](https://github.com/apache/pulsar/pull/19167) @@ -194,10 +203,8 @@ sidebar_label: Apache Pulsar 3.0.0 - [Broker] ManagedLedger: move to FENCED state in case of BadVersionException by @eolivelli in [#17736](https://github.com/apache/pulsar/pull/17736) - [Broker] Do not print error log with stacktrace for 404 by @codelipenghui in [#19130](https://github.com/apache/pulsar/pull/19130) - [Broker] Fix executeWithRetry result is null by @coderzc in [#17694](https://github.com/apache/pulsar/pull/17694) -- [Broker] Fix cursor skip read by @coderzc in [#19124](https://github.com/apache/pulsar/pull/19124) - [Broker] Fix can not revoke permission after update topic partition by @TakaHiR07 in [#17393](https://github.com/apache/pulsar/pull/17393) - [Broker] Add oshi library to help control OS resources. by @mattisonchao in [#18990](https://github.com/apache/pulsar/pull/18990) -- [Broker] Implement Filter out all delayed messages and skip them when reading messages from bookies - part7 by @coderzc in [#19035](https://github.com/apache/pulsar/pull/19035) - [Broker] Fix missing return when internalGetReplicatedSubscriptionStatus by @HQebupt in [#19054](https://github.com/apache/pulsar/pull/19054) - [Broker] Throw AlreadyClosedException while request to a closed metadata store by @codelipenghui in [#19055](https://github.com/apache/pulsar/pull/19055) - [Broker] Fix thread safety issue in info-internal admin api for partitioned topics by @lhotari in [#19021](https://github.com/apache/pulsar/pull/19021) @@ -208,7 +215,6 @@ sidebar_label: Apache Pulsar 3.0.0 - [Broker] Fix direct memory leak in RawReaderImpl by @lhotari in [#18928](https://github.com/apache/pulsar/pull/18928) - [Broker] Make it Configurable to Require Subscription Permission by @michaeljmarshall in [#15576](https://github.com/apache/pulsar/pull/15576) - [Broker] Add config `fsyncEnable` for `RocksdbMetadataStore` by @coderzc in [#18801](https://github.com/apache/pulsar/pull/18801) -- [Broker] Implement delayed message bucket snapshot recover - part5 by @coderzc in [#18420](https://github.com/apache/pulsar/pull/18420) - [Broker] Remove the unused method internalGetList from PersistentTopicsBase by @codelipenghui in [#18775](https://github.com/apache/pulsar/pull/18775) - [Broker] Fix duplicated schemas creation by @codelipenghui in [#18701](https://github.com/apache/pulsar/pull/18701) - [Broker] Use correct file path for tls trust certificates by @michaeljmarshall in [#18712](https://github.com/apache/pulsar/pull/18712) @@ -221,8 +227,6 @@ sidebar_label: Apache Pulsar 3.0.0 - [Broker] Fix namespace not found will cause request timeout by @labuladong in [#18512](https://github.com/apache/pulsar/pull/18512) - [Broker] Avoid sharing the same random in multi-threads due to a performance issue by @Shawyeok in [#18660](https://github.com/apache/pulsar/pull/18660) - [Broker] System topic writer/reader connection not counted. by @Technoboy- in [#18369](https://github.com/apache/pulsar/pull/18369) -- [Broker] Implement BookkeeperBucketSnapshotStorage - part3 by @coderzc in [#17677](https://github.com/apache/pulsar/pull/17677) -- [Broker] Support internal cursor properties - part4 by @coderzc in [#17712](https://github.com/apache/pulsar/pull/17712) - [Broker] Avoid unnecessary creation of BitSetRecyclable objects by @lordcheng10 in [#17998](https://github.com/apache/pulsar/pull/17998) - [Broker] Fix broker cache eviction of entries read by active cursors by @lhotari in [#17273](https://github.com/apache/pulsar/pull/17273) - [Broker] Add return for PersistentMessageExpiryMonitor#findEntryFailed by @Pomelongan in [#17816](https://github.com/apache/pulsar/pull/17816) @@ -252,7 +256,6 @@ sidebar_label: Apache Pulsar 3.0.0 - [Broker] Cleanup NamespaceService by @AnonHxy in [#18259](https://github.com/apache/pulsar/pull/18259) - [Broker] Make CompactedTopicImpl.findStartPointLoop work more efficiently by @poorbarcode in [#17976](https://github.com/apache/pulsar/pull/17976) - [Broker] Do not serialize field bundleStats by @thetumbled in [#18150](https://github.com/apache/pulsar/pull/18150) -- [Broker] Implement delayed message index bucket snapshot (create/load) - part2 by @coderzc in [#17611](https://github.com/apache/pulsar/pull/17611) - [Broker] Fix flaky test: PrometheusMetricsTest.testDuplicateMetricTypeDefinitions by @nicoloboschi in [#18077](https://github.com/apache/pulsar/pull/18077) - [Broker] Avoid runtime check hasFilter in EntryFilterSupport by @AnonHxy in [#18066](https://github.com/apache/pulsar/pull/18066) - [Broker] Remove methods no longer used in AdminResource #7 by @Pomelongan in [#17969](https://github.com/apache/pulsar/pull/17969) @@ -264,7 +267,6 @@ sidebar_label: Apache Pulsar 3.0.0 - [Broker] Remove unnecessary variable in PersistentTopic by @poorbarcode in [#17511](https://github.com/apache/pulsar/pull/17511) - [Broker] Make cursor properties support modify single value concurrently. by @coderzc in [#17164](https://github.com/apache/pulsar/pull/17164) - [Broker] Do not log stacktrace for 'Failed to flush mark-delete position' case by @eolivelli in [#17432](https://github.com/apache/pulsar/pull/17432) -- [Broker] New bucket based delayed message tracker - interface&config&proto -part 1 by @coderzc in [#17344](https://github.com/apache/pulsar/pull/17344) - [Broker] Refactor reflection method in delayed and service module by @HQebupt in [#17347](https://github.com/apache/pulsar/pull/17347) - [Broker] Remove unnecessary parameters(reusefuture) and related logic by @Pomelongan in [#17378](https://github.com/apache/pulsar/pull/17378) - [Broker] Fix NPE when updating topic properties by @Flowermin in [#17352](https://github.com/apache/pulsar/pull/17352) @@ -428,7 +430,7 @@ sidebar_label: Apache Pulsar 3.0.0 ### Documentation -Here are the most noteworthy changes in this release. +Here are the most noteworthy changes in this release. - Add fresh new docs for Pulsar admin API [#504](https://github.com/apache/pulsar-site/pull/504), [#498](https://github.com/apache/pulsar-site/pull/498), [#493](https://github.com/apache/pulsar-site/pull/493), [#490](https://github.com/apache/pulsar-site/pull/490), [#471](https://github.com/apache/pulsar-site/pull/471), [#462](https://github.com/apache/pulsar-site/pull/462), [#403](https://github.com/apache/pulsar-site/pull/403) - Refactor the information architecture of client library docs [#393](https://github.com/apache/pulsar-site/pull/393) @@ -442,7 +444,7 @@ Here are the most noteworthy changes in this release. - Add docs for enabling replicated subscription [#510](https://github.com/apache/pulsar-site/pull/510) - Add docs for env variables (CLI) [#422](https://github.com/apache/pulsar-site/pull/422) - Add docs for schema support (Python functions) [#486](https://github.com/apache/pulsar-site/pull/486) -- Add docs for message dispatch throttling [#386](https://github.com/apache/pulsar-site/pull/386) +- Add docs for message dispatch throttling [#386](https://github.com/apache/pulsar-site/pull/386) - Add docs for proxy version protocol [#531](https://github.com/apache/pulsar-site/pull/531) For the complete list, including all features, enhancements, and bug fixes, check out the following resources.
