momo-jun commented on code in PR #14398: URL: https://github.com/apache/pulsar/pull/14398#discussion_r848192263
########## site2/website/release-notes.md: ########## @@ -1,6 +1,301 @@ ## Apache Pulsar Release Notes +### 2.10.0 +#### 2022-04-13 + +### Important notice +- Remove -XX:-ResizePLAB JVM option which degrades performance on JDK11 [#12940](https://github.com/apache/pulsar/pull/12940) +- Enable TCP keepAlive flag on the sockets [#12982](https://github.com/apache/pulsar/pull/12982) +- Reduce the time it takes for namespace bundle unloading to time out [#12995](https://github.com/apache/pulsar/pull/12995) +- Align configurations defaults between default file and Java object (broker.conf, proxy.conf, websocket.conf) [#13272](https://github.com/apache/pulsar/pull/13272) +- [PIP 118] Do not restart brokers when ZooKeeper session expires as default [#13341](https://github.com/apache/pulsar/pull/13341) +- [PIP 119] Enabled consistent hashing by default on KeyShared subscriptions dispatcher [#13352](https://github.com/apache/pulsar/pull/13352) +- [PIP 120] Enabled client memory limit controller by default [#13344](https://github.com/apache/pulsar/pull/13344) +- Make Docker images non-root, by default, and OpenShift compliant [#13376](https://github.com/apache/pulsar/pull/13376) +- [PIP 122] Change loadBalancer default loadSheddingStrategy to ThresholdShedder [#13733](https://github.com/apache/pulsar/pull/13733) +- Fix netcat returning early for probe [#14088](https://github.com/apache/pulsar/pull/14088) + +### PIPs +- [PIP 79] Added lazy-loading feature to PartitionedProducer [#10279](https://github.com/apache/pulsar/pull/10279) +- [PIP 84] Pulsar client: Redeliver command add epoch [#10478](https://github.com/apache/pulsar/pull/10478) +- [PIP 86] Pulsar Functions: Preload and release external resources [#13205](https://github.com/apache/pulsar/pull/13205) +- [PIP 92] Topic policy across multiple clusters [#12517](https://github.com/apache/pulsar/pull/12517) +- [PIP 104] Add new consumer type: TableView [#12838](https://github.com/apache/pulsar/pull/12838) +- [PIP-105] Support pluggable entry filter in Dispatcher [#12869](https://github.com/apache/pulsar/pull/12869) [#12970](https://github.com/apache/pulsar/pull/12970) [#12979](https://github.com/apache/pulsar/pull/12979) +- [PIP 106] Broker extensions to provide operators of enterprise-wide clusters better control and flexibility [#12536](https://github.com/apache/pulsar/pull/12536) +- [PIP 107] Introduce chunk message ID [#12403](https://github.com/apache/pulsar/pull/12403) +- [PIP 110] Support Topic metadata - PART-1 create topic with properties [#12818](https://github.com/apache/pulsar/pull/12818) +- [PIP 121] Pulsar cluster level auto failover on client side [#13316](https://github.com/apache/pulsar/pull/13316) +- [PIP-124] Create init subscription before sending message to DLQ [#13355](https://github.com/apache/pulsar/pull/13355) +- [PIP-130] Apply redelivery backoff policy for ack timeout [#13707](https://github.com/apache/pulsar/pull/13707) +- [PIP 131] Resolve produce chunk messages failed when topic level maxMessageSize is set [#13599](https://github.com/apache/pulsar/pull/13599) +- [PIP 135] Include MetadataStore backend for Etcd [#13225](https://github.com/apache/pulsar/pull/13225) + +### Broker +- [PIP 45] Pluggable metadata interface + - Added BookKeeper metadata adapter based on MetadataStore [#12770](https://github.com/apache/pulsar/pull/12770) + - Add Rocksdb metadata store [#12776](https://github.com/apache/pulsar/pull/12776) + - Converted BookieRackAffinityMapping to use MetadataStore [#12841](https://github.com/apache/pulsar/pull/12841) + - Allow to configure metadata store URL in broker.conf [#13077](https://github.com/apache/pulsar/pull/13077) + - Removed old ZK caches implementations [#13075](https://github.com/apache/pulsar/pull/13075) + - Allow to start bookie with Pulsar metadata store backend [#13296](https://github.com/apache/pulsar/pull/13296) + - Removed ZooKeeperClientFactory [#13303](https://github.com/apache/pulsar/pull/13303) + - Use reference counting in RocksDB metadata store [#13309](https://github.com/apache/pulsar/pull/13309) + - Allow to configure metadata store URL in proxy.conf [#13777](https://github.com/apache/pulsar/pull/13777) + - Allow to configure metadata store URL in functions_worker.yml [#13782](https://github.com/apache/pulsar/pull/13782) + - Add configuration metadata store url setting in WebSocket [#13786](https://github.com/apache/pulsar/pull/13786) + - Allow configuring configurationMetadataStore when initialize cluster metadata [#13889](https://github.com/apache/pulsar/pull/13889) + - Using the consistent metadata store scheme name [#13937](https://github.com/apache/pulsar/pull/13937) + - Allow configuring `metadataStoreUrl` in `pulsar-perf managed-ledger` [#14145](https://github.com/apache/pulsar/pull/14145) + - Deprecated zookeeper settings [#14147](https://github.com/apache/pulsar/pull/14147) + - Fixed metadata cache inconsistency on doing refresh [#14283](https://github.com/apache/pulsar/pull/14283) +- Support broker level dispatch rate limiter [#11325](https://github.com/apache/pulsar/pull/11325) +- Support setting geo-replication clusters on topic level [#12136](https://github.com/apache/pulsar/pull/12136) +- Add Multi-tiered storage key to broker.conf [#12173](https://github.com/apache/pulsar/pull/12173) +- Support configuration to rate-limit dispatching on batch message [#12294](https://github.com/apache/pulsar/pull/12294) +- Support split the largest bundle of the namespace [#12361](https://github.com/apache/pulsar/pull/12361) +- Support create subscription by specifying the earliest or latest position [#12872](https://github.com/apache/pulsar/pull/12872) +- Support roll-over ledgers for inactive topics [#13073](https://github.com/apache/pulsar/pull/13073) +- Support graceful shutdown for Broker [#14114](https://github.com/apache/pulsar/pull/14114) +- Transparent batching of ZK operations [#13043](https://github.com/apache/pulsar/pull/13043) +- Add uniform load shedder strategy to distribute traffic uniformly across brokers [#12902](https://github.com/apache/pulsar/pull/12902) +- Provide option to split bundle based on load [#12378](https://github.com/apache/pulsar/pull/12378) +- Use AuthorizationService#grantPermissionAsync to grant topic permission [#12515](https://github.com/apache/pulsar/pull/12515) +- Only store authentication data after authentication is complete [#12077](https://github.com/apache/pulsar/pull/12077) +- Allow to have different instances LocalMemoryMetadataStore that share the same state [#12390](https://github.com/apache/pulsar/pull/12390) +- Allow `GetTopicsOfNamespace` op with `consume` permission [#12600](https://github.com/apache/pulsar/pull/12600) +- Add local filesystem backend for package manager [#12708](https://github.com/apache/pulsar/pull/12708) +- Add stop replicator producer logic when start replicator cluster failed [#12724](https://github.com/apache/pulsar/pull/12724) +- Apply PolicyHierarchyValue to inactiveTopicPolicies [#12687](https://github.com/apache/pulsar/pull/12687) +- Fix can not get leader broker in follower brokers [#11353](https://github.com/apache/pulsar/pull/11353) +- Fix broker gc log options [#11285](https://github.com/apache/pulsar/pull/11285) +- Fix NPE of ZkBookieRackAffinityMapping [#11947](https://github.com/apache/pulsar/pull/11947) +- Fix prefix setting in JWT authn and avoid multi calls for the getProperty [#12132](https://github.com/apache/pulsar/pull/12132) +- Fix missed check exit code of stop before calling start [#12368](https://github.com/apache/pulsar/pull/12368) +- Fix getting children of parent nodes in LocalMemoryMetadataStore [#12491](https://github.com/apache/pulsar/pull/12491) +- Fix collection get bug in ResourceGroupService [#12499](https://github.com/apache/pulsar/pull/12499) +- Fix deadlock in metadata-store callback thread [#12753](https://github.com/apache/pulsar/pull/12753) +- Improve exceptions thrown when handling the schema resource [#12155](https://github.com/apache/pulsar/pull/12155) +- Trim topic name [#12453](https://github.com/apache/pulsar/pull/12453) +- Avoid unnecessary recalculation of maxSubscriptionsPerTopic in AbstractTopic [#12658](https://github.com/apache/pulsar/pull/12658) +- Optimize isValidPath check in MetadataStore [#12663](https://github.com/apache/pulsar/pull/12663) +- Close connection after receiving unexpected SendCommand [#12780](https://github.com/apache/pulsar/pull/12780) +- Ensure cache is refreshed (and not just invalidated) after a store write [#12788](https://github.com/apache/pulsar/pull/12788) +- Optimize topicMaxMessageSize with topic local cache. [#12830](https://github.com/apache/pulsar/pull/12830) +- Optimize blocking backlogQuotaCheck to non-blocking in ServerCnx#handleProducer [#12874](https://github.com/apache/pulsar/pull/12874) +- Only refresh metadata if path is already in cache after write. [#12896](https://github.com/apache/pulsar/pull/12896) +- Optimize put and get methods in AbstractMetadataStore [#12916](https://github.com/apache/pulsar/pull/12916) +- Fix zk-node leak of admin path [#12972](https://github.com/apache/pulsar/pull/12972) +- Optimize TopicPolicy#deduplicationEnabled with HierarchyTopicPolicies [#13000](https://github.com/apache/pulsar/pull/13000) +- Fix race condition in FaultInjectionMetadataStore#programmedFailure [#13007](https://github.com/apache/pulsar/pull/13007) +- Fix usage of PULSAR_EXTRA_OPTS/BOOKIE_EXTRA_OPTS in startup scripts [#13025](https://github.com/apache/pulsar/pull/13025) +- Consider topics in pulsar/system namespace as system topics [#13050](https://github.com/apache/pulsar/pull/13050) +- Fix wrong result for looking up a non-exist topic by rest api [#13055](https://github.com/apache/pulsar/pull/13055) +- Key_Shared dispatcher with no connected consumers should be recreated if allowOutOfOrderDelivery changes [#13063](https://github.com/apache/pulsar/pull/13063) +- Make load-balancer config dynamic for the runtime tuning [#13074](https://github.com/apache/pulsar/pull/13074) +- Optimize TopicPolicy#maxProducersPerTopic with HierarchyTopicPolicies [#13082](https://github.com/apache/pulsar/pull/13082) +- Load balancer supports disabling max-session for bundle split [#13108](https://github.com/apache/pulsar/pull/13108) +- Optimize TopicPolicies#subscriptionTypesEnabled with HierarchyTopicPolicies [#13121](https://github.com/apache/pulsar/pull/13121) +- Log thread dump when Zookeeper session expires to help detect possible deadlocks [#13124](https://github.com/apache/pulsar/pull/13124) +- Add removeMaxConsumersPerSubscription method for v1 namespace [#13192](https://github.com/apache/pulsar/pull/13192) +- Fixe error value for 1MB in metrics [#13213](https://github.com/apache/pulsar/pull/13213) +- Optimize TopicPolicies#messageTTLInSeconds with HierarchyTopicPolicies [#13241](https://github.com/apache/pulsar/pull/13241) +- Add schema compatibility strategy on topic level [#13297](https://github.com/apache/pulsar/pull/13297) +- Optimize TopicPolicies#maxConsumerPerTopic with HierarchyTopicPolicies [#13361](https://github.com/apache/pulsar/pull/13361) +- Optimize TopicPolicies#replicationClusters with HierarchyTopicPolicies [#13495](https://github.com/apache/pulsar/pull/13495) +- Fix compatibility issue with other metadata store implementations in resources [#13394](https://github.com/apache/pulsar/pull/13394) +- Fix race condition in stopping replicator while it is starting [#13412](https://github.com/apache/pulsar/pull/13412) +- Support backlog quota across multiple clusters [#13445](https://github.com/apache/pulsar/pull/13445) +- Return null instead of RestException when get bookieAffinityGroup [#13462](https://github.com/apache/pulsar/pull/13462) +- Fix failed to get Partitioned metadata for health checking [#13525](https://github.com/apache/pulsar/pull/13525) +- Optimize TopicPolicies#maxConsumersPerSubscription with HierarchyTopicPolicies [#13548](https://github.com/apache/pulsar/pull/13548) +- Fix raw use of generic types in OwnershipCache [#13592](https://github.com/apache/pulsar/pull/13592) +- Optimize TopicPolicies#maxUnackedMessagesOnConsumer with HierarchyTopicPolicies [#13618](https://github.com/apache/pulsar/pull/13618) +- Optimize TopicPolicies#delayedDelivery Enabled and TickTimeMillis with HierarchyTopicPolicies [#13649](https://github.com/apache/pulsar/pull/13649) +- Optimize TopicPolicies#compactionThreshold with HierarchyTopicPolicies [#13710](https://github.com/apache/pulsar/pull/13710) +- Add deleteSubscriptionDispatchRate method for v1 namespace [#13711](https://github.com/apache/pulsar/pull/13711) +- Fix NPE of checkReplication [#13720](https://github.com/apache/pulsar/pull/13720) +- Optimize deduplicationSnapshotIntervalSeconds with HierarchyTopicPolicies [#13769](https://github.com/apache/pulsar/pull/13769) +- Use shared executors for broker and geo-replication clients [#13839](https://github.com/apache/pulsar/pull/13839) +- Fix call sync method in async rest api for + - for internalDeletePartitionedTopic [#13805](https://github.com/apache/pulsar/pull/13805) + - for internalCreateSubscription. [#13873](https://github.com/apache/pulsar/pull/13873) + - for internalGetBacklogSizeByMessageId [#13871](https://github.com/apache/pulsar/pull/13871) + - for internalTriggerCompaction [#13853](https://github.com/apache/pulsar/pull/13853) + - for internalGetManagedLedgerInfo [#13847](https://github.com/apache/pulsar/pull/13847) + - for internalGetSubscriptions [#13846](https://github.com/apache/pulsar/pull/13846) + - for internalUnloadTopic [#13845](https://github.com/apache/pulsar/pull/13845) + - for internalGetLastMessageId [#13882](https://github.com/apache/pulsar/pull/13882) + - for internalExpireMessagesByTimestamp [#13880](https://github.com/apache/pulsar/pull/13880) + - for internalResetCursorOnPosition [#13879](https://github.com/apache/pulsar/pull/13879) + - for internalExpireMessagesByPosition [#13878](https://github.com/apache/pulsar/pull/13878) + - for internalGetMessageById. [#13876](https://github.com/apache/pulsar/pull/13876) + - for internalSetReplicatedSubscriptionStatus [#13887](https://github.com/apache/pulsar/pull/13887) + - for internalGetPartitionedStats [#13886](https://github.com/apache/pulsar/pull/13886) + - for onPoliciesUpdate method [#13885](https://github.com/apache/pulsar/pull/13885) + - for internalGetPartitionedStatsInternal [#13884](https://github.com/apache/pulsar/pull/13884) + - for internalTerminatePartitionedTopic [#13890](https://github.com/apache/pulsar/pull/13890) + - for internalSkipAllMessages [#13901](https://github.com/apache/pulsar/pull/13901) + - for preValidation [#13962](https://github.com/apache/pulsar/pull/13962) + - for internalSetReplicationClusters and internalRemoveReplicationClusters [#13961](https://github.com/apache/pulsar/pull/13961) +- Make PulsarAuthorizationProvider#grantPermissionAsync actually async [#13897](https://github.com/apache/pulsar/pull/13897) +- Fix no value present when updating dispatch rate [#13900](https://github.com/apache/pulsar/pull/13900) +- Optimize retentionPolicies with HierarchyTopicPolicies [#13909](https://github.com/apache/pulsar/pull/13909) +- Improve 403 response message wih consume permission on namespace [#13912](https://github.com/apache/pulsar/pull/13912) +- Fix arithmetic exception for uniform load shedder [#13914](https://github.com/apache/pulsar/pull/13914) +- Add threshold for each unload round for uniform load shedder [#13915](https://github.com/apache/pulsar/pull/13915) +- Fix call sync method in async rest api for internalDeleteSubscription #13884 [#13917](https://github.com/apache/pulsar/pull/13917) +- Change ``BrokersBase`` api ``getActiveBrokers`` and ``getLeaderBroker`` to pure async. [#13935](https://github.com/apache/pulsar/pull/13935) +- Do not create missing topic when loading namespace [#13948](https://github.com/apache/pulsar/pull/13948) +- Only reply to client when code completes producerFuture [#13949](https://github.com/apache/pulsar/pull/13949) +- Improve error logging for topic not found [#13950](https://github.com/apache/pulsar/pull/13950) +- Make validateTenantOperation method async in PulsarWebResource [#14008](https://github.com/apache/pulsar/pull/14008) +- Optimize maxUnackedMessagesOnSubscription with HierarchyTopicPolicies [#14011](https://github.com/apache/pulsar/pull/14011) +- Set default value of applied to false in getSchemaCompatibilityStrategy [#14012](https://github.com/apache/pulsar/pull/14012) +- Make validateTopicPolicyOperation method async in PulsarWebResource [#14024](https://github.com/apache/pulsar/pull/14024) +- Fix producerFuture.completeExceptionally not called before sendErrorResponse [#14025](https://github.com/apache/pulsar/pull/14025) +- Make triggerOffload method async [#14027](https://github.com/apache/pulsar/pull/14027) +- Make offloadStatus method async [#14029](https://github.com/apache/pulsar/pull/14029) +- Make validateNamespacePolicyOperation method async in PulsarWebResource [#14033](https://github.com/apache/pulsar/pull/14033) +- Make internalSkipMessages method async [#14045](https://github.com/apache/pulsar/pull/14045) +- Make PersistentTopicsBase#internalSetBacklogQuota async [#14051](https://github.com/apache/pulsar/pull/14051) +- Adjust the validation for policy schemaCompatibilityStrategy [#14061](https://github.com/apache/pulsar/pull/14061) +- Remove Persistent Topics v3 API - use custom media type instead [#14117](https://github.com/apache/pulsar/pull/14117) +- Fix race condition in PulsarLedgerIdGenerator#generateShortLedgerId [#14118](https://github.com/apache/pulsar/pull/14118) +- Optimize topic policy with HierarchyTopicPolicies about subscriptionDispatchRate [#14151](https://github.com/apache/pulsar/pull/14151) +- Make `BrokerBase#deleteDynamicConfiguration` to pure async method [#14163](https://github.com/apache/pulsar/pull/14163) +- Set default value of applied to false on topic policy [#14181](https://github.com/apache/pulsar/pull/14181) +- Change broker producer fence log level [#14196](https://github.com/apache/pulsar/pull/14196) +- Remove duplicated filter for UniformLoadShedder#findBundlesForUnloading [#14198](https://github.com/apache/pulsar/pull/14198) +- Adjust topic exists check logic in http lookup process [#14199](https://github.com/apache/pulsar/pull/14199) +- Fix NPE of internalExpireMessagesByTimestamp [#14243](https://github.com/apache/pulsar/pull/14243) +- Fix rackaware placement policy does not take effect after delete rack configuration [#14248](https://github.com/apache/pulsar/pull/14248) +- Fix print error log when server return redirect (http code 307) [#14259](https://github.com/apache/pulsar/pull/14259) +- Optimize topic policy with HierarchyTopicPolicies about publishRate [#14267](https://github.com/apache/pulsar/pull/14267) +- Fix ack-hole and backlog for persistent-replicator [#14282](https://github.com/apache/pulsar/pull/14282) +- Fix NPE in internalSkipMessages [#14297](https://github.com/apache/pulsar/pull/14297) +- Validate blank advertised listener name [#14306](https://github.com/apache/pulsar/pull/14306) +- Let entries expire in the metadata caches [#14154](https://github.com/apache/pulsar/pull/14154) +- Fix transaction system topic loop creation [#12749](https://github.com/apache/pulsar/pull/12749) +- Fix topic transaction buffer handle null snapshot [#12758](https://github.com/apache/pulsar/pull/12758) +- Optimize changeToCloseState method [#14277](https://github.com/apache/pulsar/pull/14277) + +### Clients +- [Java] Support create a consumer in the paused state [#11974](https://github.com/apache/pulsar/pull/11974) +- [Java] Support passing existing executor providers to the client [#12037](https://github.com/apache/pulsar/pull/12037) +- [Java] Fix the producer OOM if got exception while add message to batch container [#12170](https://github.com/apache/pulsar/pull/12170) +- [Java] Allow to config client allocator out of memory policy [#12200](https://github.com/apache/pulsar/pull/12200) +- [Java] Support negative ack redelivery backoff [#12566](https://github.com/apache/pulsar/pull/12566) +- [Java] Fix confusing logs in UnAckedMessageTracker [#13017](https://github.com/apache/pulsar/pull/13017) +- [Java] Fix parseProtobufSchema method will be called two times [#13163](https://github.com/apache/pulsar/pull/13163) +- [Java] Add getNumPartitions method into PartitionedProducerImpl [#13239](https://github.com/apache/pulsar/pull/13239) +- [Java] Allow config client dns bind addr and port [#13390](https://github.com/apache/pulsar/pull/13390) +- [Java] Support add custom properties for the reconsumeLater interface [#13461](https://github.com/apache/pulsar/pull/13461) +- [Java] Allow Client Builder set Dnslookup params [#13503](https://github.com/apache/pulsar/pull/13503) +- [Java] Avoid repeatedly set startMessageIdData to null for ConsumerImpl [#13606](https://github.com/apache/pulsar/pull/13606) +- [Java] Let the 'properties' to be empty for ConsumerBuilder and ProducerBuilder [#14074](https://github.com/apache/pulsar/pull/14074) +- [Java] Log producer batchSize and msgSize percentiles [#14229](https://github.com/apache/pulsar/pull/14229) +- [C++] Add Wireshark cmake and fix build with latest Wireshark [#13236](https://github.com/apache/pulsar/pull/13236) +- [C++] Wireshark Pulsar dissector naming replace yahoo with apache [#13251](https://github.com/apache/pulsar/pull/13251) +- [C++] Support arm64 optimized CRC32c hardware-instructions [#13246](https://github.com/apache/pulsar/pull/13246) +- [C++] Support more pulsar command name and version in Wireshark dissector [#13286](https://github.com/apache/pulsar/pull/13286) Review Comment: ```suggestion - [C++] Support more pulsar command names and versions in Wireshark dissector [#13286](https://github.com/apache/pulsar/pull/13286) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
