This is an automated email from the ASF dual-hosted git repository. technoboy pushed a commit to branch release-3.2.0 in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
commit a2dfc5a0881114da84d2b31968d87334185da877 Author: Jiwe Guo <[email protected]> AuthorDate: Thu Jan 4 12:01:32 2024 +0800 need to update function part --- release-notes/versioned/pulsar-3.2.0.md | 290 ++++++++++++++++++++++++++++++++ 1 file changed, 290 insertions(+) diff --git a/release-notes/versioned/pulsar-3.2.0.md b/release-notes/versioned/pulsar-3.2.0.md new file mode 100644 index 000000000000..a71980e8bb19 --- /dev/null +++ b/release-notes/versioned/pulsar-3.2.0.md @@ -0,0 +1,290 @@ +--- +id: pulsar-3.2.0 +title: Apache Pulsar 3.2.0 +sidebar_label: Apache Pulsar 3.2.0 +--- + +#### 2024-01-04 + +### Important notice +- + +### PIPs +- PIP-323: Complete Backlog Quota Telemetry [#21709](https://github.com/apache/pulsar/pull/21709) +- PIP-322: Pulsar Rate Limiting Refactoring [#21680](https://github.com/apache/pulsar/pull/21680) +- PIP-320: OpenTelemetry Scaffolding [#21635](https://github.com/apache/pulsar/pull/21635) +- PIP-318: Don't retain null-key messages during topic compaction [#21541](https://github.com/apache/pulsar/pull/21541) +- PIP-315: Configurable max delay limit for delayed delivery [#21490](https://github.com/apache/pulsar/pull/21490) +- PIP-313: Support force unsubscribe using consumer api [#21452](https://github.com/apache/pulsar/pull/21452) +- PIP-312: + - Use StateStoreProvider to manage state in Pulsar Functions endpoints [#21597](https://github.com/apache/pulsar/pull/21597) + - Use StateStoreProvider to manage state in Pulsar Functions endpoints [#21438](https://github.com/apache/pulsar/pull/21438) +- PIP-307: + - Support subscribing multi topics for WebSocket [#21390](https://github.com/apache/pulsar/pull/21390) + - Optimize Bundle Unload(Transfer) Protocol for ExtensibleLoadManager [#20748](https://github.com/apache/pulsar/pull/20748) +- PIP-305: Customize DNS servers to use for Pulsar Client [#21352](https://github.com/apache/pulsar/pull/21352) +- PIP-303: Add optional parameters for getPartitionedStats [#21228](https://github.com/apache/pulsar/pull/21228) +- PIP-302: Introduce refreshAsync API for TableView [#21271](https://github.com/apache/pulsar/pull/21271) +- PIP-301: Introduce LoadBalanceResources to unify the load-date CRUD [#21129](https://github.com/apache/pulsar/pull/21129) +- PIP-300: Add custom dynamic configuration for plugins [#21127](https://github.com/apache/pulsar/pull/21127) +- PIP-299: Stop dispatch messages if the individual acks will be lost in the persistent storage [#21118](https://github.com/apache/pulsar/pull/21118) +- PIP-297: Support terminating Function & Connector with the fatal exception [#21079](https://github.com/apache/pulsar/pull/21079) +- PIP-293: Delete config `disableBrokerInterceptors` [#20993](https://github.com/apache/pulsar/pull/20993) +- PIP-290: Provide a way to implement WSS E2E encryption and not need to expose the private key to the WebSocket Proxy [#20923](https://github.com/apache/pulsar/pull/20923) +- PIP-286: Make the TopicCompactionService to support find entry based on publishTime or index [#20867](https://github.com/apache/pulsar/pull/20867) +- PIP-277: Add `current` option in the Clusters list cmd [#20614](https://github.com/apache/pulsar/pull/20614) +- PIP-264: Enhanced OTel-based metric system [#21080](https://github.com/apache/pulsar/pull/21080) + +### Broker +- Fix returns wrong webServiceUrl when both webServicePort and webServicePortTls are set [#21842](https://github.com/apache/pulsar/pull/21842) +- Fix the ExtensibleLoadManagerImpl internal system getTopic failure when the leadership changes [#21764](https://github.com/apache/pulsar/pull/21764) +- Fix closing of AbstractDispatcherSingleActiveConsumer and reduce flakiness of test [#21736](https://github.com/apache/pulsar/pull/21736) +- Skip topic auto-creation for ExtensibleLoadManager internal topics [#21729](https://github.com/apache/pulsar/pull/21729) +- Fix ServiceUnitStateChannel monitor to tombstone only inactive bundle states [#21721](https://github.com/apache/pulsar/pull/21721) +- Avoid compaction task stuck when the last message to compact is a marker [#21718](https://github.com/apache/pulsar/pull/21718) +- Fix the issue of topics possibly being deleted. [#21704](https://github.com/apache/pulsar/pull/21704) +- CheckTopicExists supports checking partitioned topic without index [#21701](https://github.com/apache/pulsar/pull/21701) +- Remove internalGetListAsync and validatePartitionTopicUpdateAsync [#21699](https://github.com/apache/pulsar/pull/21699) +- Cleanup the empty subscriptionAuthenticationMap in zk when revoke subscription permission [#21696](https://github.com/apache/pulsar/pull/21696) +- Fix typo in the config key [#21690](https://github.com/apache/pulsar/pull/21690) +- network package lost if enable haProxyProtocolEnabled [#21684](https://github.com/apache/pulsar/pull/21684) +- Pulsar Rate Limiting Refactoring changes (PIP-322) [#21681](https://github.com/apache/pulsar/pull/21681) +- Record GeoPersistentReplicator.msgOut before producersendAsync [#21673](https://github.com/apache/pulsar/pull/21673) +- Print recoverBucketSnapshot log if cursorProperties are empty [#21651](https://github.com/apache/pulsar/pull/21651) +- Fix memory leak during topic compaction [#21647](https://github.com/apache/pulsar/pull/21647) +- Fix lookupRequestSemaphore leak when topic not found [#21646](https://github.com/apache/pulsar/pull/21646) +- Fix returns wrong webServiceUrl when both webServicePort and webServicePortTls are set [#21633](https://github.com/apache/pulsar/pull/21633) +- Fix getting incorrect KeyValue schema version [#21632](https://github.com/apache/pulsar/pull/21632) +- Add optional parameters for getPartitionedStats [#21611](https://github.com/apache/pulsar/pull/21611) +- PIP-301 Part-3: Add QuotaResources [#21596](https://github.com/apache/pulsar/pull/21596) +- Delete topic timeout due to NPE [#21595](https://github.com/apache/pulsar/pull/21595) +- Fix incorrect unack count when using shared subscription on non-persistent topic [#21592](https://github.com/apache/pulsar/pull/21592) +- Duplicate LedgerOffloader creation when namespace/topic… [#21591](https://github.com/apache/pulsar/pull/21591) +- Support not retaining null-key message during topic compaction [#21578](https://github.com/apache/pulsar/pull/21578) +- Correct schema deletion for partitioned topic [#21574](https://github.com/apache/pulsar/pull/21574) +- Fix create topic with different auto creation strategies causes race condition [#21545](https://github.com/apache/pulsar/pull/21545) +- Fix thousands orphan PersistentTopic caused OOM [#21540](https://github.com/apache/pulsar/pull/21540) +- Fix setReplicatedSubscriptionStatus incorrect behavior [#21510](https://github.com/apache/pulsar/pull/21510) +- Fix failure while creating non-durable cursor with inactive managed-ledger [#21508](https://github.com/apache/pulsar/pull/21508) +- Fix PulsarService/BrokerService shutdown when brokerShutdownTimeoutMs=0 [#21496](https://github.com/apache/pulsar/pull/21496) +- Do not write replicated snapshot marker when the topic which is not enable replication [#21495](https://github.com/apache/pulsar/pull/21495) +- Avoid pass null role in MultiRolesTokenAuthorizationProvider [#21486](https://github.com/apache/pulsar/pull/21486) +- Fix the deadlock when using BookieRackAffinityMapping with rackaware policy [#21481](https://github.com/apache/pulsar/pull/21481) +- Fix issue with consumer read uncommitted messages from compacted topic [#21465](https://github.com/apache/pulsar/pull/21465) +- Delete methods marked Lombok @Getter [#21462](https://github.com/apache/pulsar/pull/21462) +- Fix resource_quota_zpath [#21461](https://github.com/apache/pulsar/pull/21461) +- Fix namespace bundle stuck in unloading status [#21445](https://github.com/apache/pulsar/pull/21445) +- Ignore individual acknowledgment for CompactorSubscription when an entry has been filtered. [#21434](https://github.com/apache/pulsar/pull/21434) +- Fix MultiRoles token provider NPE when using anonymous clients [#21429](https://github.com/apache/pulsar/pull/21429) +- Allow broker deployment in heterogeneous hw config cluster without restricting nic speed detection [#21409](https://github.com/apache/pulsar/pull/21409) +- Fix avoid creating new topic after migration is started [#21368](https://github.com/apache/pulsar/pull/21368) +- Fix race condition of replication cluster connection during migration topic [#21364](https://github.com/apache/pulsar/pull/21364) +- Optimize and clean up aggregation of topic stats [#21361](https://github.com/apache/pulsar/pull/21361) +- Fix heartbeat namespace create event topic and cannot delete heartbeat topic [#21360](https://github.com/apache/pulsar/pull/21360) +- PIP-188 Fix cluster migration state store into local metadatastore [#21359](https://github.com/apache/pulsar/pull/21359) +- PIP-188 Add support to auto create topic resources into green cluster before migration [part-3] [#21354](https://github.com/apache/pulsar/pull/21354) +- Fix heartbeat namespace create transaction internal topic [#21348](https://github.com/apache/pulsar/pull/21348) +- Fix unload operation stuck when use ExtensibleLoadManager [#21332](https://github.com/apache/pulsar/pull/21332) +- Fix typos in Subscription interface [#21321](https://github.com/apache/pulsar/pull/21321) +- Use ConcurrentHashMap in ServiceUnitStateChannel and avoid recursive update error [#21282](https://github.com/apache/pulsar/pull/21282) +- Cleanup deprecated method of metadata store etcd [#21253](https://github.com/apache/pulsar/pull/21253) +- Miss headersAndPayload and messageIdData in MessagePublishContext [#21245](https://github.com/apache/pulsar/pull/21245) +- Improve error messages when updating partition count [#21243](https://github.com/apache/pulsar/pull/21243) +- Fixed produce and consume when anonymousUserRole enabled [#21237](https://github.com/apache/pulsar/pull/21237) +- Fix inconsistent topic policy [#21231](https://github.com/apache/pulsar/pull/21231) +- Fixed reset for AggregatedNamespaceStats [#21225](https://github.com/apache/pulsar/pull/21225) +- fix flaky test PatternTopicsConsumerImplTest [#21222](https://github.com/apache/pulsar/pull/21222) +- Let the producer request success at the first time if the previous one is inactive [#21220](https://github.com/apache/pulsar/pull/21220) +- Fix lookup heartbeat and sla namespace bundle when using extensible load manager [#21213](https://github.com/apache/pulsar/pull/21213) +- Not close the socket if lookup failed caused by bundle unloading or metadata ex [#21211](https://github.com/apache/pulsar/pull/21211) +- Make the TopicCompactionService to support find entry based on publishTime or index [#21208](https://github.com/apache/pulsar/pull/21208) +- Replicator leak when removeReplicator in NonPersistentTopic [#21205](https://github.com/apache/pulsar/pull/21205) +- Print warn log if ssl handshake error & print ledger id when switch ledger [#21201](https://github.com/apache/pulsar/pull/21201) +- Avoid record inactiveproducers when deduplication is disable. [#21193](https://github.com/apache/pulsar/pull/21193) +- Make the new exclusive consumer instead the inactive one faster [#21183](https://github.com/apache/pulsar/pull/21183) +- improve read entry error log for troubleshooting [#21169](https://github.com/apache/pulsar/pull/21169) +- Disable balancing based on DirectMemory. [#21168](https://github.com/apache/pulsar/pull/21168) +- Make specified producer could override the previous one [#21155](https://github.com/apache/pulsar/pull/21155) +- Upgrade bookkeeper to 4.16.3 [#21146](https://github.com/apache/pulsar/pull/21146) +- Improve logs for troubleshooting [#21141](https://github.com/apache/pulsar/pull/21141) +- Fix web tls url null cause NPE [#21137](https://github.com/apache/pulsar/pull/21137) +- Fix unack count when mixing non batch index and batch index acks [#21126](https://github.com/apache/pulsar/pull/21126) +- Fix unsubscribe non-durable subscription error [#21099](https://github.com/apache/pulsar/pull/21099) +- Fix isolated group not work problem. [#21096](https://github.com/apache/pulsar/pull/21096) +- Fix deleting topic not delete the related topic policy and schema. [#21093](https://github.com/apache/pulsar/pull/21093) +- revert remove duplicate topics name when deleteNamespace [#21087](https://github.com/apache/pulsar/pull/21087) +- Cleanup correctly heartbeat bundle ownership when handling broker deletion event [#21083](https://github.com/apache/pulsar/pull/21083) +- Fix write duplicate entries into the compacted ledger after RawReader reconnects [#21081](https://github.com/apache/pulsar/pull/21081) +- Remove bundle-data in local metadata store. [#21078](https://github.com/apache/pulsar/pull/21078) +- Fix typos: lister -> listener [#21068](https://github.com/apache/pulsar/pull/21068) +- Make sure all inflight writes have finished before completion of compaction [#21067](https://github.com/apache/pulsar/pull/21067) +- Make read compacted entries support maxReadSizeBytes limitation [#21065](https://github.com/apache/pulsar/pull/21065) +- Improve cache handling for partitioned topic metadata when doing lookup [#21063](https://github.com/apache/pulsar/pull/21063) +- Use MessageDigest.isEqual when comparing digests [#21061](https://github.com/apache/pulsar/pull/21061) +- Removing webURL check null [#21043](https://github.com/apache/pulsar/pull/21043) +- Fix potential case cause retention policy not working on topic level [#21041](https://github.com/apache/pulsar/pull/21041) +- Fix UniformLoadShedder selecet wrong overloadbroker and underloadbroker [#21025](https://github.com/apache/pulsar/pull/21025) +- Fix PulsarService.getLookupServiceAddress returns wrong port if TLS is enabled [#21015](https://github.com/apache/pulsar/pull/21015) +- PIP-293: Delete config disableBrokerInterceptors [#20995](https://github.com/apache/pulsar/pull/20995) +- Fix incorrect unack msk count when dup ack a message [#20990](https://github.com/apache/pulsar/pull/20990) +- Fix can't stop phase-two of compaction even though messageId read reaches lastReadId [#20988](https://github.com/apache/pulsar/pull/20988) +- Fix get outdated compactedTopicContext after compactionHorizon has been updated [#20984](https://github.com/apache/pulsar/pull/20984) +- Fix compaction subscription delete by inactive subscription check. [#20983](https://github.com/apache/pulsar/pull/20983) +- Fix message loss during topic compaction [#20980](https://github.com/apache/pulsar/pull/20980) +- Fix incorrect number of read compacted entries [#20978](https://github.com/apache/pulsar/pull/20978) +- Producer is blocked on creation because backlog exceeded on topic, when dedup is enabled and no producer is there [#20951](https://github.com/apache/pulsar/pull/20951) +- Fix http error message of Bookie rack placement config API when bookie-rack is not configured. [#20949](https://github.com/apache/pulsar/pull/20949) +- Fix chunked messages will be filtered by duplicating [#20948](https://github.com/apache/pulsar/pull/20948) +- Rackaware policy is ineffective when delete zk rack info after bkclient initialize [#20944](https://github.com/apache/pulsar/pull/20944) +- Check that the super user role is in the MultiRolesTokenAuthorizationProvider plugin [#20939](https://github.com/apache/pulsar/pull/20939) +- Fix consumers are not redirected to migrated cluster [#20928](https://github.com/apache/pulsar/pull/20928) +- PIP-300: Add custom dynamic configuration for plugins [#20884](https://github.com/apache/pulsar/pull/20884) +- Delete unused param [#20866](https://github.com/apache/pulsar/pull/20866) +- Improve performance of `StatsOutputStream` by using `ArrayDeque` instead of `Stack` [#20808](https://github.com/apache/pulsar/pull/20808) +- Fix missing generate some metrics in BrokerOperabilityMetrics [#20720](https://github.com/apache/pulsar/pull/20720) +- Fix ModularLoadManagerImpl always delete active bundle-data. sec ver. [#20620](https://github.com/apache/pulsar/pull/20620) +- Consider iowait as idle. [#19110](https://github.com/apache/pulsar/pull/19110) +- Fix bug caused by optimistic locking [#18390](https://github.com/apache/pulsar/pull/18390) + +### Pulsar IO and Pulsar Functions +- [IO] Not restart instance when kafka source poll exception. [#20795](https://github.com/apache/pulsar/pull/20795) +- [IO] Add notifyError method on PushSource. [#20791](https://github.com/apache/pulsar/pull/20791) +- [IO] elastic-search sink connector not support JSON.String schema [#20741](https://github.com/apache/pulsar/pull/20741) +- [IO] Close the Kafka source connector got stuck [#20698](https://github.com/apache/pulsar/pull/20698) +- [IO] Close the Kafka source connector if there is uncaught exception [#20424](https://github.com/apache/pulsar/pull/20424) +- [IO] add protobuf ByteString to pulsar-io jdbc core [#20259](https://github.com/apache/pulsar/pull/20259) +- [IO] KCA: handle kafka sources that use commitRecord [#20121](https://github.com/apache/pulsar/pull/20121) +- [IO] Fix the wrong format of the logs [#20907](https://github.com/apache/pulsar/pull/20907) +- [IO] Remove useless configuration for Pulsar SQL [#20605](https://github.com/apache/pulsar/pull/20605) +- [IO] Some optimized points in 9224 [#19027](https://github.com/apache/pulsar/pull/19027) +- [FN] Exit JVM when the main thread throws exceptions [#20689](https://github.com/apache/pulsar/pull/20689) +- [FN] TLS args admin download command use zero arity [#20513](https://github.com/apache/pulsar/pull/20513) +- [FN] Optimize string concatenation in user metrics gen [#20499](https://github.com/apache/pulsar/pull/20499) +- [FN] Support customizing TLS config for function download command [#20482](https://github.com/apache/pulsar/pull/20482) +- [FN] enable Go function token auth and TLS [#20468](https://github.com/apache/pulsar/pull/20468) +- [FN] Reset idle timer correctly [#20450](https://github.com/apache/pulsar/pull/20450) +- [FN] Add stateStorageURL and pulsarWebService URL to go InstanceConfig [#20443](https://github.com/apache/pulsar/pull/20443) +- [FN] Go functions must retrieve consumers by non-partitioned topic ID [#20413](https://github.com/apache/pulsar/pull/20413) +- [FN] Go functions need to use static grpcPort in k8s runtime [#20404](https://github.com/apache/pulsar/pull/20404) +- [FN] Correct TLS cert config translation from broker to fn worker [#20297](https://github.com/apache/pulsar/pull/20297) +- [FN] JavaInstanceStarter --tls_allow_insecure default to false [#20267](https://github.com/apache/pulsar/pull/20267) +- [FN] Allow unknown fields in connectors config [#20116](https://github.com/apache/pulsar/pull/20116) +- [FN] Use functions classloader in TopicSchema.newSchemaInstance() to fix ClassNotFoundException when using custom SerDe classes. (targeted for master) [#20115](https://github.com/apache/pulsar/pull/20115) +- [FN] Upgrade Kotlin version from 1.4.32 to 1.8.20. (targeted for master) [#20114](https://github.com/apache/pulsar/pull/20114) +- [FN] Check user metric len before iterating [#20021](https://github.com/apache/pulsar/pull/20021) +- [FN] Support multiple input topics for Go runtime [#20000](https://github.com/apache/pulsar/pull/20000) +- [FN] Make pulsar-admin support update py/go with package url [#19897](https://github.com/apache/pulsar/pull/19897) +- [FN] Fix JavaInstanceStarter inferring type class name error [#19896](https://github.com/apache/pulsar/pull/19896) +- [FN] Fix function update error [#19895](https://github.com/apache/pulsar/pull/19895) +- [FN] Make KubernetesRuntime translate characters in function tenant, namespace, and name during function removal to avoid label errors [#19584](https://github.com/apache/pulsar/pull/19584) + + +### Observability +- [Metrics] Add JVM start time metric [#20381](https://github.com/apache/pulsar/pull/20381) +- [Metrics] Topic with double quote breaks the prometheus format [#20230](https://github.com/apache/pulsar/pull/20230) + + +### CLI +- [Admin] Set ns level backlog quota does not take effect if retention exists [#20690](https://github.com/apache/pulsar/pull/20690) +- [Admin] Remove duplicate topics name when `deleteNamespace` [#20683](https://github.com/apache/pulsar/pull/20683) +- [Admin] Make admin.cli throw exception instead of silent catch [#20530](https://github.com/apache/pulsar/pull/20530) +- [Admin] Pretty print bookies racks-placement command output [#20516](https://github.com/apache/pulsar/pull/20516) +- [Admin] Make failed `bin/pulsar-admin source` command exit with code `1 (failed)` instead of `0 (success)` [#20503](https://github.com/apache/pulsar/pull/20503) +- [Admin] Using AuthorizationService to revoke the permissions [#20478](https://github.com/apache/pulsar/pull/20478) +- [Admin] Return BAD_REQUEST on cluster data is null for createCluster [#20346](https://github.com/apache/pulsar/pull/20346) +- [Admin] Enhancing Transaction Buffer Stats and Introducing TransactionBufferInternalStats API [#20330](https://github.com/apache/pulsar/pull/20330) +- [Admin] Fix examine messages if total message is zero [#20152](https://github.com/apache/pulsar/pull/20152) +- [Admin] Make subscriptions on NonPersistentTopic non-durable [#19741](https://github.com/apache/pulsar/pull/19741) +- [Admin] Report earliest msg in partitioned backlog [#19465](https://github.com/apache/pulsar/pull/19465) +- [CLI] Fix the variable producerName of BatchMsgContainer is null [#20819](https://github.com/apache/pulsar/pull/20819) +- [CLI] Fix error messages in `CmdNameSpaces` and `CmdTopics` [#20701](https://github.com/apache/pulsar/pull/20701) +- [CLI] Dedicated docs tools [#20700](https://github.com/apache/pulsar/pull/20700) +- [CLI] pulsar-client produce: allow to produce AVRO message from strings [#20654](https://github.com/apache/pulsar/pull/20654) +- [CLI] Client: allow to specify KeyValue's schema key (Avro Key support) [#20447](https://github.com/apache/pulsar/pull/20447) +- [CLI] Fulfill add-opens to function-localrunner also [#20417](https://github.com/apache/pulsar/pull/20417) +- [CLI] Allow pulser-client consume create a replicated subscription [#20316](https://github.com/apache/pulsar/pull/20316) +- [CLI] Add `--cleanupSubscription` to pulsar-admin [#20028](https://github.com/apache/pulsar/pull/20028) +- [CLI] Fix logging noise while admin tool exit [#19884](https://github.com/apache/pulsar/pull/19884) +- [CLI] PerformanceProducer do not produce expected number of messages [#19775](https://github.com/apache/pulsar/pull/19775) +- [CLI] Pulsar-perf: refactor to reduce code duplication [#19279](https://github.com/apache/pulsar/pull/19279) + +### Others +- [TXN] Add log when can't send TcClientConnectRequest to broker [#20880](https://github.com/apache/pulsar/pull/20880) +- [TXN] Use PulsarResource check for topic existence instead of brokerservice.getTopic() [#20569](https://github.com/apache/pulsar/pull/20569) +- [TXN] Unwrap the completion exception. [#20396](https://github.com/apache/pulsar/pull/20396) +- [TXN] Implement compatibility for transaction buffer segmented snapshot feature upgrade [#20235](https://github.com/apache/pulsar/pull/20235) +- [TXN] Fix transaction is not aborted when send or ACK failed [#20055](https://github.com/apache/pulsar/pull/20055) +- [TXN] fix consumer can receive aborted txn message when readType is replay [#19815](https://github.com/apache/pulsar/pull/19815) +- [ML] cleanup numManagedLedgerWorkerThreads [#20454](https://github.com/apache/pulsar/pull/20454) +- [ML] Fix ledger left in OPEN state when enabling `inactiveLedgerRollOverTimeMs` [#20276](https://github.com/apache/pulsar/pull/20276) +- [ML] There are two same-named managed ledgers in the one broker [#18688](https://github.com/apache/pulsar/pull/18688) +- [META] Bookie Info lost by notification race condition. [#20642](https://github.com/apache/pulsar/pull/20642) +- [META] Adding the missed bookie id in the registration manager [#20641](https://github.com/apache/pulsar/pull/20641) +- [META] Upgrade to jetcd to 0.7.5 [#20339](https://github.com/apache/pulsar/pull/20339) +- [META] Switch to the metadata store thread after zk operation [#20303](https://github.com/apache/pulsar/pull/20303) +- [META] deadlock of zkSessionWatcher when zkConnection loss [#20122](https://github.com/apache/pulsar/pull/20122) +- [META] Support set metadata size threshold for compression [#19561](https://github.com/apache/pulsar/pull/19561) +- [WS] Add cryptoKeyReaderFactoryClassName into the file websocket.conf [#20840](https://github.com/apache/pulsar/pull/20840) +- [WS] Remove unnecessary ping/pong implementation [#20733](https://github.com/apache/pulsar/pull/20733) +- [WS] Use allowTopicOperationAsync for authz checks [#20299](https://github.com/apache/pulsar/pull/20299) +- [WS] Use async auth method to support OIDC [#20238](https://github.com/apache/pulsar/pull/20238) +- [MISC] Stop logging full stack trace when `SecurityUtility.loadConscryptProvider()` fails to find Conscrypt with an unknown error [#20705](https://github.com/apache/pulsar/pull/20705) +- [MISC] Fix typo in pip template [#20556](https://github.com/apache/pulsar/pull/20556) +- [MISC] Use ubuntu 22.04 for Pulsar images [#20475](https://github.com/apache/pulsar/pull/20475) +- [MISC] Upgrade Netty to 4.1.93.Final [#20423](https://github.com/apache/pulsar/pull/20423) +- [misc] Consolidate certs in broker (and some proxy) tests [#20353](https://github.com/apache/pulsar/pull/20353) +- [MISC] Catch TimeoutException when logging about time outs [#20349](https://github.com/apache/pulsar/pull/20349) +- [Schema] Only handle exceptions when there has [#20730](https://github.com/apache/pulsar/pull/20730) +- [Schema] Add admin cli for testCompatibility [#19974](https://github.com/apache/pulsar/pull/19974) +- [Build] Fix generating javadoc issue [#20922](https://github.com/apache/pulsar/pull/20922) +- [build] Do not bundle swagger dependencies [#20687](https://github.com/apache/pulsar/pull/20687) +- [Build] Build function-go in golang image [#20665](https://github.com/apache/pulsar/pull/20665) +- [Build] Configure git-commit-id-plugin to skip git describe [#20550](https://github.com/apache/pulsar/pull/20550) +- [Build] Fix the pulsar-all image may use the wrong upstream image [#20435](https://github.com/apache/pulsar/pull/20435) +- [Build] Fix publish image script [#20305](https://github.com/apache/pulsar/pull/20305) +- [Build] Capture local build scans on ge.apache.org to benefit from deep build insights [#20187](https://github.com/apache/pulsar/pull/20187) + +### Library updates +- Upgrade Guava to 32.1.1 to address CVE-2023-2976 [#20699](https://github.com/apache/pulsar/pull/20699) +- Upgrade snappy-java to address multiple CVEs [#20604](https://github.com/apache/pulsar/pull/20604) +- Upgrade Jetty to address CVEs [#20461](https://github.com/apache/pulsar/pull/20461) +- Suppress etcd CVE warnings [#20342](https://github.com/apache/pulsar/pull/20342) +- Upgrade PyYaml version to 6.0.1 [#20831](https://github.com/apache/pulsar/pull/20831) +- Upgrade jackson version to 2.15.0 for CVE-2022-1471 [#20177](https://github.com/apache/pulsar/pull/20177) +- Upgrade prometheus client_golang to v1.12.2 to fix CVE-2022-21698 [#20579](https://github.com/apache/pulsar/pull/20579) +- Upgrade Netty to 4.1.93.Final [#20423](https://github.com/apache/pulsar/pull/20423) +- Upgrade to jetcd to 0.7.5 [#20339](https://github.com/apache/pulsar/pull/20339) +- Upgrade Kotlin version from 1.4.32 to 1.8.20. (targeted for master) [#20114](https://github.com/apache/pulsar/pull/20114) + +### Documentation +- [feat][doc] add overview for broker load balancing [#621](https://github.com/apache/pulsar-site/pull/621) +- [feat][doc] add concepts for broker load balancing [#644](https://github.com/apache/pulsar-site/pull/644) +- [feat][doc] add use cases + features + benefits + types for broker load balancing [#652](https://github.com/apache/pulsar-site/pull/652) +- [feat][doc] add Quick Start for broker load balancing [#660](https://github.com/apache/pulsar-site/pull/660) +- [feat][doc] add migration tutorials for broker load balancing [#657](https://github.com/apache/pulsar-site/pull/657) +- [feat][doc] add metrics for broker load balancing [#646](https://github.com/apache/pulsar-site/pull/646) +- [feat][doc] add configurations for broker load balancing [#661](https://github.com/apache/pulsar-site/pull/661) +- [feat][doc] add development docs for broker load balancing [#662](https://github.com/apache/pulsar-site/pull/662) +- [improve][doc] clarify detailed behaviors for failover subscription type [#633](https://github.com/apache/pulsar-site/pull/633/files) +- [improve][doc] add docs for supporting reading config options from files in Python runner [#544](https://github.com/apache/pulsar-site/pull/544) +- [improve][doc] add solutions for debezium-source-mysql connection errors [#649](https://github.com/apache/pulsar-site/pull/649) +- [feat][doc] add docs for OpenID Connect Auth Provider [#555](https://github.com/apache/pulsar-site/pull/555) +- [feat][doc] add docs for openIDKeyIdCacheMissRefreshSeconds setting [#580](https://github.com/apache/pulsar-site/pull/580) +- [feat][doc] add docs for KubernetesServiceAccountTokenAuthProvider [#570](https://github.com/apache/pulsar-site/pull/570/) +- [feat][doc] add client matrix page [#513](https://github.com/apache/pulsar-site/pull/513) +- [feature][doc] add docs for client memory limits [#571](https://github.com/apache/pulsar-site/pull/571) +- [feature][doc] add docs for CPP client table view [#565](https://github.com/apache/pulsar-site/pull/565) +- [improve] [doc] add compatibility reminders for retry queues [#651](https://github.com/apache/pulsar-site/pull/651) +- [feat][doc] add docs for JVM metrics in Pulsar broker [#567](https://github.com/apache/pulsar-site/pull/567) +- [feat][doc] add docs for bucket delayed message tracker metrics [#564](https://github.com/apache/pulsar-site/pull/564) +- [fix][doc] exclude outdated versions of API documentation from search engine results [#623](https://github.com/apache/pulsar-site/pull/623) +- [feat][website] implement 'How does Pulsar work' homepage screen [#614](https://github.com/apache/pulsar-site/pull/614) + + +For the complete list, including all features, enhancements, and bug fixes, check out the following resources. + +- [Documentation merged in 3.1 (pulsar-site repo)](https://github.com/apache/pulsar-site/pulls?q=is%3Apr+is%3Amerged+milestone%3A3.1.0+) + + +> A special thanks to the following contributors (alphabetic Github IDs) who contributed to Pulsar 3.1.0: +> Technoboy-, crossoverJie, lifepuzzlefun, poorbarcode, coderzc, liangyepianzhou, Demogorgon314, mattisonchao, BewareMyPower, yaalsn, zymap, lhotari, daziz, shibd, rdhabalia, eltociear, tisonkun, aloyszhang, michaeljmarshall, 315157973, JooHyukKim, zhanghaou, heesung-sn, codelipenghui, eolivelli, ethqunzhong, massakam, StevenLuMT, gaoran10, ericsyh, zuobiao-zhou, AnonHxy, xiaotongwang1, maanders-tibco, yuruguo, vineeth1995, flowchartsman, hanmz, nicoloboschi, Shawyeok, RobertIndie, nodec [...]
