momo-jun commented on code in PR #670: URL: https://github.com/apache/pulsar-site/pull/670#discussion_r1286712149
########## release-notes/versioned/pulsar-3.1.0.md: ########## @@ -0,0 +1,282 @@ +--- +id: pulsar-3.1.0 +title: Apache Pulsar 3.1.0 +sidebar_label: Apache Pulsar 3.1.0 +--- + +#### 2023-08-10 + +### Important notice +- + +### PIPs +- PIP-275:Introduce topicOrderedExecutorThreadNum to deprecate numWorkerThreadsForNonPersistentTopic [#20504](https://github.com/apache/pulsar/pull/20504) +- PIP-255 Part-2: Make the partition assignment strategy pluggable [#20537](https://github.com/apache/pulsar/pull/20537) + +### Broker +- Fix MessageDeduplication throw NPE when enable broker dedup and set namespace disable deduplication. [#20905](https://github.com/apache/pulsar/pull/20905) +- In replication scenario, remote consumer could not be registered if there has no message was sent [#20888](https://github.com/apache/pulsar/pull/20888) +- Fix IndexOutOfBoundsException in the CompactedTopicUtils [#20887](https://github.com/apache/pulsar/pull/20887) +- Add annotation for topic compaction strategy [#20858](https://github.com/apache/pulsar/pull/20858) +- Fix inconsensus namespace policies by `getPoliciesIfCached` [#20855](https://github.com/apache/pulsar/pull/20855) +- Pass `bytesToRead` when reading compacted entries [#20850](https://github.com/apache/pulsar/pull/20850) +- Fix the built-in admin failed to delete a topic with a custom authz provider [#20848](https://github.com/apache/pulsar/pull/20848) +- Avoid print redirect exception log when get list from bundle [#20846](https://github.com/apache/pulsar/pull/20846) +- Inconsistent behaviour for topic auto_creation [#20843](https://github.com/apache/pulsar/pull/20843) +- Follow up 4196 use `PulsarByteBufAllocator` handle OOM [#20837](https://github.com/apache/pulsar/pull/20837) +- Add broker filter sync method back to guarantee the API compatibility [#20826](https://github.com/apache/pulsar/pull/20826) +- Fix direct memory leak by delayed index OutOfDirectMemory [#20823](https://github.com/apache/pulsar/pull/20823) +- Avoid infinite bundle unloading [#20822](https://github.com/apache/pulsar/pull/20822) +- Add the MessageExpirer interface to make code clear [#20800](https://github.com/apache/pulsar/pull/20800) +- Broker failed to load v1 namespace resources cache [#20783](https://github.com/apache/pulsar/pull/20783) +- Can not receive any messages after switch to standby cluster [#20767](https://github.com/apache/pulsar/pull/20767) +- Fix get topic policies as null during clean cache [#20763](https://github.com/apache/pulsar/pull/20763) +- Fix typo in ServerCnx.java [#20762](https://github.com/apache/pulsar/pull/20762) +- Avoid throwing RestException in BrokerService [#20761](https://github.com/apache/pulsar/pull/20761) +- Call ServerCnxcloseProducer from correct thread [#20747](https://github.com/apache/pulsar/pull/20747) +- Ignore openIDTokenIssuerTrustCertsFilePath conf when blank [#20745](https://github.com/apache/pulsar/pull/20745) +- Support pluggable topic compaction service - part2 [#20718](https://github.com/apache/pulsar/pull/20718) +- Revert "Skip loading broker interceptor when disableBrokerInterceptors is true 20422" [#20710](https://github.com/apache/pulsar/pull/20710) +- Gracefully shutdown does not work with admin cli in standalone [#20709](https://github.com/apache/pulsar/pull/20709) +- Improve CompactedTopicImpl lock [#20697](https://github.com/apache/pulsar/pull/20697) +- Added the skipped message handler for ServiceUnitStateChannel [#20677](https://github.com/apache/pulsar/pull/20677) +- Make ExtensibleLoadManagerImpl's broker filter pure async [#20666](https://github.com/apache/pulsar/pull/20666) +- Add log on topic or subscription when enable rate limit. [#20660](https://github.com/apache/pulsar/pull/20660) +- Make get list from bundle Admin API async [#20652](https://github.com/apache/pulsar/pull/20652) +- Fix the publish latency spike from the contention of MessageDeduplication [#20647](https://github.com/apache/pulsar/pull/20647) +- Support pluggable topic compaction service - part1 [#20645](https://github.com/apache/pulsar/pull/20645) +- Fix get owned service units NPE [#20625](https://github.com/apache/pulsar/pull/20625) +- Topic policy can not be work well if replay policy message has any exception. [#20613](https://github.com/apache/pulsar/pull/20613) +- Fix the publish latency spike issue with large number of producers [#20607](https://github.com/apache/pulsar/pull/20607) +- AuthenticationFilter.doFilter [#20598](https://github.com/apache/pulsar/pull/20598) +- Fix NPE when reset Replicator's cursor by position. [#20597](https://github.com/apache/pulsar/pull/20597) +- Add consumer-id into the log when doing subscribe. [#20568](https://github.com/apache/pulsar/pull/20568) +- Release orphan replicator after topic closed [#20567](https://github.com/apache/pulsar/pull/20567) +- New load balancer system topic should not be auto-created now [#20566](https://github.com/apache/pulsar/pull/20566) +- Remove unused cache executor in `PulsarService` [#20563](https://github.com/apache/pulsar/pull/20563) +- Handle get owned namespaces admin API in ExtensibleLoadManager [#20552](https://github.com/apache/pulsar/pull/20552) +- Handle heartbeat namespace in ExtensibleLoadManager [#20551](https://github.com/apache/pulsar/pull/20551) +- Validate authz earlier in delete subscription logic [#20549](https://github.com/apache/pulsar/pull/20549) +- Should not throw NotFoundException when metadata already exists. [#20539](https://github.com/apache/pulsar/pull/20539) +- REST Client Producer fails with TLS only [#20535](https://github.com/apache/pulsar/pull/20535) +- Fix redirect loop when using ExtensibleLoadManager and list in bundle admin API [#20528](https://github.com/apache/pulsar/pull/20528) +- Restore solution for certain topic unloading race conditions [#20527](https://github.com/apache/pulsar/pull/20527) +- Emit the namespace bundle listener event on extensible load manager [#20525](https://github.com/apache/pulsar/pull/20525) +- Choose random thread for consumerFlow in PersistentDispatcherSingleActiveConsumer [#20522](https://github.com/apache/pulsar/pull/20522) +- Disable EntryFilters for system topics [#20514](https://github.com/apache/pulsar/pull/20514) +- Support get/remove permissions for AuthorizationProvider [#20496](https://github.com/apache/pulsar/pull/20496) +- Improve efficiency of checking message deletion [#20490](https://github.com/apache/pulsar/pull/20490) +- Reformat property in generateResponseWithEntry [#20481](https://github.com/apache/pulsar/pull/20481) +- Add metric prefix for `topic_load_times` [#20472](https://github.com/apache/pulsar/pull/20472) +- Fix return the earliest position when query position by timestamp. [#20457](https://github.com/apache/pulsar/pull/20457) +- Support revoking permission for AuthorizationProvider [#20456](https://github.com/apache/pulsar/pull/20456) +- Replace String.intern() with Guava Interner [#20432](https://github.com/apache/pulsar/pull/20432) +- Remove uneffective solution for reducing GC pressure [#20428](https://github.com/apache/pulsar/pull/20428) +- Skip loading broker interceptor when disableBrokerInterceptors is true [#20422](https://github.com/apache/pulsar/pull/20422) +- Avoid `PersistentSubscription.expireMessages` logic check backlog twice. [#20416](https://github.com/apache/pulsar/pull/20416) +- Change limitStatsLogging config default value to true [#20409](https://github.com/apache/pulsar/pull/20409) +- PIP-255 Part-1: Add listener interface for namespace service [#20406](https://github.com/apache/pulsar/pull/20406) +- Use immutable base cursor properties [#20400](https://github.com/apache/pulsar/pull/20400) +- Fix partitioned __change_events topic is policy topic [#20392](https://github.com/apache/pulsar/pull/20392) +- Do not expose bucketDelayedIndexStats [#20383](https://github.com/apache/pulsar/pull/20383) +- Fix flaky test - testClusterMigrationWithReplica… [#20379](https://github.com/apache/pulsar/pull/20379) +- Change some static fields referencing mutable objects to ordinary instance fields in BrokerService [#20372](https://github.com/apache/pulsar/pull/20372) +- Fix pre-create non-partitioned system topics for load balance extension [#20370](https://github.com/apache/pulsar/pull/20370) +- Fix memory leak when rebatchMessage [#20369](https://github.com/apache/pulsar/pull/20369) +- Invalidate metadata children cache after key deleted [#20363](https://github.com/apache/pulsar/pull/20363) +- Fix managedLedger.getConfig().getProperties().putAll(properties) NPE [#20361](https://github.com/apache/pulsar/pull/20361) +- Update rest endpoint method names to avoid swagger conflict [#20359](https://github.com/apache/pulsar/pull/20359) +- Fix broker load manager class filter NPE [#20350](https://github.com/apache/pulsar/pull/20350) +- Copy BrokerEntryMetadata when rebatchMessage [#20337](https://github.com/apache/pulsar/pull/20337) +- In Key_Shared mode: remove unnecessary mechanisms of message skip to avoid unnecessary consumption stuck [#20335](https://github.com/apache/pulsar/pull/20335) +- When creating a namespace anti-affinity group with the default local policies, use the bundle size supplied by the user. [#20327](https://github.com/apache/pulsar/pull/20327) +- Fix skip message API when hole messages exists [#20326](https://github.com/apache/pulsar/pull/20326) +- Gracefully shut down load balancer extension [#20315](https://github.com/apache/pulsar/pull/20315) +- Fix class name typo `PrecisPublishLimiter` to "Precise" [#20310](https://github.com/apache/pulsar/pull/20310) +- Allow Access to System Topic Metadata for Reader Creation Post-Namespace Deletion [#20304](https://github.com/apache/pulsar/pull/20304) +- Fix NPE cause by topic publish rate limiter. [#20302](https://github.com/apache/pulsar/pull/20302) +- Remove Streaming Dispatcher Code [#20279](https://github.com/apache/pulsar/pull/20279) +- Fix `UnsupportedOperationException` when update topic properties. [#20261](https://github.com/apache/pulsar/pull/20261) +- Fix ledger cachemiss size metric [#20257](https://github.com/apache/pulsar/pull/20257) +- Fix default bundle size used while setting bookie affinity [#20250](https://github.com/apache/pulsar/pull/20250) +- Fix the behavior of delayed message in Key_Shared mode [#20233](https://github.com/apache/pulsar/pull/20233) +- Fix infinite ack of Replicator after topic is closed [#20232](https://github.com/apache/pulsar/pull/20232) +- Producer created by replicator is not displayed in topic stats [#20229](https://github.com/apache/pulsar/pull/20229) +- Use bulk operation instead of iteration [#20219](https://github.com/apache/pulsar/pull/20219) +- Fix Return value of getPartitionedStats doesn't contain subscription type [#20210](https://github.com/apache/pulsar/pull/20210) +- Improve knownBrokers update in ModularLoadManagerImpl [#20196](https://github.com/apache/pulsar/pull/20196) +- Skip split bundle if only one broker [#20190](https://github.com/apache/pulsar/pull/20190) +- Fix deadlock of metadata store [#20189](https://github.com/apache/pulsar/pull/20189) +- Fix `RoaringBitmap.contains` can't check value 65535 [#20176](https://github.com/apache/pulsar/pull/20176) +- Release EntryBuffer after parse proto object [#20170](https://github.com/apache/pulsar/pull/20170) +- Remove useless field in the DelayedMessageIndexBucketSegment.proto [#20166](https://github.com/apache/pulsar/pull/20166) +- Optimization protobuf code in the bucket delayed tracker [#20158](https://github.com/apache/pulsar/pull/20158) +- Move bitmap from lastMutableBucket to ImmutableBucket [#20156](https://github.com/apache/pulsar/pull/20156) +- Producer/Consumer should call allowTopicOperationAsync [#20142](https://github.com/apache/pulsar/pull/20142) +- Fix entry filter feature for the non-persistent topic [#20141](https://github.com/apache/pulsar/pull/20141) +- Optimize delayed metadata index bitmap [#20136](https://github.com/apache/pulsar/pull/20136) +- Fast fix infinite HTTP call getSubscriptions caused by wrong topicName [#20131](https://github.com/apache/pulsar/pull/20131) +- Make timer execute immediately after load index [#20126](https://github.com/apache/pulsar/pull/20126) +- Cache LedgerHandle in BookkeeperBucketSnapshotStorage [#20117](https://github.com/apache/pulsar/pull/20117) +- Pin AppendIndexMetadataInterceptor to field in `ManagedLedgerInterceptorImpl` [#20112](https://github.com/apache/pulsar/pull/20112) +- Close temporary open ledger in BookkeeperBucketSnapshotStorage [#20111](https://github.com/apache/pulsar/pull/20111) +- Fix use error TimeUnit to record publish latency [#20074](https://github.com/apache/pulsar/pull/20074) +- Fix the reason label of authentication metrics [#20030](https://github.com/apache/pulsar/pull/20030) +- Save createIfMissing in TopicLoadingContext [#19993](https://github.com/apache/pulsar/pull/19993) +- Log resource usage rate of brokers that need to be offloaded in ThresholdShedder [#19983](https://github.com/apache/pulsar/pull/19983) +- Fix getPartitionedStats miss subscription's messageAckRate [#19870](https://github.com/apache/pulsar/pull/19870) +- Fix wrong logic of method TopicName.getPartition(int index) [#19841](https://github.com/apache/pulsar/pull/19841) +- Fix `Update contains no change` error when use `--update-auth-data` flag to update function/sink/source [#19450](https://github.com/apache/pulsar/pull/19450) +- Add `topic_load_failed` metric [#19236](https://github.com/apache/pulsar/pull/19236) +- Support getStats/update partitioned topic with `-partition-` [#19235](https://github.com/apache/pulsar/pull/19235) +- Do not filter system topic while shedding. [#18949](https://github.com/apache/pulsar/pull/18949) +- If ledger lost, cursor mark delete position can not forward [#18620](https://github.com/apache/pulsar/pull/18620) +- Do not grant permission for each partition to reduce unnecessary zk metadata [#18222](https://github.com/apache/pulsar/pull/18222) +- Support cgroup v2 by using `jdk.internal.platform.Metrics` in Pulsar Loadbalancer [#16832](https://github.com/apache/pulsar/pull/16832) + +### Clients +- Disable polling pattern topics when TopicListWatcher is enabled. [#20779](https://github.com/apache/pulsar/pull/20779) +- fix negative message re-delivery twice issue [#20750](https://github.com/apache/pulsar/pull/20750) +- Fix subscribing pattern topics through Proxy not working [#20739](https://github.com/apache/pulsar/pull/20739) +- Messages lost when consumer reconnect [#20695](https://github.com/apache/pulsar/pull/20695) +- Fix deadlock issue of consumer while using multiple IO threads [#20669](https://github.com/apache/pulsar/pull/20669) +- Make the whole grabCnx() progress atomic [#20595](https://github.com/apache/pulsar/pull/20595) +- Swallow Conscrypt ClassNotFoundException [#20371](https://github.com/apache/pulsar/pull/20371) +- thread-safe seek [#20321](https://github.com/apache/pulsar/pull/20321) +- Document Java Client's Seek logic thread-safety improved in 20242 [#20284](https://github.com/apache/pulsar/pull/20284) +- Fix where the function getMsgNumInReceiverQueue always returns 0 when using message listener [#20245](https://github.com/apache/pulsar/pull/20245) +- Java Client's Seek Logic Not Threadsafe 1 [#20242](https://github.com/apache/pulsar/pull/20242) +- SchemaDefinition handle JSR310_CONVERSION_ENABLED property [#20201](https://github.com/apache/pulsar/pull/20201) +- Enable custom Encrypt/Decrypt Methods for ReaderImplementation. [#12599](https://github.com/apache/pulsar/pull/12599) + +### Pulsar IO and Pulsar Functions +- [IO] Not restart instance when kafka source poll exception. [#20795](https://github.com/apache/pulsar/pull/20795) +- [IO] Add notifyError method on PushSource. [#20791](https://github.com/apache/pulsar/pull/20791) +- [IO] elastic-search sink connector not support JSON.String schema. [#20741](https://github.com/apache/pulsar/pull/20741) +- [IO] Close the kafka source connector got stuck [#20698](https://github.com/apache/pulsar/pull/20698) +- [IO] Close the kafka source connector if there is uncaught exception [#20424](https://github.com/apache/pulsar/pull/20424) Review Comment: ```suggestion - [IO] Close the Kafka source connector if there is an uncaught exception [#20424](https://github.com/apache/pulsar/pull/20424) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
