[I] Metric for Producer Creation Failure [pulsar]

2025-03-11 Thread via GitHub
Meet0861 opened a new issue, #24070: URL: https://github.com/apache/pulsar/issues/24070 ### Search before asking - [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Motivation Having metric at the server side can h

Re: [PR] [improve][fn] Implement pip 401: Support set batching configurations for Pulsar Functions&Sources [pulsar]

2025-03-11 Thread via GitHub
nlu90 merged PR #23860: URL: https://github.com/apache/pulsar/pull/23860 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apac

(pulsar) branch master updated: [improve][fn] Implement pip 401: Support set batching configurations for Pulsar Functions&Sources (#23860)

2025-03-11 Thread nlu90
This is an automated email from the ASF dual-hosted git repository. nlu90 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new fad925f7c52 [improve][fn] Implement pip 401: Supp

Re: [PR] Fix: SendAsync callback was not invoked when producer is in reconnecting [pulsar-client-go]

2025-03-11 Thread via GitHub
gunli commented on PR #1333: URL: https://github.com/apache/pulsar-client-go/pull/1333#issuecomment-2716320971 > SendAsync can wait until the runEventLoop processes it and pushes it into the pendingQueue or a batch, just like the Java client. Before entering the pendingQueue, SendAsync its

Re: [PR] [fix][broker]Fix failed consumption after loaded up a terminated topic [pulsar]

2025-03-11 Thread via GitHub
codecov-commenter commented on PR #24063: URL: https://github.com/apache/pulsar/pull/24063#issuecomment-2716232302 ## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/24063?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments

(pulsar) branch master updated: [fix][test] Update partitioned topic subscription assertions in IncrementPartitionsTest (#24056)

2025-03-11 Thread hrsakai
This is an automated email from the ASF dual-hosted git repository. hrsakai pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 114aaf07f09 [fix][test] Update partitioned topi

Re: [PR] [fix][test] Update partitioned topic subscription assertions in IncrementPartitionsTest [pulsar]

2025-03-11 Thread via GitHub
hrsakai merged PR #24056: URL: https://github.com/apache/pulsar/pull/24056 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.ap

Re: [PR] [fix][client] Fix building broken batched message when publishing [pulsar]

2025-03-11 Thread via GitHub
poorbarcode commented on code in PR #24061: URL: https://github.com/apache/pulsar/pull/24061#discussion_r1984397528 ## pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java: ## @@ -2353,7 +2353,9 @@ private void batchMessageAndSend(boolean shouldScheduleNe

Re: [D] ✘ Container pulsar-init service "pulsar-init" didn't complete successfully: exit 127 [pulsar]

2025-03-11 Thread via GitHub
GitHub user yousrayk closed a discussion: ✘ Container pulsar-init service "pulsar-init" didn't complete successfully: exit 127 GitHub link: https://github.com/apache/pulsar/discussions/24068 This is an automatically sent email for commits@pulsar.apache.org. To unsubscribe, please send

Re: [D] ✘ Container pulsar-init service "pulsar-init" didn't complete successfully: exit 127 [pulsar]

2025-03-11 Thread via GitHub
GitHub user yousrayk closed a discussion: ✘ Container pulsar-init service "pulsar-init" didn't complete successfully: exit 127 GitHub link: https://github.com/apache/pulsar/discussions/24068 This is an automatically sent email for commits@pulsar.apache.org. To unsubscribe, please send

Re: [D] ✘ Container pulsar-init service "pulsar-init" didn't complete successfully: exit 127 [pulsar]

2025-03-11 Thread via GitHub
GitHub user yousrayk edited a discussion: ✘ Container pulsar-init service "pulsar-init" didn't complete successfully: exit 127 GitHub link: https://github.com/apache/pulsar/discussions/24068 This is an automatically sent email for commits@pulsar.apache.org. To unsubscribe, please send

Re: [PR] [fix][test] Update partitioned topic subscription assertions in IncrementPartitionsTest [pulsar]

2025-03-11 Thread via GitHub
codecov-commenter commented on PR #24056: URL: https://github.com/apache/pulsar/pull/24056#issuecomment-2715934901 ## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/24056?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments

Re: [I] Discarding Corrupt Message due to apparent decompression failure [pulsar-client-go]

2025-03-11 Thread via GitHub
RobertIndie commented on issue #1331: URL: https://github.com/apache/pulsar-client-go/issues/1331#issuecomment-2703744906 > we are using CompressionType.LZ4 That's strange. The LZ4 decompression doesn't seem to use the reader, so it shouldn't cause the EOF error. Maybe I missed some

Re: [PR] Fix: Potential data/write conflicts [pulsar-client-go]

2025-03-11 Thread via GitHub
crossoverJie merged PR #1336: URL: https://github.com/apache/pulsar-client-go/pull/1336 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

Re: [I] [Bug] Topic partition backlog grows after bookies restart when using ExtensibleLoadManagerImpl [pulsar]

2025-03-11 Thread via GitHub
heesung-sn commented on issue #23908: URL: https://github.com/apache/pulsar/issues/23908#issuecomment-2715763033 @szkoludasebastian I can do an online meeting with you to dive deep more on this. let me know your availability and email address to schedule this meeting, perhaps via goo

Re: [I] [Bug] Topic partition backlog grows after bookies restart when using ExtensibleLoadManagerImpl [pulsar]

2025-03-11 Thread via GitHub
szkoludasebastian commented on issue #23908: URL: https://github.com/apache/pulsar/issues/23908#issuecomment-2710803382 > Thank you for reporting this bug. Seems like there are multiple things to check to isolate the problem. Possible Theories: > > * consumers failed to reconnect

Re: [I] [Bug] "Not enough non-faulty bookies available error code: -6" persisting after outage while all bookies are reported as healthy [pulsar]

2025-03-11 Thread via GitHub
MichalKoziorowski-TomTom commented on issue #23807: URL: https://github.com/apache/pulsar/issues/23807#issuecomment-2710465502 Hi @mrtworo Have you found a fix for this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[I] Breaking change with using multiple schemas on a topic [pulsar-dotpulsar]

2025-03-11 Thread via GitHub
smbecker opened a new issue, #260: URL: https://github.com/apache/pulsar-dotpulsar/issues/260 ### Description I have a topic that was created using `NewConsumer(Schema.String)`. However, for retries and dead letters, I frequently create the producers using `NewProducer()` since I don

Re: [PR] Change Pulsar Proxy service load balancer type to ClusterIP [pulsar-helm-chart]

2025-03-11 Thread via GitHub
lhotari merged PR #588: URL: https://github.com/apache/pulsar-helm-chart/pull/588 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pu

Re: [PR] Fix segmentation fault when use token auth [pulsar-client-node]

2025-03-11 Thread via GitHub
doug8679 commented on PR #407: URL: https://github.com/apache/pulsar-client-node/pull/407#issuecomment-2715481099 @shibd what is the testing / release process and timing for an update to this in npm? We had this issue crop up in on of our services under active development and we have had

(pulsar-helm-chart) branch master updated: Add Broker Cache and Sockets dashboards (#586)

2025-03-11 Thread lhotari
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-helm-chart.git The following commit(s) were added to refs/heads/master by this push: new 601e78d Add Broker Cache and Socket

Re: [I] [Bug] Topic partition backlog grows after bookies restart when using ExtensibleLoadManagerImpl [pulsar]

2025-03-11 Thread via GitHub
szkoludasebastian commented on issue #23908: URL: https://github.com/apache/pulsar/issues/23908#issuecomment-2710423521 > From this stats, [#23908 (comment)](https://github.com/apache/pulsar/issues/23908#issuecomment-2626817353) > > I noticed that "unackedMessages" is non-zero. Seems

Re: [PR] [fix][broker] Typo correction: from backlogQuotaDefaultLimitByte to backlogQuotaDefaultLimitBytes [pulsar]

2025-03-11 Thread via GitHub
AnonHxy merged PR #24066: URL: https://github.com/apache/pulsar/pull/24066 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.ap

(pulsar) branch master updated: [fix][broker] Typo correction: from backlogQuotaDefaultLimitByte to backlogQuotaDefaultLimitBytes (#24066)

2025-03-11 Thread houxiaoyu
This is an automated email from the ASF dual-hosted git repository. houxiaoyu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new d7b34dce94b [fix][broker] Typo correction: fr

Re: [PR] [improve][client] PIP-407 Add newMessage with schema and transactions [pulsar]

2025-03-11 Thread via GitHub
BewareMyPower commented on code in PR #23942: URL: https://github.com/apache/pulsar/pull/23942#discussion_r1982686712 ## pulsar-client-api/src/main/java/org/apache/pulsar/client/api/Producer.java: ## @@ -132,6 +132,21 @@ public interface Producer extends Closeable { * @sin

Re: [PR] [fix] Avoid negative estimated entry count [pulsar]

2025-03-11 Thread via GitHub
gaoran10 commented on code in PR #24055: URL: https://github.com/apache/pulsar/pull/24055#discussion_r1983064036 ## managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java: ## @@ -3811,9 +3811,11 @@ public int applyMaxSizeCap(int maxEntries, long

Re: [PR] [fix] Avoid negative estimated entry count [pulsar]

2025-03-11 Thread via GitHub
gaoran10 commented on code in PR #24055: URL: https://github.com/apache/pulsar/pull/24055#discussion_r1983064036 ## managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java: ## @@ -3811,9 +3811,11 @@ public int applyMaxSizeCap(int maxEntries, long

Re: [I] Compatibility issue between Pulsar Client Python 3.6.0 and Pulsar 4.0.2 and Databricks [pulsar-client-python]

2025-03-11 Thread via GitHub
BewareMyPower commented on issue #246: URL: https://github.com/apache/pulsar-client-python/issues/246#issuecomment-2705311491 Could you try 3.6.1? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

(pulsar-helm-chart) branch master updated: Update RELEASE.md

2025-03-11 Thread lhotari
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-helm-chart.git The following commit(s) were added to refs/heads/master by this push: new 3f75320 Update RELEASE.md 3f75320 i

Re: [I] [Bug] Topic partition backlog grows after bookies restart when using ExtensibleLoadManagerImpl [pulsar]

2025-03-11 Thread via GitHub
szkoludasebastian commented on issue #23908: URL: https://github.com/apache/pulsar/issues/23908#issuecomment-2710416567 > [compose.yaml.txt](https://github.com/user-attachments/files/18877610/compose.yaml.txt) > > I tried to reproduce this issue by the above test setting, but I couldn

Re: [PR] Fix wrong result of reader.hasNext/Next after seeking by id or time [pulsar-client-go]

2025-03-11 Thread via GitHub
nodece commented on code in PR #1340: URL: https://github.com/apache/pulsar-client-go/pull/1340#discussion_r1986593581 ## pulsar/consumer_partition.go: ## @@ -2256,12 +2306,14 @@ func convertToMessageID(id *pb.MessageIdData) *trackingMessageID { msgID := &trackingMess

(pulsar-client-go) branch dependabot/go_modules/golang.org/x/net-0.33.0 deleted (was 1939d04b)

2025-03-11 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/golang.org/x/net-0.33.0 in repository https://gitbox.apache.org/repos/asf/pulsar-client-go.git was 1939d04b chore(deps): bump golang.org/x/net from 0.23.0 to 0.33.

Re: [PR] [fix][client] Fix building broken batched message when publishing [pulsar]

2025-03-11 Thread via GitHub
poorbarcode commented on code in PR #24061: URL: https://github.com/apache/pulsar/pull/24061#discussion_r1986539041 ## pulsar-client/src/main/java/org/apache/pulsar/client/impl/BatchMessageContainerImpl.java: ## @@ -162,13 +162,11 @@ protected ByteBuf getCompressedBatchMetadata

Re: [I] [Bug] Topic partition backlog grows after bookies restart when using ExtensibleLoadManagerImpl [pulsar]

2025-03-11 Thread via GitHub
heesung-sn commented on issue #23908: URL: https://github.com/apache/pulsar/issues/23908#issuecomment-2711252967 @szkoludasebastian Maybe, we can use google drive to share the heap dump? `heesung.s...@streamnative.io` can you try to update it and share with this account? I only see the gc

Re: [PR] opt: delete redundant channel connection.incomingCmdCh [pulsar-client-go]

2025-03-11 Thread via GitHub
BewareMyPower commented on code in PR #1343: URL: https://github.com/apache/pulsar-client-go/pull/1343#discussion_r1984387719 ## pulsar/internal/connection.go: ## @@ -534,7 +524,7 @@ func (c *connection) writeCommand(cmd *pb.BaseCommand) { } func (c *connection) receivedComm

Re: [PR] [improve][pip] PIP-406: Introduce metrics related to dispatch_throttled_count [pulsar]

2025-03-11 Thread via GitHub
shibd commented on code in PR #23945: URL: https://github.com/apache/pulsar/pull/23945#discussion_r1986242227 ## pip/pip-406.md: ## @@ -0,0 +1,152 @@ +# PIP-406: Introduce metrics related to dispatch_throttled_count + +# Background knowledge + +## Motivation + +Currently, users

Re: [PR] Fix wrong result of reader.hasNext/Next after seeking by id or time [pulsar-client-go]

2025-03-11 Thread via GitHub
nodece merged PR #1340: URL: https://github.com/apache/pulsar-client-go/pull/1340 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pu

svn commit: r75495 - /dev/pulsar/helm-chart/4.0.0-candidate-3/

2025-03-11 Thread lhotari
Author: lhotari Date: Mon Mar 10 23:51:17 2025 New Revision: 75495 Log: Add artifacts for Helm Chart 4.0.0-candidate-3 Added: dev/pulsar/helm-chart/4.0.0-candidate-3/index.yaml dev/pulsar/helm-chart/4.0.0-candidate-3/pulsar-4.0.0.tgz (with props) dev/pulsar/helm-chart/4.0.0-candidat

[PR] [improve][broker]clean up duplicate code and fix typo [pulsar]

2025-03-11 Thread via GitHub
3pacc opened a new pull request, #24069: URL: https://github.com/apache/pulsar/pull/24069 ### Motivation - the impliment code of getNumberOfConsumers in NonPersistentTopic and PersistentTopic are duplicate - there are some typo errors in OwnedBundle ### Modifications - mo

[I] Support for Key Shared Policy [pulsar-client-node]

2025-03-11 Thread via GitHub
iamseki opened a new issue, #408: URL: https://github.com/apache/pulsar-client-node/issues/408 **pulsar-client version:** 1.11.1 **Pulsar version:** 3.0.8 I've encountered an issue with a slow consumer in a key-shared subscription, causing new consumers to stall. This behavior coul

Re: [PR] [fix] [common] Fix RawMessageImpl.getProperties() failed when the message metadata contains the same key but with different values [pulsar]

2025-03-11 Thread via GitHub
horizonzy commented on PR #23927: URL: https://github.com/apache/pulsar/pull/23927#issuecomment-2713556232 > > If a message has duplicated keys in its property, I think we should allow the downstream to determine how to process such duplicated keys rather than retain a random value for such

Re: [PR] [fix] [common] Fix RawMessageImpl.getProperties() failed when the message metadata contains the same key but with different values [pulsar]

2025-03-11 Thread via GitHub
mattisonchao merged PR #23927: URL: https://github.com/apache/pulsar/pull/23927 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@puls

(pulsar) branch master updated: [fix] [common] Fix RawMessageImpl.getProperties() failed when the message metadata contains the same key but with different values (#23927)

2025-03-11 Thread mattisonchao
This is an automated email from the ASF dual-hosted git repository. mattisonchao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 47f32235245 [fix] [common] Fix RawMessageI

Re: [PR] [fix] [common] Fix RawMessageImpl.getProperties() failed when the message metadata contains the same key but with different values [pulsar]

2025-03-11 Thread via GitHub
mattisonchao commented on PR #23927: URL: https://github.com/apache/pulsar/pull/23927#issuecomment-2713953359 well, I can approve it since the `MessageImpl` side has a kind of same logic. https://github.com/apache/pulsar/pull/6390 umm. it seems like that was caused by proto2 not suppo

Re: [PR] [fix] [common] Fix RawMessageImpl.getProperties() failed when the message metadata contains the same key but with different values [pulsar]

2025-03-11 Thread via GitHub
BewareMyPower commented on PR #23927: URL: https://github.com/apache/pulsar/pull/23927#issuecomment-2713814794 That makes sense to me. I approved it because it solves the issue when duplicated keys are already written as the properties. Regarding the producer side to prevent duplicated keys

Re: [PR] [improve] Update deliverAfter and deliverAt api comment [pulsar-client-go]

2025-03-11 Thread via GitHub
RobertIndie commented on PR #1339: URL: https://github.com/apache/pulsar-client-go/pull/1339#issuecomment-2713659250 > @geniusjoe Great fix, I found no unit test for `SubscriptionType=KeyShared`, could you add one? +1 -- This is an automated message from the Apache Git Service. To

Re: [PR] [fix] [common] Fix RawMessageImpl.getProperties() failed when the message metadata contains the same key but with different values [pulsar]

2025-03-11 Thread via GitHub
BewareMyPower commented on PR #23927: URL: https://github.com/apache/pulsar/pull/23927#issuecomment-2713110226 If a message has duplicated keys in its property, I think we should allow the downstream to determine how to process such duplicated keys. For example, we can support throwing an e

Re: [PR] [improve][pip] PIP-406: Introduce metrics related to dispatch_throttled_count [pulsar]

2025-03-11 Thread via GitHub
lhotari commented on code in PR #23945: URL: https://github.com/apache/pulsar/pull/23945#discussion_r1988922335 ## pip/pip-406.md: ## @@ -0,0 +1,152 @@ +# PIP-406: Introduce metrics related to dispatch_throttled_count + +# Background knowledge + +## Motivation + +Currently, user

Re: [PR] [fix] [common] Fix RawMessageImpl.getProperties() failed when the message metadata contains the same key but with different values [pulsar]

2025-03-11 Thread via GitHub
horizonzy commented on PR #23927: URL: https://github.com/apache/pulsar/pull/23927#issuecomment-2713553884 > If a message has duplicated keys in its property, I think we should allow the downstream to determine how to process such duplicated keys rather than retain a random value for such k

Re: [PR] [improve][pip] PIP-406: Introduce metrics related to dispatch_throttled_count [pulsar]

2025-03-11 Thread via GitHub
RobertIndie commented on code in PR #23945: URL: https://github.com/apache/pulsar/pull/23945#discussion_r1988818723 ## pip/pip-406.md: ## @@ -0,0 +1,153 @@ +# PIP-406: Introduce metrics related to dispatch throttled number of times + +# Background knowledge + +## Motivation + +C

Re: [D] DNS [pulsar]

2025-03-11 Thread via GitHub
GitHub user sindhushreem25 added a comment to the discussion: DNS @lhotari i have shared the github gist in the slack channel with you . GitHub link: https://github.com/apache/pulsar/discussions/24030#discussioncomment-12459542 This is an automatically sent email for commits@pulsar.ap

(pulsar-dotpulsar) branch master updated: Updated NuGet packages

2025-03-11 Thread blankensteiner
This is an automated email from the ASF dual-hosted git repository. blankensteiner pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-dotpulsar.git The following commit(s) were added to refs/heads/master by this push: new e7add2c Updated NuGet package

Re: [PR] [fix] [common] Fix RawMessageImpl.getProperties() failed when the message metadata contains the same key but with different values [pulsar]

2025-03-11 Thread via GitHub
mattisonchao commented on PR #23927: URL: https://github.com/apache/pulsar/pull/23927#issuecomment-2713081937 Some questions: 1. why do the message properties allow duplicated keys? 2. If the message allows a duplicate key, then we remove one of them during reading. Is it some prop

Re: [I] [Bug] Topic partition backlog grows after bookies restart when using ExtensibleLoadManagerImpl [pulsar]

2025-03-11 Thread via GitHub
szkoludasebastian commented on issue #23908: URL: https://github.com/apache/pulsar/issues/23908#issuecomment-2713052616 I was able to get all broker logs which we have: [broker-sumo-logic-results.csv](https://github.com/user-attachments/files/19178370/broker-sumo-logic-results.csv)

Re: [PR] [fix] [common] Fix RawMessageImpl.getProperties() failed when the message metadata contains the same key but with different values [pulsar]

2025-03-11 Thread via GitHub
codecov-commenter commented on PR #23927: URL: https://github.com/apache/pulsar/pull/23927#issuecomment-2712927593 ## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/23927?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments