[GitHub] [pulsar] michaeljmarshall commented on pull request #20151: [fix][broker] Fix auth test cases.
michaeljmarshall commented on PR #20151: URL: https://github.com/apache/pulsar/pull/20151#issuecomment-1543369427 That behavior only affects the WebSocket Proxy when subscription auth mode by prefix is in use. The test was added a while ago https://github.com/apache/pulsar/pull/899. I think we should migrate away from using those methods and then we can deprecate them. I don't see a need to change the behavior. Here is a PR to remove some of the final usages of the `canConsume` and `canProduce` methods https://github.com/apache/pulsar/pull/20299. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall opened a new pull request, #20299: [improve][ws] Use allowTopicOperationAsync for authz checks
michaeljmarshall opened a new pull request, #20299: URL: https://github.com/apache/pulsar/pull/20299 Similar to: https://github.com/apache/pulsar/pull/20142 ### Motivation In https://github.com/apache/pulsar/pull/20142 we changed the `Consumer` and the `Producer` logic to call the correct `AuthorizationService` method. Our goal is to deprecate the `AuthorizationService` methods for `canProduce` and `canConsume`, so this change helps us move in the right direction. This PR follows the same logic and updates the WebSocket proxy to remove all calls to the `can*` methods in the `AuthorizationService` ### Modifications * Update `ProducerHandler`, `ConsumerHandler`, and `ReaderHander` in the WebSocket Proxy to call the `AuthorizationService#allowTopicOperationAsync` method. ### Verifying this change This change is trivial. ### Documentation - [x] `doc-not-needed` ### Matching PR in forked repository PR in forked repository: Skipping PR as I ran tests locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on pull request #19026: [fix][proxy] Update clientAuthData in ProxyConnection as needed
michaeljmarshall commented on PR #19026: URL: https://github.com/apache/pulsar/pull/19026#issuecomment-1543351202 Superseded by https://github.com/apache/pulsar/pull/20067. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall closed pull request #19026: [fix][proxy] Update clientAuthData in ProxyConnection as needed
michaeljmarshall closed pull request #19026: [fix][proxy] Update clientAuthData in ProxyConnection as needed URL: https://github.com/apache/pulsar/pull/19026 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall opened a new issue, #20298: Flaky-test: MultiTopicsReaderTest.testMultiTopic
michaeljmarshall opened a new issue, #20298: URL: https://github.com/apache/pulsar/issues/20298 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Example failure https://github.com/apache/pulsar/actions/runs/4944000364/jobs/8839164318?pr=20297#step:10:732 ### Exception stacktrace ``` Error: Tests run: 18, Failures: 1, Errors: 0, Skipped: 10, Time elapsed: 135.535 s <<< FAILURE! - in org.apache.pulsar.client.impl.MultiTopicsReaderTest Error: org.apache.pulsar.client.impl.MultiTopicsReaderTest.stateCheck[public void org.apache.pulsar.client.impl.MultiTopicsReaderTest.testMultiTopic() throws java.lang.Exception](0) Time elapsed: 81.829 s <<< FAILURE! org.apache.pulsar.client.admin.PulsarAdminException$ConflictException: Cluster already exists at org.apache.pulsar.client.admin.PulsarAdminException.wrap(PulsarAdminException.java:252) at org.apache.pulsar.client.admin.internal.BaseResource.sync(BaseResource.java:352) at org.apache.pulsar.client.admin.internal.ClustersImpl.createCluster(ClustersImpl.java:78) at org.apache.pulsar.client.impl.MultiTopicsReaderTest.setup(MultiTopicsReaderTest.java:79) at org.apache.pulsar.tests.TestRetrySupport.stateCheck(TestRetrySupport.java:60) at jdk.internal.reflect.GeneratedMethodAccessor113.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139) at org.testng.internal.invokers.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:69) at org.testng.internal.invokers.ConfigInvoker.invokeConfigurationMethod(ConfigInvoker.java:361) at org.testng.internal.invokers.ConfigInvoker.invokeConfigurations(ConfigInvoker.java:296) at org.testng.internal.invokers.TestInvoker.runConfigMethods(TestInvoker.java:823) at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:590) at org.testng.internal.invokers.TestInvoker.retryFailed(TestInvoker.java:263) at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:62) at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:969) at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:194) at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148) at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at org.testng.TestRunner.privateRun(TestRunner.java:829) at org.testng.TestRunner.run(TestRunner.java:602) at org.testng.SuiteRunner.runTest(SuiteRunner.java:437) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:431) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:391) at org.testng.SuiteRunner.run(SuiteRunner.java:330) at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95) at org.testng.TestNG.runSuitesSequentially(TestNG.java:1256) at org.testng.TestNG.runSuitesLocally(TestNG.java:1176) at org.testng.TestNG.runSuites(TestNG.java:1099) at org.testng.TestNG.run(TestNG.java:1067) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:155) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:102) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeLazy(TestNGDirectoryTestSuite.java:117) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:86) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:137) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Suppressed: org.apache.pulsar.client.admin.PulsarAdminException$ConflictException: Cluster already exists at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:287) at org.apache.pulsar.client.admin.internal.BaseResource$1.failed(BaseResource.java:136) at
[GitHub] [pulsar] lifepuzzlefun commented on a diff in pull request #20276: [fix][ml] Fix ledger left in OPEN state when enable `inactiveLedgerRollOverTimeMs`
lifepuzzlefun commented on code in PR #20276:
URL: https://github.com/apache/pulsar/pull/20276#discussion_r1190629105
##
managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java:
##
@@ -4353,7 +4353,21 @@ public void checkInactiveLedgerAndRollOver() {
long currentTimeMs = System.currentTimeMillis();
if (inactiveLedgerRollOverTimeMs > 0 && currentTimeMs >
(lastAddEntryTimeMs + inactiveLedgerRollOverTimeMs)) {
log.info("[{}] Closing inactive ledger, last-add entry {}", name,
lastAddEntryTimeMs);
-ledgerClosed(currentLedger);
+if (STATE_UPDATER.compareAndSet(this, State.LedgerOpened,
State.ClosingLedger)) {
+currentLedger.asyncClose((rc, lh, o) -> {
Review Comment:
you are right. fixed has been applied.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] michaeljmarshall opened a new pull request, #20297: [fix][fn] Correct TLS cert config translation from broker to fn worker
michaeljmarshall opened a new pull request, #20297: URL: https://github.com/apache/pulsar/pull/20297 ### Motivation The `initializeWorkerConfigFromBrokerConfig` method converts a `ServiceConfiguration` object to a `WorkerConfig`. This is used when the function worker is running within the broker and when pulsar is running in standalone mode. The TLS certificates that are trusted by the broker should also be trusted by the function worker, and the TLS certificates that are trusted by the broker client should be trusted by the function worker's client. ### Modifications * Improve the `PulsarFunctionTlsTest` test by adding awaitility. Before this change, the test was flaky on my machine. * Fix the mapping of broker config to function worker config. ### Verifying this change A test is added. Note that the old test doesn't technically fail due to the misconfiguration at the moment. The error is in the logs. Here is one of the stack traces that is gone after this change: ``` 2023-05-10T23:19:27,087 - WARN - [pulsar-client-io-253-3:ClientCnx@344] - [localhost/127.0.0.1:59715] Got exception io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: General OpenSslEngine problem at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:499) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: javax.net.ssl.SSLHandshakeException: General OpenSslEngine problem at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.handshakeException(ReferenceCountedOpenSslEngine.java:1907) at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.wrap(ReferenceCountedOpenSslEngine.java:834) at java.base/javax.net.ssl.SSLEngine.wrap(SSLEngine.java:564) at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:1042) at io.netty.handler.ssl.SslHandler.wrapNonAppData(SslHandler.java:928) at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1418) at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1256) at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1296) at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) ... 17 more Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439) at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306) at java.base/sun.security.validator.Validator.validate(Validator.java:264) at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:285) at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:144) at io.netty.handler.ssl.ReferenceCountedOpenSslClientContext$ExtendedTrustManagerVerifyCallback.verify(ReferenceCountedOpenSslClientContext.java:234) at
[GitHub] [pulsar] lhotari commented on a diff in pull request #20276: [fix][ml] Fix ledger left in OPEN state when enable `inactiveLedgerRollOverTimeMs`
lhotari commented on code in PR #20276:
URL: https://github.com/apache/pulsar/pull/20276#discussion_r1190616415
##
managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java:
##
@@ -4353,7 +4353,21 @@ public void checkInactiveLedgerAndRollOver() {
long currentTimeMs = System.currentTimeMillis();
if (inactiveLedgerRollOverTimeMs > 0 && currentTimeMs >
(lastAddEntryTimeMs + inactiveLedgerRollOverTimeMs)) {
log.info("[{}] Closing inactive ledger, last-add entry {}", name,
lastAddEntryTimeMs);
-ledgerClosed(currentLedger);
+if (STATE_UPDATER.compareAndSet(this, State.LedgerOpened,
State.ClosingLedger)) {
+currentLedger.asyncClose((rc, lh, o) -> {
Review Comment:
please extract a local variable for `currentLedger` since there could be a
race condition where a new ledger might be created while the ledger is being
closed. Opening the new ledger would update the `currentLedger` field and this
would make the callback code to fail.
```suggestion
LedgerHandle currentLedger = this.currentLedger;
currentLedger.asyncClose((rc, lh, o) -> {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] crossoverJie commented on issue #20262: [Bug] When using the admin.topics().skipMessages API, the actual number of skipped messages may be less than the expected number when the valu
crossoverJie commented on issue #20262: URL: https://github.com/apache/pulsar/issues/20262#issuecomment-1543304177 @rdhabalia This issue is related to this [PR](https://github.com/apache/pulsar/pull/3819/files#diff-fad355f91bd15cc041161f9a46fce62b7fee87fbfb8f0ff8a8b724a1bd1f29eeR1287-R1324), please take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated (96367e1c2bf -> b07f2a9b85a)
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git from 96367e1c2bf [cleanup] Deduplicate test certificates to simplify management (#20289) add b07f2a9b85a [fix][test] Fix flaky test ConnectionPoolTest.testSetProxyToTargetBrokerAddress (#20293) No new revisions were added by this update. Summary of changes: .../pulsar/client/impl/ConnectionPoolTest.java | 35 +++--- 1 file changed, 18 insertions(+), 17 deletions(-)
[GitHub] [pulsar] lhotari merged pull request #20293: [fix][test] Fix flaky test ConnectionPoolTest.testSetProxyToTargetBrokerAddress
lhotari merged PR #20293: URL: https://github.com/apache/pulsar/pull/20293 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lhotari closed issue #20290: Flaky-test: ConnectionPoolTest.testSetProxyToTargetBrokerAddress
lhotari closed issue #20290: Flaky-test: ConnectionPoolTest.testSetProxyToTargetBrokerAddress URL: https://github.com/apache/pulsar/issues/20290 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] codecov-commenter commented on pull request #20293: [fix][test] Fix flaky test ConnectionPoolTest.testSetProxyToTargetBrokerAddress
codecov-commenter commented on PR #20293: URL: https://github.com/apache/pulsar/pull/20293#issuecomment-1543280059 ## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/20293?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#20293](https://app.codecov.io/gh/apache/pulsar/pull/20293?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (4685df3) into [master](https://app.codecov.io/gh/apache/pulsar/commit/ea56197d18d8d1f5b06c26246d920cb736d4e6a6?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (ea56197) will **increase** coverage by `35.35%`. > The diff coverage is `n/a`. [](https://app.codecov.io/gh/apache/pulsar/pull/20293?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #20293 +/- ## = + Coverage 37.57% 72.93% +35.35% - Complexity1260231961+19359 = Files 1691 1868 +177 Lines129033 138601 +9568 Branches 1406815240 +1172 = + Hits 48490 101089+52599 + Misses7422629469-44757 - Partials 6317 8043 +1726 ``` | Flag | Coverage Δ | | |---|---|---| | inttests | `24.05% <ø> (-0.04%)` | :arrow_down: | | systests | `24.81% <ø> (+0.05%)` | :arrow_up: | | unittests | `72.23% <ø> (+39.00%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. [see 1430 files with indirect coverage changes](https://app.codecov.io/gh/apache/pulsar/pull/20293/indirect-changes?src=pr=tree-more_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] crossoverJie commented on pull request #20245: [fix][client] Fix where the function getMsgNumInReceiverQueue always returns 0 when using message listener
crossoverJie commented on PR #20245: URL: https://github.com/apache/pulsar/pull/20245#issuecomment-1543277911 @tisonkun @Technoboy- PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar-site] branch add_195_3.0_note updated (e1538eb0875 -> 21ee9624562)
This is an automated email from the ASF dual-hosted git repository. zhaocong pushed a change to branch add_195_3.0_note in repository https://gitbox.apache.org/repos/asf/pulsar-site.git from e1538eb0875 Update release-notes/versioned/pulsar-3.0.0.md add 21ee9624562 Update release-notes/versioned/pulsar-3.0.0.md No new revisions were added by this update. Summary of changes: release-notes/versioned/pulsar-3.0.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[GitHub] [pulsar] lhotari commented on pull request #20276: [fix][ml] Fix ledger left in OPEN state when enable `inactiveLedgerRollOverTimeMs`
lhotari commented on PR #20276: URL: https://github.com/apache/pulsar/pull/20276#issuecomment-1543254471 > https://github.com/lifepuzzlefun/pulsar/pull/18 all test pass in my local repo. some of them fail on update cov file. +1, please update this link to the PR description so that it can be found there. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lifepuzzlefun commented on pull request #19993: [improve][broker] Save createIfMissing in TopicLoadingContext
lifepuzzlefun commented on PR #19993: URL: https://github.com/apache/pulsar/pull/19993#issuecomment-1543246928 > @lifepuzzlefun Could you write a test? yeah, i'll add one later : - ) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lifepuzzlefun commented on pull request #20276: [fix][ml] Fix ledger left in OPEN state when enable `inactiveLedgerRollOverTimeMs`
lifepuzzlefun commented on PR #20276: URL: https://github.com/apache/pulsar/pull/20276#issuecomment-1543246602 https://github.com/lifepuzzlefun/pulsar/pull/18 all test pass in my local repo. some of them fail on update cov file. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lordcheng10 commented on pull request #20292: [feat][broker] introduce precise broker publish rate limiting
lordcheng10 commented on PR #20292: URL: https://github.com/apache/pulsar/pull/20292#issuecomment-1543237883 @Jason918 PTAL,thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on issue #20291: PIP-251 Enhancing Transaction Buffer Stats and Introducing TransactionBufferInternalStats API
liangyepianzhou commented on issue #20291: URL: https://github.com/apache/pulsar/issues/20291#issuecomment-1543237730 >this PIP Number seems to be already in use, see: https://github.com/apache/pulsar/issues?q=is%3Aissue+is%3Aopen+PIP+251 This is the same one. I have close that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] liangyepianzhou closed issue #19628: PIP-251: Add snapshot stats in TransactionBufferStats
liangyepianzhou closed issue #19628: PIP-251: Add snapshot stats in TransactionBufferStats URL: https://github.com/apache/pulsar/issues/19628 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lordcheng10 commented on pull request #20292: [feat][broker] introduce precise broker publish rate limiting
lordcheng10 commented on PR #20292: URL: https://github.com/apache/pulsar/pull/20292#issuecomment-1543231936 @codelipenghui PTAL，thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] liangyepianzhou closed pull request #20235: [fix][txn] Implement compatibility for transaction buffer segmented snapshot feature upgrade
liangyepianzhou closed pull request #20235: [fix][txn] Implement compatibility for transaction buffer segmented snapshot feature upgrade URL: https://github.com/apache/pulsar/pull/20235 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] Anonymitaet commented on a diff in pull request #565: Add CPP client table view doc.
Anonymitaet commented on code in PR #565:
URL: https://github.com/apache/pulsar-site/pull/565#discussion_r1190565077
##
docs/client-libraries-tableviews.md:
##
@@ -4,36 +4,106 @@ title: Work with TableView
sidebar_label: "Work with TableView"
---
+mdx-code-block
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+
After setting up your clients, you can explore more to start working with
[TableView](concepts-clients.md#tableview).
## Configure TableView
-The following is an example of how to configure a TableView.
-```java
-TableView tv = client.newTableViewBuilder(Schema.STRING)
- .topic("my-tableview")
- .create()
-```
+mdx-code-block
+
+
+
+ The following is an example of how to configure a TableView.
+
+ ```java
+TableView tv = client.newTableViewBuilder(Schema.STRING)
+.topic("my-tableview")
+.create()
+ ```
+
+You can use the available parameters in the `loadConf` configuration or the
API
[`TableViewBuilder`](/api/client/@pulsar:version_number@/org/apache/pulsar/client/api/TableViewBuilder.html)
to customize your TableView.
Review Comment:
Have you previewed the API link?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-site] Anonymitaet commented on pull request #542: Add doc for available-functions and functions reload
Anonymitaet commented on PR #542: URL: https://github.com/apache/pulsar-site/pull/542#issuecomment-1543200551 Thanks for your contribution! Have you previewed your changes and ensured everything goes as expected? If not, please [preview your changes locally](https://pulsar.apache.org/contribute/document-preview/#preview-changes) and attach the screenshots in this PR. In this way, you can get your PR merged more quickly. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar-site] branch main updated: Docs sync done from apache/pulsar (#96367e1)
This is an automated email from the ASF dual-hosted git repository. urfree pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new 3751ad1d35e Docs sync done from apache/pulsar (#96367e1) 3751ad1d35e is described below commit 3751ad1d35e9a9182cc00f85627528e57fcbb1ab Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> AuthorDate: Thu May 11 01:55:55 2023 + Docs sync done from apache/pulsar (#96367e1) --- static/reference/next/config/reference-configuration-broker.md | 2 +- static/reference/next/config/reference-configuration-standalone.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/static/reference/next/config/reference-configuration-broker.md b/static/reference/next/config/reference-configuration-broker.md index 66b2d32a722..bf77a08a0bb 100644 --- a/static/reference/next/config/reference-configuration-broker.md +++ b/static/reference/next/config/reference-configuration-broker.md @@ -2979,7 +2979,7 @@ Enable cluster's failure-domain which can distribute brokers into logical region **Category**: Server ### haProxyProtocolEnabled -Enable or disable the proxy protocol. +Enable or disable the proxy protocol. If true, the real IP addresses of consumers and producers can be obtained when getting topic statistics data. **Type**: `boolean` diff --git a/static/reference/next/config/reference-configuration-standalone.md b/static/reference/next/config/reference-configuration-standalone.md index 66b2d32a722..bf77a08a0bb 100644 --- a/static/reference/next/config/reference-configuration-standalone.md +++ b/static/reference/next/config/reference-configuration-standalone.md @@ -2979,7 +2979,7 @@ Enable cluster's failure-domain which can distribute brokers into logical region **Category**: Server ### haProxyProtocolEnabled -Enable or disable the proxy protocol. +Enable or disable the proxy protocol. If true, the real IP addresses of consumers and producers can be obtained when getting topic statistics data. **Type**: `boolean`
[GitHub] [pulsar] github-actions[bot] commented on pull request #17846: [feature][compact] Enhanced topic compression
github-actions[bot] commented on PR #17846: URL: https://github.com/apache/pulsar/pull/17846#issuecomment-1543181869 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #17964: [improvement][pulsar-broker] Admin API should be able to return messages in human-readable formats.
github-actions[bot] commented on PR #17964: URL: https://github.com/apache/pulsar/pull/17964#issuecomment-1543181716 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #18007: [fix][broker] Reduce unnecessary persistence of markdelete
github-actions[bot] commented on PR #18007: URL: https://github.com/apache/pulsar/pull/18007#issuecomment-1543181570 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #18263: [improve][broker] allow client skips the non-continued deleted ranges stats
github-actions[bot] commented on PR #18263: URL: https://github.com/apache/pulsar/pull/18263#issuecomment-1543180912 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #18981: [improve][client]PIP-230 Throw exception when MessageIdImpl and BatchMessageIdImpl compare with each other
github-actions[bot] commented on PR #18981: URL: https://github.com/apache/pulsar/pull/18981#issuecomment-1543179277 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #19026: [fix][proxy] Update clientAuthData in ProxyConnection as needed
github-actions[bot] commented on PR #19026: URL: https://github.com/apache/pulsar/pull/19026#issuecomment-1543179131 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #19344: [improve] [ml] Reduce the lock time when calculating backlog size
github-actions[bot] commented on PR #19344: URL: https://github.com/apache/pulsar/pull/19344#issuecomment-1543177959 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #19348: [fix][broker] fix incorrect unack stats when dup ack a message
github-actions[bot] commented on PR #19348: URL: https://github.com/apache/pulsar/pull/19348#issuecomment-1543177856 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #19465: [fix][admin] Report earliest msg in partitioned backlog
github-actions[bot] commented on PR #19465: URL: https://github.com/apache/pulsar/pull/19465#issuecomment-1543177383 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #19498: [fix][client] fix receive dumplicated messages after seek to a timestamp in MultiTo…
github-actions[bot] commented on PR #19498: URL: https://github.com/apache/pulsar/pull/19498#issuecomment-1543177159 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #19501: [fix][broker] unify topic-level policies enable judgment conditions
github-actions[bot] commented on PR #19501: URL: https://github.com/apache/pulsar/pull/19501#issuecomment-1543177040 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #19561: [improve][meta] Support set metadata size threshold for compression.
github-actions[bot] commented on PR #19561: URL: https://github.com/apache/pulsar/pull/19561#issuecomment-1543176804 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #19619: [impr][broker] Reduce string allocations for stats generation
github-actions[bot] commented on PR #19619: URL: https://github.com/apache/pulsar/pull/19619#issuecomment-1543176568 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #19626: [fix][broker] Only send AuthChallenge when client supports it
github-actions[bot] commented on PR #19626: URL: https://github.com/apache/pulsar/pull/19626#issuecomment-1543176459 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #19702: [improve] [broker] Support expose metadataStore zookeeper client metrics.
github-actions[bot] commented on PR #19702: URL: https://github.com/apache/pulsar/pull/19702#issuecomment-1543176193 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #19783: [improve][offloaders] Automatically evict Offloaded Ledgers from memory
github-actions[bot] commented on PR #19783: URL: https://github.com/apache/pulsar/pull/19783#issuecomment-1543175851 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #19797: [cleanup][proxy] Remove authorization config
github-actions[bot] commented on PR #19797: URL: https://github.com/apache/pulsar/pull/19797#issuecomment-1543175701 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #19883: [improve][cli] topic offline internal-info provide schema ledger info
github-actions[bot] commented on PR #19883: URL: https://github.com/apache/pulsar/pull/19883#issuecomment-1543175482 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #20036: [Doc] color code
github-actions[bot] commented on issue #20036: URL: https://github.com/apache/pulsar/issues/20036#issuecomment-1543175077 The issue had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on pull request #20261: [fix][broker] Fix `UnsupportedOperationException` when update topic properties.
liangyepianzhou commented on PR #20261: URL: https://github.com/apache/pulsar/pull/20261#issuecomment-1543004989 >Why remove these labels? @poorbarcode Because this [PR](https://github.com/apache/pulsar/pull/17711) didn’t been cherry-picked. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on pull request #20261: [fix][broker] Fix `UnsupportedOperationException` when update topic properties.
liangyepianzhou commented on PR #20261: URL: https://github.com/apache/pulsar/pull/20261#issuecomment-1543000448 >@liangyepianzhou - would you mind sharing the stack trace for the UnsupportedOperationException? Thanks! @michaeljmarshall Sure, of course. ```shell Message: java.util.concurrent.CompletionException: java.lang.UnsupportedOperationException Stacktrace: org.apache.bookkeeper.mledger.ManagedLedgerException$MetaStoreException: java.util.concurrent.CompletionException: java.lang.UnsupportedOperationException Caused by: java.util.concurrent.CompletionException: java.lang.UnsupportedOperationException at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315) at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320) at java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:722) at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482) at org.apache.bookkeeper.common.util.OrderedExecutor$TimedRunnable.run(OrderedExecutor.java:201) at org.apache.bookkeeper.common.util.SingleThreadSafeScheduledExecutorService$SafeRunnable.run(SingleThreadSafeScheduledExecutorService.java:46) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: java.lang.UnsupportedOperationException at java.base/java.util.AbstractMap.put(AbstractMap.java:209) at java.base/java.util.AbstractMap.putAll(AbstractMap.java:281) at org.apache.pulsar.broker.admin.impl.PersistentTopicsBase$1.updatePropertiesComplete(PersistentTopicsBase.java:701) at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl$26.operationComplete(ManagedLedgerImpl.java:4177) at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl$26.operationComplete(ManagedLedgerImpl.java:4173) at org.apache.bookkeeper.mledger.impl.MetaStoreImpl.lambda$asyncUpdateLedgerIds$7(MetaStoreImpl.java:203) at java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:718) ... 10 more at org.apache.pulsar.client.admin.PulsarAdminException.wrap(PulsarAdminException.java:252) at org.apache.pulsar.client.admin.internal.BaseResource.sync(BaseResource.java:352) at org.apache.pulsar.client.admin.internal.TopicsImpl.updateProperties(TopicsImpl.java:419) at org.apache.pulsar.broker.admin.TopicPoliciesTest.updatePropertiesForAutoCreatedTopicTest(TopicPoliciesTest.java:142) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139) at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:677) at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:221) at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50) at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:969) at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:194) at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148) at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at org.testng.TestRunner.privateRun(TestRunner.java:829) at org.testng.TestRunner.run(TestRunner.java:602) at org.testng.SuiteRunner.runTest(SuiteRunner.java:437) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:431) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:391) at org.testng.SuiteRunner.run(SuiteRunner.java:330) at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
[GitHub] [pulsar-site] Anonymitaet commented on pull request #543: Add doc for transform functions before sinks
Anonymitaet commented on PR #543: URL: https://github.com/apache/pulsar-site/pull/543#issuecomment-1543000280 > Thanks for your contribution! > > 1. Have you previewed your changes and ensured everything goes as expected? >If not, please [preview your changes locally](https://pulsar.apache.org/contribute/document-preview/#preview-changes) and attach the screenshots to this PR. >In this way, you can get your PR merged more quickly. > 2. Does this change apply to multiple doc versions (e.g., 3.0.x)? >If so, please [update them all](https://github.com/apache/pulsar-site/tree/main/versioned_docs). > > Thank you! Seems that the 1st task is ignored -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] nkurihar commented on pull request #19983: [improve][broker] Log resource usage rate of brokers that need to be offloaded in ThresholdShedder
nkurihar commented on PR #19983: URL: https://github.com/apache/pulsar/pull/19983#issuecomment-1542987660 @eolivelli ping -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] Anonymitaet commented on a diff in pull request #558: Minor improvements to OIDC docs
Anonymitaet commented on code in PR #558: URL: https://github.com/apache/pulsar-site/pull/558#discussion_r1190503177 ## docs/security-openid-connect.md: ## @@ -21,9 +25,9 @@ After authenticating with the Identity Provider, the Pulsar client gets an acces 7. When token validation is successful, the Pulsar Server extracts the `sub` claim from the token (or the configured `openIDRoleClaim`) and uses it as the principal for authorization. 8. When the token expires, the Pulsar Server challenges the client to re-authenticate with the Identity Provider and provide a new access token. If the client fails to re-authenticate, the Pulsar Server closes the connection. -## Enable OpenID Connect Authentication in the Brokers, Proxies, and WebSocket Proxies +## Enable OpenID Connect Authentication in the Broker and Proxy -To configure Pulsar Servers to authenticate clients using OpenID Connect, add the following parameters to the `conf/broker.conf`, the `conf/proxy.conf`, and the `conf/websocket.conf` files. If you use a standalone Pulsar, you need to add these parameters to the `conf/standalone.conf` file: +To configure Pulsar Servers to authenticate clients using OpenID Connect, add the following parameters to the `conf/broker.conf` and the `conf/proxy.conf`. If you use a standalone Pulsar, you need to add these parameters to the `conf/standalone.conf` file: Review Comment: I think we do not need to capitalize them because they are not proper nouns (e.g., Mac, Apple Watch). ## docs/security-openid-connect.md: ## @@ -21,9 +25,9 @@ After authenticating with the Identity Provider, the Pulsar client gets an acces 7. When token validation is successful, the Pulsar Server extracts the `sub` claim from the token (or the configured `openIDRoleClaim`) and uses it as the principal for authorization. 8. When the token expires, the Pulsar Server challenges the client to re-authenticate with the Identity Provider and provide a new access token. If the client fails to re-authenticate, the Pulsar Server closes the connection. -## Enable OpenID Connect Authentication in the Brokers, Proxies, and WebSocket Proxies +## Enable OpenID Connect Authentication in the Broker and Proxy -To configure Pulsar Servers to authenticate clients using OpenID Connect, add the following parameters to the `conf/broker.conf`, the `conf/proxy.conf`, and the `conf/websocket.conf` files. If you use a standalone Pulsar, you need to add these parameters to the `conf/standalone.conf` file: +To configure Pulsar Servers to authenticate clients using OpenID Connect, add the following parameters to the `conf/broker.conf` and the `conf/proxy.conf`. If you use a standalone Pulsar, you need to add these parameters to the `conf/standalone.conf` file: Review Comment: We do not need to capitalize them because they are not proper nouns (e.g., Mac, Apple Watch). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] Anonymitaet commented on a diff in pull request #558: Minor improvements to OIDC docs
Anonymitaet commented on code in PR #558: URL: https://github.com/apache/pulsar-site/pull/558#discussion_r1190502294 ## docs/security-openid-connect.md: ## @@ -8,6 +8,10 @@ Apache Pulsar supports authenticating clients using [OpenID Connect](https://ope The source code for the OpenID Connect implementation is in the [pulsar-broker-auth-oidc](https://github.com/apache/pulsar/blob/master/pulsar-broker-auth-oidc/) submodule in the Apache Pulsar git repo. +:::note +Pulsar's OpenID Connect integration was introduced in Pulsar 3.0.0. As always, if you encounter any issues, please ask questions on Pulsar channels and open issues in GitHub. Review Comment: How about this? ::: note Pulsar's OpenID Connect integration is available from 3.0.0. Currently, this feature is in beta. ::: ## docs/security-openid-connect.md: ## @@ -8,6 +8,10 @@ Apache Pulsar supports authenticating clients using [OpenID Connect](https://ope The source code for the OpenID Connect implementation is in the [pulsar-broker-auth-oidc](https://github.com/apache/pulsar/blob/master/pulsar-broker-auth-oidc/) submodule in the Apache Pulsar git repo. +:::note +Pulsar's OpenID Connect integration was introduced in Pulsar 3.0.0. As always, if you encounter any issues, please ask questions on Pulsar channels and open issues in GitHub. Review Comment: How about this? > ::: note > > Pulsar's OpenID Connect integration is available from 3.0.0. Currently, this feature is in beta. > > ::: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] Anonymitaet commented on a diff in pull request #562: Add PIP-195 to 3.0 release note
Anonymitaet commented on code in PR #562: URL: https://github.com/apache/pulsar-site/pull/562#discussion_r1190499038 ## release-notes/versioned/pulsar-3.0.0.md: ## @@ -89,7 +89,7 @@ sidebar_label: Apache Pulsar 3.0.0 - Clear delayed message when unsubscribe & Make clear operation asynchronous by @coderzc in [#19901](https://github.com/apache/pulsar/pull/19901) - Merge multiple buckets at once by @coderzc in [#19927](https://github.com/apache/pulsar/pull/19927) - Ensure previous delayed index be removed from snapshotSegmentLastIndexTable & Make load operate asynchronous by @coderzc in [#20086](https://github.com/apache/pulsar/pull/20086) - - Fix avoid future of clear delayed message can't complete by @coderzc in [#20075](https://github.com/apache/pulsar/pull/20075) + - Fix the issue which the `future` of clean up delayed messages can't be completed by @coderzc in [#20075](https://github.com/apache/pulsar/pull/20075) Review Comment: ```suggestion - Fix the issue of the `future` of clean-up delayed messages can't be completed by @coderzc in [#20075](https://github.com/apache/pulsar/pull/20075) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] scrottty commented on issue #20282: [Bug] DnsResolverUtil "Cannot get DNS TTL settings from sun.net.InetAddressCachePolicy class" in Java 17
scrottty commented on issue #20282: URL: https://github.com/apache/pulsar/issues/20282#issuecomment-1542844755 This was solved by downgrading to Pulsar Version `2.10.4` and Java version `11` due to netty issues. See https://apache-pulsar.slack.com/archives/C5Z4T36F7/p1683674853984909 for slack discussion. Leaving open as it might be needed to be fixed in v3+ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch branch-2.10 updated: Revert "[fix] [broker] [branch-2.10] Upgrade rocksDB version to 6.29.4.1 to keep in sync with BookKeeper's RocksDB version (#20287)
This is an automated email from the ASF dual-hosted git repository. wave pushed a commit to branch branch-2.10 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-2.10 by this push: new 88038e343b6 Revert "[fix] [broker] [branch-2.10] Upgrade rocksDB version to 6.29.4.1 to keep in sync with BookKeeper's RocksDB version (#20287) 88038e343b6 is described below commit 88038e343b6ca51e6f61ff4743085240f1c4758c Author: Hang Chen AuthorDate: Thu May 11 03:51:30 2023 +0800 Revert "[fix] [broker] [branch-2.10] Upgrade rocksDB version to 6.29.4.1 to keep in sync with BookKeeper's RocksDB version (#20287) --- distribution/server/src/assemble/LICENSE.bin.txt | 2 +- pom.xml | 2 +- pulsar-broker/src/test/java/org/apache/pulsar/RoaringbitmapTest.java | 2 +- pulsar-sql/presto-distribution/LICENSE | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/distribution/server/src/assemble/LICENSE.bin.txt b/distribution/server/src/assemble/LICENSE.bin.txt index 138e54fc4a3..49fea8189ab 100644 --- a/distribution/server/src/assemble/LICENSE.bin.txt +++ b/distribution/server/src/assemble/LICENSE.bin.txt @@ -450,7 +450,7 @@ The Apache Software License, Version 2.0 - org.eclipse.jetty-jetty-alpn-conscrypt-server-9.4.51.v20230217.jar - org.eclipse.jetty-jetty-alpn-server-9.4.51.v20230217.jar * SnakeYaml -- org.yaml-snakeyaml-2.0.jar - * RocksDB - org.rocksdb-rocksdbjni-6.29.4.1.jar + * RocksDB - org.rocksdb-rocksdbjni-6.10.2.jar * Google Error Prone Annotations - com.google.errorprone-error_prone_annotations-2.5.1.jar * Apache Thrift - org.apache.thrift-libthrift-0.14.2.jar * OkHttp3 diff --git a/pom.xml b/pom.xml index 1febc3a06e7..41f7a79a434 100644 --- a/pom.xml +++ b/pom.xml @@ -118,7 +118,7 @@ flexible messaging model and an intuitive client API. 1.10.50 0.5.0 3.9.8 -6.29.4.1 +6.10.2 1.7.32 3.2.2 2.18.0 diff --git a/pulsar-broker/src/test/java/org/apache/pulsar/RoaringbitmapTest.java b/pulsar-broker/src/test/java/org/apache/pulsar/RoaringbitmapTest.java index 477e7413ef9..0f606e22007 100644 --- a/pulsar-broker/src/test/java/org/apache/pulsar/RoaringbitmapTest.java +++ b/pulsar-broker/src/test/java/org/apache/pulsar/RoaringbitmapTest.java @@ -1,4 +1,4 @@ -/* +/** * Licensed to the Apache Software Foundation (ASF) under one * or more contributor license agreements. See the NOTICE file * distributed with this work for additional information diff --git a/pulsar-sql/presto-distribution/LICENSE b/pulsar-sql/presto-distribution/LICENSE index ad858497b38..4c59ab739e2 100644 --- a/pulsar-sql/presto-distribution/LICENSE +++ b/pulsar-sql/presto-distribution/LICENSE @@ -411,7 +411,7 @@ The Apache Software License, Version 2.0 - presto-spi-332.jar - presto-record-decoder-332.jar * RocksDB JNI -- rocksdbjni-6.29.4.1.jar +- rocksdbjni-6.10.2.jar * SnakeYAML - snakeyaml-2.0.jar * Bean Validation API
[GitHub] [pulsar] dave2wave merged pull request #20287: Revert "[fix] [broker] [branch-2.10] Upgrade rocksDB version to 6.29.4.1 to keep in sync with BookKeeper's RocksDB version
dave2wave merged PR #20287: URL: https://github.com/apache/pulsar/pull/20287 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] dave2wave commented on pull request #20288: Revert "[fix] [broker] [branch-2.9] Upgrade rocksDB version to 6.29.4.1 to keep in sync with BookKeeper's RocksDB version
dave2wave commented on PR #20288: URL: https://github.com/apache/pulsar/pull/20288#issuecomment-1542715858 Thank you. Branch 2.9 is back to the expected Bookie behavior. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] nlu90 added a comment to the discussion: Pulsar Functions lifecycle and depolyment details.
GitHub user nlu90 added a comment to the discussion: Pulsar Functions lifecycle and depolyment details. If you use `ThreadRuntime`, they all run within the Function Worker process. If you use `ProcessRuntime`, each function's instance gets its own process. If you use `KubernetesRuntime`, each function's instance gets its own pod. GitHub link: https://github.com/apache/pulsar/discussions/20195#discussioncomment-5864425 This is an automatically sent email for commits@pulsar.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@pulsar.apache.org
[GitHub] [pulsar] nlu90 opened a new pull request, #20294: [function] refine function default schema mapping for primitive types
nlu90 opened a new pull request, #20294: URL: https://github.com/apache/pulsar/pull/20294 Fixes #xyz Master Issue: #xyz PIP: #xyz ### Motivation The current default schema mapping doesn't follow the pulsar schema doc guidance: https://pulsar.apache.org/docs/3.0.x/schema-understand/#primitive-type Primitive types (except BYTES and STRING) are mapped to JSON schema by default. This causes confusion and inconvenience. ### Modifications add primitive type schema mapping according to the pulsar schema doc ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If the box was checked, please highlight the changes* - [ ] Dependencies (add or upgrade a dependency) - [ ] The public API - [ ] The schema - [ ] The default values of configurations - [ ] The threading model - [ ] The binary protocol - [ ] The REST endpoints - [ ] The admin CLI options - [ ] The metrics - [ ] Anything that affects deployment ### Documentation - [ ] `doc` - [ ] `doc-required` - [X] `doc-not-needed` - [ ] `doc-complete` ### Matching PR in forked repository PR in forked repository: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on pull request #19993: [improve][broker] Save createIfMissing in TopicLoadingContext
poorbarcode commented on PR #19993: URL: https://github.com/apache/pulsar/pull/19993#issuecomment-1542610644 @lifepuzzlefun Could you write a test? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on a diff in pull request #18688: [fix][ml] There are two same-named managed ledgers in the one broker
poorbarcode commented on code in PR #18688:
URL: https://github.com/apache/pulsar/pull/18688#discussion_r1190241330
##
managed-ledger/src/test/java/org/apache/bookkeeper/mledger/impl/DuplicateManagedLedgerTest.java:
##
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.bookkeeper.mledger.impl;
+
+import static org.mockito.Mockito.any;
+import static org.mockito.Mockito.doAnswer;
+import static org.mockito.Mockito.spy;
+import io.netty.buffer.ByteBuf;
+import java.nio.charset.Charset;
+import java.util.UUID;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.bookkeeper.client.LedgerHandle;
+import org.apache.bookkeeper.mledger.AsyncCallbacks;
+import org.apache.bookkeeper.mledger.ManagedLedger;
+import org.apache.bookkeeper.mledger.ManagedLedgerConfig;
+import org.apache.bookkeeper.mledger.ManagedLedgerException;
+import org.apache.bookkeeper.mledger.Position;
+import org.apache.bookkeeper.test.MockedBookKeeperTestCase;
+import org.awaitility.Awaitility;
+import org.testng.Assert;
+import org.testng.annotations.Test;
+
+@Test(groups = "broker")
+@Slf4j
+public class DuplicateManagedLedgerTest extends MockedBookKeeperTestCase {
+
+private void triggerLedgerRollover(ManagedLedger ledger, int
maxEntriesPerLedger) {
+new Thread(() -> {
+int writeLedgerCount = 2;
+for (int i = 0; i < writeLedgerCount; i++) {
+for (int j = 0; j < maxEntriesPerLedger; j++) {
+byte[] data = String.format("%s_%s", i,
j).getBytes(Charset.defaultCharset());
+Object ctx = "";
+ledger.asyncAddEntry(data, new
AsyncCallbacks.AddEntryCallback() {
+@Override
+public void addComplete(Position position, ByteBuf
entryData, Object ctx) {
+
+}
+
+@Override
+public void addFailed(ManagedLedgerException
exception, Object ctx) {
+
+}
+}, ctx);
+}
+}
+}).start();
+}
+
+@Test
+public void testConcurrentCloseLedgerAndSwitchLedgerForReproduceIssue()
throws Exception {
+String managedLedgerName = "lg_" +
UUID.randomUUID().toString().replaceAll("-", "_");
+int maxEntriesPerLedger = 5;
+ManagedLedgerConfig config = new ManagedLedgerConfig();
+config.setThrottleMarkDelete(1);
+config.setMaximumRolloverTime(Integer.MAX_VALUE, TimeUnit.SECONDS);
+config.setMaxEntriesPerLedger(5);
+final ProcessCoordinator processCoordinator = new ProcessCoordinator();
+
+// call "switch ledger" and "managedLedger.close" concurrently.
+final ManagedLedgerImpl managedLedger1 = (ManagedLedgerImpl)
factory.open(managedLedgerName, config);
+waitManagedLedgerStateEquals(managedLedger1,
ManagedLedgerImpl.State.LedgerOpened);
+processCoordinator.on();
+final ManagedLedgerImpl sequentiallyManagedLedger1 =
+makeManagedLedgerWorksWithStrictlySequentially(managedLedger1,
processCoordinator);
+triggerLedgerRollover(sequentiallyManagedLedger1, maxEntriesPerLedger);
+sequentiallyManagedLedger1.close();
+waitManagedLedgerStateNotEquals(managedLedger1,
ManagedLedgerImpl.State.Closed);
+
+// create managedLedger2.
+final ManagedLedgerImpl managedLedger2 = (ManagedLedgerImpl)
factory.open(managedLedgerName, config);
+Assert.assertEquals(factory.ledgers.size(), 1);
+Assert.assertNotEquals(managedLedger1, managedLedger2);
+waitManagedLedgerInFactoryEquals(managedLedger2);
+processCoordinator.off();
+managedLedger1.close();
+waitManagedLedgerStateEquals(managedLedger1,
ManagedLedgerImpl.State.Closed);
+Assert.assertFalse(factory.ledgers.isEmpty());
+
Assert.assertEquals(factory.ledgers.get(managedLedger2.getName()).join(),
managedLedger2);
+
+
[GitHub] [pulsar] poorbarcode commented on a diff in pull request #18688: [fix][ml] There are two same-named managed ledgers in the one broker
poorbarcode commented on code in PR #18688:
URL: https://github.com/apache/pulsar/pull/18688#discussion_r1190224889
##
managed-ledger/src/test/java/org/apache/bookkeeper/mledger/impl/DuplicateManagedLedgerTest.java:
##
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.bookkeeper.mledger.impl;
+
+import static org.mockito.Mockito.any;
+import static org.mockito.Mockito.doAnswer;
+import static org.mockito.Mockito.spy;
+import io.netty.buffer.ByteBuf;
+import java.nio.charset.Charset;
+import java.util.UUID;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.bookkeeper.client.LedgerHandle;
+import org.apache.bookkeeper.mledger.AsyncCallbacks;
+import org.apache.bookkeeper.mledger.ManagedLedger;
+import org.apache.bookkeeper.mledger.ManagedLedgerConfig;
+import org.apache.bookkeeper.mledger.ManagedLedgerException;
+import org.apache.bookkeeper.mledger.Position;
+import org.apache.bookkeeper.test.MockedBookKeeperTestCase;
+import org.awaitility.Awaitility;
+import org.testng.Assert;
+import org.testng.annotations.Test;
+
+@Test(groups = "broker")
+@Slf4j
+public class DuplicateManagedLedgerTest extends MockedBookKeeperTestCase {
+
+private void triggerLedgerRollover(ManagedLedger ledger, int
maxEntriesPerLedger) {
+new Thread(() -> {
+int writeLedgerCount = 2;
+for (int i = 0; i < writeLedgerCount; i++) {
+for (int j = 0; j < maxEntriesPerLedger; j++) {
+byte[] data = String.format("%s_%s", i,
j).getBytes(Charset.defaultCharset());
+Object ctx = "";
+ledger.asyncAddEntry(data, new
AsyncCallbacks.AddEntryCallback() {
+@Override
+public void addComplete(Position position, ByteBuf
entryData, Object ctx) {
+
+}
+
+@Override
+public void addFailed(ManagedLedgerException
exception, Object ctx) {
+
+}
+}, ctx);
+}
+}
+}).start();
+}
+
+@Test
+public void testConcurrentCloseLedgerAndSwitchLedgerForReproduceIssue()
throws Exception {
+String managedLedgerName = "lg_" +
UUID.randomUUID().toString().replaceAll("-", "_");
+int maxEntriesPerLedger = 5;
+ManagedLedgerConfig config = new ManagedLedgerConfig();
+config.setThrottleMarkDelete(1);
+config.setMaximumRolloverTime(Integer.MAX_VALUE, TimeUnit.SECONDS);
+config.setMaxEntriesPerLedger(5);
+final ProcessCoordinator processCoordinator = new ProcessCoordinator();
+
+// call "switch ledger" and "managedLedger.close" concurrently.
+final ManagedLedgerImpl managedLedger1 = (ManagedLedgerImpl)
factory.open(managedLedgerName, config);
+waitManagedLedgerStateEquals(managedLedger1,
ManagedLedgerImpl.State.LedgerOpened);
+processCoordinator.on();
+final ManagedLedgerImpl sequentiallyManagedLedger1 =
+makeManagedLedgerWorksWithStrictlySequentially(managedLedger1,
processCoordinator);
+triggerLedgerRollover(sequentiallyManagedLedger1, maxEntriesPerLedger);
+sequentiallyManagedLedger1.close();
+waitManagedLedgerStateNotEquals(managedLedger1,
ManagedLedgerImpl.State.Closed);
+
+// create managedLedger2.
+final ManagedLedgerImpl managedLedger2 = (ManagedLedgerImpl)
factory.open(managedLedgerName, config);
+Assert.assertEquals(factory.ledgers.size(), 1);
+Assert.assertNotEquals(managedLedger1, managedLedger2);
+waitManagedLedgerInFactoryEquals(managedLedger2);
+processCoordinator.off();
+managedLedger1.close();
+waitManagedLedgerStateEquals(managedLedger1,
ManagedLedgerImpl.State.Closed);
+Assert.assertFalse(factory.ledgers.isEmpty());
+
Assert.assertEquals(factory.ledgers.get(managedLedger2.getName()).join(),
managedLedger2);
+
+
[GitHub] [pulsar] poorbarcode commented on pull request #20261: [fix][broker] Fix `UnsupportedOperationException` when update topic properties.
poorbarcode commented on PR #20261: URL: https://github.com/apache/pulsar/pull/20261#issuecomment-1542541072 Why were the labels lost?  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-manager] bpereto commented on issue #505: Starting Docker container pulsar-manager-0.3.0 with internal PostgreSQL does not work
bpereto commented on issue #505: URL: https://github.com/apache/pulsar-manager/issues/505#issuecomment-1542412933 @tisonkun I will test [v0.4.0-candidate-2](https://github.com/apache/pulsar-manager/releases/tag/v0.4.0-candidate-2) if you will release a docker image. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-manager] bpereto commented on issue #505: Starting Docker container pulsar-manager-0.3.0 with internal PostgreSQL does not work
bpereto commented on issue #505: URL: https://github.com/apache/pulsar-manager/issues/505#issuecomment-1542410152 ^ > [streamnative/pulsar-manager:v0.3.1-rc2](https://hub.docker.com/layers/streamnative/pulsar-manager/v0.3.1-rc2/images/sha256-2b1f9eaac6300f4a1d5a5e8cfd03a7ed789056fb8db4757f52d056c5199a716e?context=explore) > > Since the official release may take some time, I tried to build an image manually, can you try it? @bpereto @Zand3r24 @doug-ba @malaskowski @splio-rbechon keeps crashing: ``` 2023-05-10 15:28:19,418 INFO success: pulsar-manager-backend entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2023-05-10 15:28:19,418 INFO exited: pulsar-manager-backend (exit status 1; not expected) 2023-05-10 15:28:20,422 INFO spawned: 'pulsar-manager-backend' with pid 1145 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated (fb9c4d034bf -> 96367e1c2bf)
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git from fb9c4d034bf [fix][fn] Make pulsar-admin support update py/go with package url (#19897) add 96367e1c2bf [cleanup] Deduplicate test certificates to simplify management (#20289) No new revisions were added by this update. Summary of changes: .../pulsar/broker/admin/AdminApiTlsAuthTest.java | 38 ++ .../broker/admin/BrokerAdminClientTlsAuthTest.java | 18 -- .../broker/auth/MockedPulsarServiceBaseTest.java | 15 + .../authentication/tls-http/admin.cert.pem | 26 --- .../authentication/tls-http/admin.key-pk8.pem | 28 .../authentication/tls-http/broker.cert.pem| 27 --- .../authentication/tls-http/broker.key-pk8.pem | 28 .../resources/authentication/tls-http/ca.cert.pem | 29 - .../authentication/tls-http/proxy.cert.pem | 26 --- .../authentication/tls-http/proxy.key-pk8.pem | 28 .../authentication/tls-http/superproxy.cert.pem| 26 --- .../authentication/tls-http/superproxy.key-pk8.pem | 28 .../authentication/tls-http/user1.cert.pem | 26 --- .../authentication/tls-http/user1.key-pk8.pem | 28 .../proxy/server/AuthedAdminProxyHandlerTest.java | 32 -- .../SuperUserAuthedAdminProxyHandlerTest.java | 26 +++ .../authentication/tls-admin-proxy/admin.cert.pem | 26 --- .../tls-admin-proxy/admin.key-pk8.pem | 28 .../authentication/tls-admin-proxy/broker.cert.pem | 27 --- .../tls-admin-proxy/broker.key-pk8.pem | 28 .../authentication/tls-admin-proxy/ca.cert.pem | 29 - .../authentication/tls-admin-proxy/proxy.cert.pem | 26 --- .../tls-admin-proxy/proxy.key-pk8.pem | 28 .../tls-admin-proxy/randouser.cert.pem | 19 --- .../tls-admin-proxy/randouser.key-pk8.pem | 28 .../tls-admin-proxy/superproxy.cert.pem| 26 --- .../tls-admin-proxy/superproxy.key-pk8.pem | 28 .../authentication/tls-admin-proxy/user1.cert.pem | 26 --- .../tls-admin-proxy/user1.key-pk8.pem | 28 29 files changed, 64 insertions(+), 712 deletions(-) delete mode 100644 pulsar-broker/src/test/resources/authentication/tls-http/admin.cert.pem delete mode 100644 pulsar-broker/src/test/resources/authentication/tls-http/admin.key-pk8.pem delete mode 100644 pulsar-broker/src/test/resources/authentication/tls-http/broker.cert.pem delete mode 100644 pulsar-broker/src/test/resources/authentication/tls-http/broker.key-pk8.pem delete mode 100644 pulsar-broker/src/test/resources/authentication/tls-http/ca.cert.pem delete mode 100644 pulsar-broker/src/test/resources/authentication/tls-http/proxy.cert.pem delete mode 100644 pulsar-broker/src/test/resources/authentication/tls-http/proxy.key-pk8.pem delete mode 100644 pulsar-broker/src/test/resources/authentication/tls-http/superproxy.cert.pem delete mode 100644 pulsar-broker/src/test/resources/authentication/tls-http/superproxy.key-pk8.pem delete mode 100644 pulsar-broker/src/test/resources/authentication/tls-http/user1.cert.pem delete mode 100644 pulsar-broker/src/test/resources/authentication/tls-http/user1.key-pk8.pem delete mode 100644 pulsar-proxy/src/test/resources/authentication/tls-admin-proxy/admin.cert.pem delete mode 100644 pulsar-proxy/src/test/resources/authentication/tls-admin-proxy/admin.key-pk8.pem delete mode 100644 pulsar-proxy/src/test/resources/authentication/tls-admin-proxy/broker.cert.pem delete mode 100644 pulsar-proxy/src/test/resources/authentication/tls-admin-proxy/broker.key-pk8.pem delete mode 100644 pulsar-proxy/src/test/resources/authentication/tls-admin-proxy/ca.cert.pem delete mode 100644 pulsar-proxy/src/test/resources/authentication/tls-admin-proxy/proxy.cert.pem delete mode 100644 pulsar-proxy/src/test/resources/authentication/tls-admin-proxy/proxy.key-pk8.pem delete mode 100644 pulsar-proxy/src/test/resources/authentication/tls-admin-proxy/randouser.cert.pem delete mode 100644 pulsar-proxy/src/test/resources/authentication/tls-admin-proxy/randouser.key-pk8.pem delete mode 100644 pulsar-proxy/src/test/resources/authentication/tls-admin-proxy/superproxy.cert.pem delete mode 100644 pulsar-proxy/src/test/resources/authentication/tls-admin-proxy/superproxy.key-pk8.pem delete mode 100644 pulsar-proxy/src/test/resources/authentication/tls-admin-proxy/user1.cert.pem delete mode 100644 pulsar-proxy/src/test/resources/authentication/tls-admin-proxy/user1.key-pk8.pem
[GitHub] [pulsar] lhotari merged pull request #20289: [cleanup] Deduplicate test certificates to simplify management
lhotari merged PR #20289: URL: https://github.com/apache/pulsar/pull/20289 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] codecov-commenter commented on pull request #20289: [cleanup] Deduplicate test certificates to simplify management
codecov-commenter commented on PR #20289: URL: https://github.com/apache/pulsar/pull/20289#issuecomment-1542318420 ## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/20289?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#20289](https://app.codecov.io/gh/apache/pulsar/pull/20289?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (78d5f50) into [master](https://app.codecov.io/gh/apache/pulsar/commit/bc1764f9ef71dd31e8cd61c7571e493442bc6395?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (bc1764f) will **increase** coverage by `38.42%`. > The diff coverage is `66.66%`. [](https://app.codecov.io/gh/apache/pulsar/pull/20289?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #20289 +/- ## = + Coverage 34.48% 72.90% +38.42% - Complexity1252831953+19425 = Files 1614 1868 +254 Lines126177 138588+12411 Branches 1377215237 +1465 = + Hits 43508 101037+57529 + Misses7705129509-47542 - Partials 5618 8042 +2424 ``` | Flag | Coverage Δ | | |---|---|---| | inttests | `24.18% <38.88%> (+0.05%)` | :arrow_up: | | systests | `24.75% <11.11%> (?)` | | | unittests | `72.20% <66.66%> (+39.10%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://app.codecov.io/gh/apache/pulsar/pull/20289?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...r/broker/authentication/AuthenticationService.java](https://app.codecov.io/gh/apache/pulsar/pull/20289?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3B1bHNhci9icm9rZXIvYXV0aGVudGljYXRpb24vQXV0aGVudGljYXRpb25TZXJ2aWNlLmphdmE=) | `69.52% <37.50%> (+35.52%)` | :arrow_up: | | [...oker/service/nonpersistent/NonPersistentTopic.java](https://app.codecov.io/gh/apache/pulsar/pull/20289?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL25vbnBlcnNpc3RlbnQvTm9uUGVyc2lzdGVudFRvcGljLmphdmE=) | `74.29% <66.66%> (+23.86%)` | :arrow_up: | | [...rvice/nonpersistent/NonPersistentSubscription.java](https://app.codecov.io/gh/apache/pulsar/pull/20289?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL25vbnBlcnNpc3RlbnQvTm9uUGVyc2lzdGVudFN1YnNjcmlwdGlvbi5qYXZh) | `53.29% <100.00%> (+16.04%)` | :arrow_up: | | [...va/org/apache/pulsar/client/impl/ConsumerImpl.java](https://app.codecov.io/gh/apache/pulsar/pull/20289?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL0NvbnN1bWVySW1wbC5qYXZh) | `76.14% <100.00%> (+15.95%)` | :arrow_up: | | [...he/pulsar/client/impl/MultiTopicsConsumerImpl.java](https://app.codecov.io/gh/apache/pulsar/pull/20289?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL011bHRpVG9waWNzQ29uc3VtZXJJbXBsLmphdmE=) | `78.26% <100.00%> (+30.98%)` | :arrow_up: | ... and [1503 files with indirect coverage changes](https://app.codecov.io/gh/apache/pulsar/pull/20289/indirect-changes?src=pr=tree-more_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to
[GitHub] [pulsar] eugene-cheverda commented on pull request #20222: [fix][sec] Update dependencies to use snakeyaml 2.0
eugene-cheverda commented on PR #20222: URL: https://github.com/apache/pulsar/pull/20222#issuecomment-1542267781 Hi @mattisonchao did you have a chance to discuss this in dev mail group? TIA -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #20290: Flaky-test: ConnectionPoolTest.testSetProxyToTargetBrokerAddress
lhotari commented on issue #20290: URL: https://github.com/apache/pulsar/issues/20290#issuecomment-1542205122 After all, I came to the conclusion that the test was wrong. Here's the PR which contains the description and fix: #20293 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #20290: Flaky-test: ConnectionPoolTest.testSetProxyToTargetBrokerAddress
lhotari commented on issue #20290: URL: https://github.com/apache/pulsar/issues/20290#issuecomment-1542204767 After all, I came to the conclusion that the test was wrong. Here's the PR which contains the description and fix: #20293 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lhotari opened a new pull request, #20293: [fix][test] Fix flaky test ConnectionPoolTest.testSetProxyToTargetBrokerAddress
lhotari opened a new pull request, #20293: URL: https://github.com/apache/pulsar/pull/20293 Fixes #20290 ### Motivation The test ConnectionPoolTest.testSetProxyToTargetBrokerAddress is flaky ### Modifications Modify test so that it's not flaky. Use separate `broker1` and `broker2` since the connection pool doesn't contain handling for separating direct and proxied connections to a broker. The key in the connection pool is the "logical address" which is the target broker. It doesn't take the "physical address" into account. ### Documentation - [ ] `doc` - [ ] `doc-required` - [x] `doc-not-needed` - [ ] `doc-complete` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Technoboy- closed issue #18966: [Bug] Cannot update functions with Package URL
Technoboy- closed issue #18966: [Bug] Cannot update functions with Package URL URL: https://github.com/apache/pulsar/issues/18966 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated: [fix][fn] Make pulsar-admin support update py/go with package url (#19897)
This is an automated email from the ASF dual-hosted git repository.
technoboy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new fb9c4d034bf [fix][fn] Make pulsar-admin support update py/go with
package url (#19897)
fb9c4d034bf is described below
commit fb9c4d034bfdcfdf256720ec2490c737cae53839
Author: jiangpengcheng
AuthorDate: Wed May 10 21:16:17 2023 +0800
[fix][fn] Make pulsar-admin support update py/go with package url (#19897)
---
.../src/main/java/org/apache/pulsar/admin/cli/CmdFunctions.java | 4
1 file changed, 4 insertions(+)
diff --git
a/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdFunctions.java
b/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdFunctions.java
index 91ec7183ff1..9b30d59f167 100644
---
a/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdFunctions.java
+++
b/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdFunctions.java
@@ -1037,6 +1037,10 @@ public class CmdFunctions extends CmdBase {
updateOptions.setUpdateAuthData(updateAuthData);
if (Utils.isFunctionPackageUrlSupported(functionConfig.getJar())) {
getAdmin().functions().updateFunctionWithUrl(functionConfig,
functionConfig.getJar(), updateOptions);
+} else if
(Utils.isFunctionPackageUrlSupported(functionConfig.getPy())) {
+getAdmin().functions().updateFunctionWithUrl(functionConfig,
functionConfig.getPy(), updateOptions);
+} else if
(Utils.isFunctionPackageUrlSupported(functionConfig.getGo())) {
+getAdmin().functions().updateFunctionWithUrl(functionConfig,
functionConfig.getGo(), updateOptions);
} else {
getAdmin().functions().updateFunction(functionConfig,
userCodeFile, updateOptions);
}
[GitHub] [pulsar] Technoboy- merged pull request #19897: [fix][fn] Make pulsar-admin support update py/go with package url
Technoboy- merged PR #19897: URL: https://github.com/apache/pulsar/pull/19897 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated (c32caba54a5 -> f04252b84a8)
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git from c32caba54a5 [fix][broker] Fix default bundle size used while setting bookie affinity (#20250) add f04252b84a8 [fix][fn] Support multiple input topics for Go runtime (#2) No new revisions were added by this update. Summary of changes: pulsar-function-go/conf/conf.go| 18 ++- pulsar-function-go/conf/conf.yaml | 6 +- pulsar-function-go/pf/instance.go | 4 +- pulsar-function-go/pf/instanceConf.go | 31 ++-- .../functions/instance/go/GoInstanceConfig.java| 5 + .../src/main/resources/findbugsExclude.xml | 10 ++ .../pulsar/functions/runtime/RuntimeUtils.java | 13 +- .../integration/functions/PulsarFunctionsTest.java | 160 + .../functions/PulsarFunctionsTestBase.java | 2 + .../functions/go/PulsarFunctionsGoTest.java| 5 + .../functions/java/PulsarFunctionsJavaTest.java| 4 +- .../python/PulsarFunctionsPythonTest.java | 8 +- 12 files changed, 179 insertions(+), 87 deletions(-)
[GitHub] [pulsar] Technoboy- merged pull request #20000: [fix][fn] Support multiple input topics for Go runtime
Technoboy- merged PR #2: URL: https://github.com/apache/pulsar/pull/2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Technoboy- closed issue #19981: [Bug] Go Pulsar functions recieves message from only one topic even when provided multiple input topics
Technoboy- closed issue #19981: [Bug] Go Pulsar functions recieves message from only one topic even when provided multiple input topics URL: https://github.com/apache/pulsar/issues/19981 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #20290: Flaky-test: ConnectionPoolTest.testSetProxyToTargetBrokerAddress
lhotari commented on issue #20290: URL: https://github.com/apache/pulsar/issues/20290#issuecomment-1542171906 Changes to the test that make it fail consistently: https://github.com/lhotari/pulsar/commit/1018fb9d0cdd304bd63d2421aa96981bfea06320 This seems to reproduce the bug in the connection pool. The connection pool should take the proxy target into account. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #20290: Flaky-test: ConnectionPoolTest.testSetProxyToTargetBrokerAddress
lhotari commented on issue #20290:
URL: https://github.com/apache/pulsar/issues/20290#issuecomment-1542163898
the flakiness goes away when I make ClientCnx.close synchronous:
```
public void close() {
if (ctx != null) {
try {
ctx.close().sync();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
```
It seems that it doesn't make sense to close the connection since that
removes the connection from the connection pool. The connection pool in Pulsar
client isn't the connection pool you would expect it to be. It's not like a
JDBC datasource connection pool.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #20290: Flaky-test: ConnectionPoolTest.testSetProxyToTargetBrokerAddress
lhotari commented on issue #20290: URL: https://github.com/apache/pulsar/issues/20290#issuecomment-1542157160 with `conf.setConnectionsPerBroker(1);` the test fails more often. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #20290: Flaky-test: ConnectionPoolTest.testSetProxyToTargetBrokerAddress
lhotari commented on issue #20290: URL: https://github.com/apache/pulsar/issues/20290#issuecomment-1542155115 The change to use 127.0.0.101 and 127.0.0.102 doesn't fix the issue. This looks like a bug in the connection pool logic itself. The key in the pool should use both the logicalAddress and physicalAddress? I wonder how this could have worked in the first place? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] hpvd commented on issue #20291: PIP-251 Enhancing Transaction Buffer Stats and Introducing TransactionBufferInternalStats API
hpvd commented on issue #20291: URL: https://github.com/apache/pulsar/issues/20291#issuecomment-1542148059 this PIP Number seems to be already in use, see: https://github.com/apache/pulsar/issues?q=is%3Aissue+is%3Aopen+PIP+251 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated: [fix][broker] Fix default bundle size used while setting bookie affinity (#20250)
This is an automated email from the ASF dual-hosted git repository.
technoboy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new c32caba54a5 [fix][broker] Fix default bundle size used while setting
bookie affinity (#20250)
c32caba54a5 is described below
commit c32caba54a5ab2c01c092ab952343f4ac405da35
Author: Raghavender Mittapalli <14803749+syk-co...@users.noreply.github.com>
AuthorDate: Wed May 10 18:05:17 2023 +0530
[fix][broker] Fix default bundle size used while setting bookie affinity
(#20250)
---
.../main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java | 2 +-
.../org/apache/pulsar/broker/service/BrokerBookieIsolationTest.java | 4
2 files changed, 5 insertions(+), 1 deletion(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java
index 19f8d7c437d..8c72d0b0286 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java
@@ -883,7 +883,7 @@ public abstract class NamespacesBase extends AdminResource {
policies -> new LocalPolicies(policies.bundles,
bookieAffinityGroup,
policies.namespaceAntiAffinityGroup))
-.orElseGet(() -> new LocalPolicies(defaultBundle(),
+.orElseGet(() -> new
LocalPolicies(getBundles(config().getDefaultNumberOfNamespaceBundles()),
bookieAffinityGroup,
null));
log.info("[{}] Successfully updated local-policies
configuration: namespace={}, map={}", clientAppId(),
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerBookieIsolationTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerBookieIsolationTest.java
index 36d4053ae49..951892f4ebf 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerBookieIsolationTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerBookieIsolationTest.java
@@ -156,6 +156,7 @@ public class BrokerBookieIsolationTest {
config.setBrokerServicePort(Optional.of(0));
config.setAdvertisedAddress("localhost");
config.setBookkeeperClientIsolationGroups(brokerBookkeeperClientIsolationGroups);
+config.setDefaultNumberOfNamespaceBundles(8);
config.setManagedLedgerDefaultEnsembleSize(2);
config.setManagedLedgerDefaultWriteQuorum(2);
@@ -207,6 +208,9 @@ public class BrokerBookieIsolationTest {
.bookkeeperAffinityGroupPrimary(tenantNamespaceIsolationGroups)
.build());
+//Checks the namespace bundles after setting the bookie affinity
+assertEquals(admin.namespaces().getBundles(ns2).getNumBundles(),
config.getDefaultNumberOfNamespaceBundles());
+
try {
admin.namespaces().getBookieAffinityGroup(ns1);
fail("ns1 should have no bookie affinity group set");
[GitHub] [pulsar] Technoboy- closed issue #20194: [Bug] While Setting BookieAffinity group, not using the broker config to set the default number of bundles
Technoboy- closed issue #20194: [Bug] While Setting BookieAffinity group, not using the broker config to set the default number of bundles URL: https://github.com/apache/pulsar/issues/20194 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Technoboy- merged pull request #20250: [fix][broker] Fix default bundle size used while setting bookie affinity
Technoboy- merged PR #20250: URL: https://github.com/apache/pulsar/pull/20250 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #20290: Flaky-test: ConnectionPoolTest.testSetProxyToTargetBrokerAddress
lhotari commented on issue #20290: URL: https://github.com/apache/pulsar/issues/20290#issuecomment-1542133779 This issue reproduces locally with `@Test(invocationCount = 50, threadPoolSize = 10)` for me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #20290: Flaky-test: ConnectionPoolTest.testSetProxyToTargetBrokerAddress
lhotari commented on issue #20290:
URL: https://github.com/apache/pulsar/issues/20290#issuecomment-1542115730
the test has a problem:
https://github.com/apache/pulsar/blob/e2863391e7f6f9b6c5060f0f78378493f8df37f3/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/ConnectionPoolTest.java#L186-L192
`new InetSocketAddress("localhost", brokerPort)` and `new
InetSocketAddress("127.0.0.1", brokerPort)` will collide and that's why test
fails in certain case.
I guess that one way to fix this would be to use different loopback address.
[IPv4 network standards reserve the entire address block 127.0.0.0/8 (more
than 16 million addresses) for loopback
purposes](https://en.wikipedia.org/wiki/Localhost#Name_resolution).
For example `127.0.0.101` and `127.0.0.102` could be used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[pulsar] branch branch-2.9 updated: Revert "[fix] [broker] [branch-2.9] Upgrade rocksDB version to 6.29.4.1 to keep in sync with BookKeeper's RocksDB version (#20288)
This is an automated email from the ASF dual-hosted git repository. zhaocong pushed a commit to branch branch-2.9 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-2.9 by this push: new 5a5c8f52290 Revert "[fix] [broker] [branch-2.9] Upgrade rocksDB version to 6.29.4.1 to keep in sync with BookKeeper's RocksDB version (#20288) 5a5c8f52290 is described below commit 5a5c8f52290fc6dc6bfa3bff7d0619d55c1cb1f3 Author: Hang Chen AuthorDate: Wed May 10 20:12:29 2023 +0800 Revert "[fix] [broker] [branch-2.9] Upgrade rocksDB version to 6.29.4.1 to keep in sync with BookKeeper's RocksDB version (#20288) --- distribution/server/src/assemble/LICENSE.bin.txt | 2 +- pom.xml | 2 +- pulsar-sql/presto-distribution/LICENSE | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/distribution/server/src/assemble/LICENSE.bin.txt b/distribution/server/src/assemble/LICENSE.bin.txt index 7b0d0b33e82..43b8389c7fc 100644 --- a/distribution/server/src/assemble/LICENSE.bin.txt +++ b/distribution/server/src/assemble/LICENSE.bin.txt @@ -450,7 +450,7 @@ The Apache Software License, Version 2.0 - org.eclipse.jetty-jetty-alpn-conscrypt-server-9.4.51.v20230217.jar - org.eclipse.jetty-jetty-alpn-server-9.4.51.v20230217.jar * SnakeYaml -- org.yaml-snakeyaml-2.0.jar - * RocksDB - org.rocksdb-rocksdbjni-6.29.4.1.jar + * RocksDB - org.rocksdb-rocksdbjni-6.16.4.jar * Google Error Prone Annotations - com.google.errorprone-error_prone_annotations-2.5.1.jar * Apache Thrift - org.apache.thrift-libthrift-0.14.2.jar * OkHttp3 diff --git a/pom.xml b/pom.xml index e6fd2b3e563..9f002baffc6 100644 --- a/pom.xml +++ b/pom.xml @@ -117,7 +117,7 @@ flexible messaging model and an intuitive client API. 1.10.9 0.5.0 3.9.8 -6.29.4.1 +6.16.4 1.7.32 3.2.2 2.18.0 diff --git a/pulsar-sql/presto-distribution/LICENSE b/pulsar-sql/presto-distribution/LICENSE index ed80b2eed6f..93be4b5dc75 100644 --- a/pulsar-sql/presto-distribution/LICENSE +++ b/pulsar-sql/presto-distribution/LICENSE @@ -396,7 +396,7 @@ The Apache Software License, Version 2.0 - presto-spi-332.jar - presto-record-decoder-332.jar * RocksDB JNI -- rocksdbjni-6.29.4.1.jar +- rocksdbjni-6.16.4.jar * SnakeYAML - snakeyaml-2.0.jar * Bean Validation API
[GitHub] [pulsar] coderzc merged pull request #20288: Revert "[fix] [broker] [branch-2.9] Upgrade rocksDB version to 6.29.4.1 to keep in sync with BookKeeper's RocksDB version
coderzc merged PR #20288: URL: https://github.com/apache/pulsar/pull/20288 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #19584: [fix][fn] Make KubernetesRuntime translate characters in function tenant, namespace, and name during function removal to avoid label errors
lhotari commented on PR #19584: URL: https://github.com/apache/pulsar/pull/19584#issuecomment-1542067246 @csthomas1 would you mind merging changes from master (or rebasing the changes) and fixing the issues with current master? Kubernetes client version was upgraded from 12.0.1 version to 18.0.0 version in 93fb4f8eb6bd12872f05d7c5ec6c00b719d53009 . There are minor API changes which need to be addressed. ``` Error: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.11.0:testCompile (default-testCompile) on project pulsar-functions-runtime: Compilation failure Error: /home/runner/work/pulsar/pulsar/pulsar-functions/runtime/src/test/java/org/apache/pulsar/functions/runtime/kubernetes/KubernetesRuntimeTest.java:[1313,21] method readNamespacedStatefulSet in class io.kubernetes.client.openapi.apis.AppsV1Api cannot be applied to given types; Error:required: java.lang.String,java.lang.String,java.lang.String Error:found: java.lang.String,java.lang.String,java.lang.Object,java.lang.Object,java.lang.Object Error:reason: actual and formal argument lists differ in length Error: -> [Help 1] ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] dragonls commented on pull request #20151: [fix][broker] Fix auth test cases.
dragonls commented on PR #20151: URL: https://github.com/apache/pulsar/pull/20151#issuecomment-1542044884 > > Hi, @michaeljmarshall After revert #20068, do you have any suggestion to fix the test cases? Maybe be fixed by #20145. > > Are the modified tests failing in master? Yes. Because this part of codes is wrong in master: https://github.com/apache/pulsar/blob/master/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/AuthorizationTest.java#L220-L223 `role1` is the tenant admin, and it should have the consume permission. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lhotari commented on a diff in pull request #18688: [fix][ml] There are two same-named managed ledgers in the one broker
lhotari commented on code in PR #18688:
URL: https://github.com/apache/pulsar/pull/18688#discussion_r1189737621
##
managed-ledger/src/test/java/org/apache/bookkeeper/mledger/impl/DuplicateManagedLedgerTest.java:
##
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.bookkeeper.mledger.impl;
+
+import static org.mockito.Mockito.any;
+import static org.mockito.Mockito.doAnswer;
+import static org.mockito.Mockito.spy;
+import io.netty.buffer.ByteBuf;
+import java.nio.charset.Charset;
+import java.util.UUID;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.bookkeeper.client.LedgerHandle;
+import org.apache.bookkeeper.mledger.AsyncCallbacks;
+import org.apache.bookkeeper.mledger.ManagedLedger;
+import org.apache.bookkeeper.mledger.ManagedLedgerConfig;
+import org.apache.bookkeeper.mledger.ManagedLedgerException;
+import org.apache.bookkeeper.mledger.Position;
+import org.apache.bookkeeper.test.MockedBookKeeperTestCase;
+import org.awaitility.Awaitility;
+import org.testng.Assert;
+import org.testng.annotations.Test;
+
+@Test(groups = "broker")
+@Slf4j
+public class DuplicateManagedLedgerTest extends MockedBookKeeperTestCase {
+
+private void triggerLedgerRollover(ManagedLedger ledger, int
maxEntriesPerLedger) {
+new Thread(() -> {
+int writeLedgerCount = 2;
+for (int i = 0; i < writeLedgerCount; i++) {
+for (int j = 0; j < maxEntriesPerLedger; j++) {
+byte[] data = String.format("%s_%s", i,
j).getBytes(Charset.defaultCharset());
+Object ctx = "";
+ledger.asyncAddEntry(data, new
AsyncCallbacks.AddEntryCallback() {
+@Override
+public void addComplete(Position position, ByteBuf
entryData, Object ctx) {
+
+}
+
+@Override
+public void addFailed(ManagedLedgerException
exception, Object ctx) {
+
+}
+}, ctx);
+}
+}
+}).start();
+}
+
+@Test
+public void testConcurrentCloseLedgerAndSwitchLedgerForReproduceIssue()
throws Exception {
+String managedLedgerName = "lg_" +
UUID.randomUUID().toString().replaceAll("-", "_");
+int maxEntriesPerLedger = 5;
+ManagedLedgerConfig config = new ManagedLedgerConfig();
+config.setThrottleMarkDelete(1);
+config.setMaximumRolloverTime(Integer.MAX_VALUE, TimeUnit.SECONDS);
+config.setMaxEntriesPerLedger(5);
+final ProcessCoordinator processCoordinator = new ProcessCoordinator();
+
+// call "switch ledger" and "managedLedger.close" concurrently.
+final ManagedLedgerImpl managedLedger1 = (ManagedLedgerImpl)
factory.open(managedLedgerName, config);
+waitManagedLedgerStateEquals(managedLedger1,
ManagedLedgerImpl.State.LedgerOpened);
+processCoordinator.on();
+final ManagedLedgerImpl sequentiallyManagedLedger1 =
+makeManagedLedgerWorksWithStrictlySequentially(managedLedger1,
processCoordinator);
+triggerLedgerRollover(sequentiallyManagedLedger1, maxEntriesPerLedger);
+sequentiallyManagedLedger1.close();
+waitManagedLedgerStateNotEquals(managedLedger1,
ManagedLedgerImpl.State.Closed);
+
+// create managedLedger2.
+final ManagedLedgerImpl managedLedger2 = (ManagedLedgerImpl)
factory.open(managedLedgerName, config);
+Assert.assertEquals(factory.ledgers.size(), 1);
+Assert.assertNotEquals(managedLedger1, managedLedger2);
+waitManagedLedgerInFactoryEquals(managedLedger2);
+processCoordinator.off();
+managedLedger1.close();
+waitManagedLedgerStateEquals(managedLedger1,
ManagedLedgerImpl.State.Closed);
+Assert.assertFalse(factory.ledgers.isEmpty());
+
Assert.assertEquals(factory.ledgers.get(managedLedger2.getName()).join(),
managedLedger2);
+
+
[pulsar] branch master updated (152b4a2612b -> ea56197d18d)
This is an automated email from the ASF dual-hosted git repository. nicoloboschi pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git from 152b4a2612b [improve][build] Upgrade maven surefire plugin and other build/test plugins/libs including TestNG version (#20270) add ea56197d18d [fix][monitor] topic with double quote breaks the prometheus format (#20230) No new revisions were added by this update. Summary of changes: .../stats/prometheus/PrometheusMetricStreams.java | 6 - .../pulsar/broker/stats/PrometheusMetricsTest.java | 26 ++ 2 files changed, 31 insertions(+), 1 deletion(-)
[GitHub] [pulsar] nicoloboschi merged pull request #20230: [fix][monitor] topic with double quote breaks the prometheus format
nicoloboschi merged PR #20230: URL: https://github.com/apache/pulsar/pull/20230 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] 7hong commented on issue #19307: [Bug] JVM crash in unit tests: PhaseIdealLoop::build_loop_late_post_work
7hong commented on issue #19307: URL: https://github.com/apache/pulsar/issues/19307#issuecomment-1541958093 I found this problem in the test environment. Pulsar version: 2.11.1 JDK version: 17.0.6 Perhaps it is related to this problem（ https://bugs.openjdk.org/browse/jdk-8285835）, so upgrade JDK to 17.0.7. There is no crash for the time being. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #18688: [fix][ml] There are two same-named managed ledgers in the one broker
lhotari commented on PR #18688: URL: https://github.com/apache/pulsar/pull/18688#issuecomment-1541947065 > Could you please help review this PR? @poorbarcode The change to production code itself looks good, but I don't like the test. Addressing that would require more changes to ML factory to make it possible to sub class it for tests with test hooks instead of relying on Mockito. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on pull request #18688: [fix][ml] There are two same-named managed ledgers in the one broker
poorbarcode commented on PR #18688: URL: https://github.com/apache/pulsar/pull/18688#issuecomment-1541933006 @lhotari > I assume this PR is still relevant? Yes. > Please merge latest changes from master and check that tests still pass. Done. Could you please help review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] eolivelli commented on a diff in pull request #20179: [fix][broker] Fix recentlyJoinedConsumers to address the out-of-order issue
eolivelli commented on code in PR #20179:
URL: https://github.com/apache/pulsar/pull/20179#discussion_r1189722242
##
pulsar-client-admin-api/src/main/java/org/apache/pulsar/common/policies/data/SubscriptionStats.java:
##
@@ -115,8 +115,8 @@ public interface SubscriptionStats {
/** Whether the Key_Shared subscription mode is AUTO_SPLIT or STICKY. */
String getKeySharedMode();
-/** This is for Key_Shared subscription to get the recentJoinedConsumers
in the Key_Shared subscription. */
-Map getConsumersAfterMarkDeletePosition();
+/** This is for Key_Shared subscription to get the recentlyJoinedConsumers
in the Key_Shared subscription. */
Review Comment:
the same here
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentSubscription.java:
##
@@ -1186,11 +1186,19 @@ public SubscriptionStatsImpl getStats(Boolean
getPreciseBacklog, boolean subscri
subStats.allowOutOfOrderDelivery =
keySharedDispatcher.isAllowOutOfOrderDelivery();
subStats.keySharedMode =
keySharedDispatcher.getKeySharedMode().toString();
-LinkedHashMap recentlyJoinedConsumers =
keySharedDispatcher
+LinkedHashMap
Review Comment:
this is a code smell to refer to this implementation class
PersistentStickyKeyDispatcherMultipleConsumers, maybe we should move the
LastSentPositions to a top level public class
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentSubscription.java:
##
@@ -1186,11 +1186,19 @@ public SubscriptionStatsImpl getStats(Boolean
getPreciseBacklog, boolean subscri
subStats.allowOutOfOrderDelivery =
keySharedDispatcher.isAllowOutOfOrderDelivery();
subStats.keySharedMode =
keySharedDispatcher.getKeySharedMode().toString();
-LinkedHashMap recentlyJoinedConsumers =
keySharedDispatcher
+LinkedHashMap
+recentlyJoinedConsumers = keySharedDispatcher
.getRecentlyJoinedConsumers();
if (recentlyJoinedConsumers != null &&
recentlyJoinedConsumers.size() > 0) {
recentlyJoinedConsumers.forEach((k, v) -> {
-
subStats.consumersAfterMarkDeletePosition.put(k.consumerName(), v.toString());
+// Dispatchers allows same name consumers
+final StringBuilder stringBuilder = new StringBuilder();
Review Comment:
could we use a struct instead of a string ? (you can use a 'record' now, but
as this fix should be cherrypick a plain old java object works better)
strings are error prone in code and also less efficient
##
pulsar-client-admin-api/src/main/java/org/apache/pulsar/common/policies/data/ConsumerStats.java:
##
@@ -69,8 +69,8 @@ public interface ConsumerStats {
/** Flag to verify if consumer is blocked due to reaching threshold of
unacked messages. */
boolean isBlockedConsumerOnUnackedMsgs();
-/** The read position of the cursor when the consumer joining. */
-String getReadPositionWhenJoining();
+/** Last sent positions per sticky key of the cursor when the consumer
joining. */
Review Comment:
This is a breaking API change, please do not do it
you can add the new method but we should keep the old value as well,
otherwise we cannot cherry-pick to old branches
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #18688: [fix][ml] There are two same-named managed ledgers in the one broker
lhotari commented on PR #18688: URL: https://github.com/apache/pulsar/pull/18688#issuecomment-1541909042 @poorbarcode I assume this PR is still relevant? Please merge latest changes from master and check that tests still pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] csthomas1 opened a new pull request, #19584: [fix][fn] Make KubernetesRuntime translate characters in function tenant, namespace, and name during function removal to avoid label erro
csthomas1 opened a new pull request, #19584: URL: https://github.com/apache/pulsar/pull/19584 Make KubernetesRuntime translate tenant, namespace, and function name characters that are not allowed within Kubernetes labels as part of function removal. Fixes #19583 ### Motivation See linked issue ticket. ### Modifications Updated KubernetesRuntime.deleteStatefulSet() to pass function tenant, namespace, and name values through the pre-existing getLabels() function to replace disallowed characters. ### Verifying this change - [ ] Make sure that the change passes the CI checks. Adds the following test cases: - KubernetesRuntimeTest: - testDeleteStatefulSetWithTranslatedKubernetesLabelChars(): Verifies that characters not supported by Kubernetes naming requirements are translated to acceptable forms ### Does this pull request potentially affect one of the following parts: *If the box was checked, please highlight the changes* - [ ] Dependencies (add or upgrade a dependency) - [ ] The public API - [ ] The schema - [ ] The default values of configurations - [ ] The threading model - [ ] The binary protocol - [ ] The REST endpoints - [ ] The admin CLI options - [ ] The metrics - [ ] Anything that affects deployment ### Documentation - [ ] `doc` - [ ] `doc-required` - [X] `doc-not-needed` - [ ] `doc-complete` ### Matching PR in forked repository PR in forked repository: (https://github.com/csthomas1/pulsar/pull/1) (All tests passed -- the Flaky tests failure occurs after execution of the tests themselves and appears to be due to a failed CodeCov upload attempt) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
