[pulsar-client-cpp] branch main updated: Fix flaky testAcknowledgeCumulativeWithPartition (#214)
This is an automated email from the ASF dual-hosted git repository.
xyz pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-client-cpp.git
The following commit(s) were added to refs/heads/main by this push:
new 2108d54 Fix flaky testAcknowledgeCumulativeWithPartition (#214)
2108d54 is described below
commit 2108d547c788f0bc735e6fbcb32384e921e3520e
Author: Yunze Xu
AuthorDate: Wed Mar 15 13:51:23 2023 +0800
Fix flaky testAcknowledgeCumulativeWithPartition (#214)
Fixes https://github.com/apache/pulsar-client-cpp/issues/213
### Motivation
In `ConsumerTest.testAcknowledgeCumulativeWithPartition`, the last two
`MessageId`s are acknowledged because they are treated as the latest two
`MessageId`s of the two partitions. However, this assumption is false
because `MultiTopicsConsumerImpl` does not guarantee receiving messages
in a round robin way. Here is an example output when I added the debug
logs for the `MessageId` received.
```
(519,48,1,-1)
(519,49,1,-1)
(518,48,0,-1)
(518,49,0,-1)
```
We can see the last two `MessageId`s are both from partition 0.
### Modifications
Use an array `latestMsgIds` to record the latest `MessageId` of a given
partition in this test.
---
tests/ConsumerTest.cc | 9 ++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/tests/ConsumerTest.cc b/tests/ConsumerTest.cc
index 944b87b..b1329e4 100644
--- a/tests/ConsumerTest.cc
+++ b/tests/ConsumerTest.cc
@@ -19,6 +19,7 @@
#include
#include
+#include
#include
#include
#include
@@ -291,14 +292,16 @@ TEST(ConsumerTest,
testAcknowledgeCumulativeWithPartition) {
}
Message msg;
+std::array latestMsgIds;
for (int i = 0; i < numMessages; i++) {
ASSERT_EQ(ResultOk, consumer.receive(msg));
// The last message of each partition topic be ACK
-if (i >= numMessages - 2) {
-consumer.acknowledgeCumulative(msg.getMessageId());
-}
+latestMsgIds[msg.getMessageId().partition()] = msg.getMessageId();
}
ASSERT_EQ(ResultTimeout, consumer.receive(msg, 2000));
+for (auto&& msgId : latestMsgIds) {
+consumer.acknowledgeCumulative(msgId);
+}
// Assert that there is no message in the tracker.
auto multiConsumerImpl =
PulsarFriend::getMultiTopicsConsumerImplPtr(consumer);
[GitHub] [pulsar-client-cpp] BewareMyPower merged pull request #214: Fix flaky testAcknowledgeCumulativeWithPartition
BewareMyPower merged PR #214: URL: https://github.com/apache/pulsar-client-cpp/pull/214 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-cpp] BewareMyPower closed issue #213: [Flaky Test] ConsumerTest.testAcknowledgeCumulativeWithPartition
BewareMyPower closed issue #213: [Flaky Test] ConsumerTest.testAcknowledgeCumulativeWithPartition URL: https://github.com/apache/pulsar-client-cpp/issues/213 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Demogorgon314 commented on a diff in pull request #19773: [improve][broker] PIP-192: Make split admin API functional
Demogorgon314 commented on code in PR #19773:
URL: https://github.com/apache/pulsar/pull/19773#discussion_r1136558612
##
pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/extensions/ExtensibleLoadManagerImplTest.java:
##
@@ -382,6 +378,56 @@ private void checkOwnershipState(String broker,
NamespaceBundle bundle)
assertFalse(otherLoadManager.checkOwnershipAsync(Optional.empty(),
bundle).get());
}
+@Test(timeOut = 30 * 1000)
+public void testSplitBundleAdminAPI() throws Exception {
+String namespace = "public/default";
+String topic = "persistent://" + namespace + "/test-split";
+admin.topics().createPartitionedTopic(topic, 10);
+BundlesData bundles = admin.namespaces().getBundles(namespace);
+int numBundles = bundles.getNumBundles();
+
+String firstBundle = bundles.getBoundaries().get(0) + "_" +
bundles.getBoundaries().get(1);
+
+admin.namespaces().splitNamespaceBundle(namespace, firstBundle, true,
null);
+
+assertEquals(admin.namespaces().getBundles(namespace).getNumBundles(),
numBundles + 1);
+
Review Comment:
Yes, updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[pulsar] branch master updated (0e96dedf66c -> 160a8643a1f)
This is an automated email from the ASF dual-hosted git repository. zhaocong pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git from 0e96dedf66c [fix][broker][PIP-195] Don't clean up BucketDelayedDeliveryTracker when all consumer disconnect (#19801) add 160a8643a1f [improve][broker][PIP-195] Add topicName and cursorName for ledger metadata of bucket snapshot (#19802) No new revisions were added by this update. Summary of changes: .../bookkeeper/mledger/impl/LedgerMetadataUtils.java | 15 +++ .../delayed/bucket/BookkeeperBucketSnapshotStorage.java | 9 + .../org/apache/pulsar/broker/delayed/bucket/Bucket.java | 6 +- .../broker/delayed/bucket/BucketSnapshotStorage.java | 8 +--- .../org/apache/pulsar/broker/service/BrokerService.java | 2 ++ .../delayed/BookkeeperBucketSnapshotStorageTest.java | 13 - .../pulsar/broker/delayed/MockBucketSnapshotStorage.java | 3 ++- .../delayed/bucket/BucketDelayedDeliveryTrackerTest.java | 2 +- 8 files changed, 39 insertions(+), 19 deletions(-)
[GitHub] [pulsar] coderzc merged pull request #19802: [improve][broker][PIP-195] Add topicName and cursorName for ledger metadata of bucket snapshot
coderzc merged PR #19802: URL: https://github.com/apache/pulsar/pull/19802 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Technoboy- commented on pull request #19730: [improve][broker] PIP-192 Made only the leader consume TopBundlesLoadDataStore
Technoboy- commented on PR #19730: URL: https://github.com/apache/pulsar/pull/19730#issuecomment-1469342501 /pulsarbot rerun-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] heesung-sn commented on a diff in pull request #19773: [improve][broker] PIP-192: Make split admin API functional
heesung-sn commented on code in PR #19773:
URL: https://github.com/apache/pulsar/pull/19773#discussion_r1136528999
##
pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/extensions/ExtensibleLoadManagerImplTest.java:
##
@@ -382,6 +378,56 @@ private void checkOwnershipState(String broker,
NamespaceBundle bundle)
assertFalse(otherLoadManager.checkOwnershipAsync(Optional.empty(),
bundle).get());
}
+@Test(timeOut = 30 * 1000)
+public void testSplitBundleAdminAPI() throws Exception {
+String namespace = "public/default";
+String topic = "persistent://" + namespace + "/test-split";
+admin.topics().createPartitionedTopic(topic, 10);
+BundlesData bundles = admin.namespaces().getBundles(namespace);
+int numBundles = bundles.getNumBundles();
+
+String firstBundle = bundles.getBoundaries().get(0) + "_" +
bundles.getBoundaries().get(1);
+
+admin.namespaces().splitNamespaceBundle(namespace, firstBundle, true,
null);
+
+assertEquals(admin.namespaces().getBundles(namespace).getNumBundles(),
numBundles + 1);
+
Review Comment:
Can we assert if the child bundle ranges exist too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[pulsar-test-infra] branch dependabot/npm_and_yarn/action-junit-report/webpack-5.76.0 created (now 7c4f87e)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/npm_and_yarn/action-junit-report/webpack-5.76.0 in repository https://gitbox.apache.org/repos/asf/pulsar-test-infra.git at 7c4f87e Bump webpack from 5.74.0 to 5.76.0 in /action-junit-report No new revisions were added by this update.
[GitHub] [pulsar-test-infra] dependabot[bot] opened a new pull request, #104: Bump webpack from 5.74.0 to 5.76.0 in /action-junit-report
dependabot[bot] opened a new pull request, #104: URL: https://github.com/apache/pulsar-test-infra/pull/104 Bumps [webpack](https://github.com/webpack/webpack) from 5.74.0 to 5.76.0. Release notes Sourced from https://github.com/webpack/webpack/releases;>webpack's releases. v5.76.0 Bugfixes Avoid cross-realm object access by https://github.com/Jack-Works;>@Jack-Works in https://redirect.github.com/webpack/webpack/pull/16500;>webpack/webpack#16500 Improve hash performance via conditional initialization by https://github.com/lvivski;>@lvivski in https://redirect.github.com/webpack/webpack/pull/16491;>webpack/webpack#16491 Serialize generatedCode info to fix bug in asset module cache restoration by https://github.com/ryanwilsonperkin;>@ryanwilsonperkin in https://redirect.github.com/webpack/webpack/pull/16703;>webpack/webpack#16703 Improve performance of hashRegExp lookup by https://github.com/ryanwilsonperkin;>@ryanwilsonperkin in https://redirect.github.com/webpack/webpack/pull/16759;>webpack/webpack#16759 Features add target to LoaderContext type by https://github.com/askoufis;>@askoufis in https://redirect.github.com/webpack/webpack/pull/16781;>webpack/webpack#16781 Security https://github.com/advisories/GHSA-3rfm-jhwj-7488;>CVE-2022-37603 fixed by https://github.com/akhilgkrishnan;>@akhilgkrishnan in https://redirect.github.com/webpack/webpack/pull/16446;>webpack/webpack#16446 Repo Changes Fix HTML5 logo in README by https://github.com/jakebailey;>@jakebailey in https://redirect.github.com/webpack/webpack/pull/16614;>webpack/webpack#16614 Replace TypeScript logo in README by https://github.com/jakebailey;>@jakebailey in https://redirect.github.com/webpack/webpack/pull/16613;>webpack/webpack#16613 Update actions/cache dependencies by https://github.com/piwysocki;>@piwysocki in https://redirect.github.com/webpack/webpack/pull/16493;>webpack/webpack#16493 New Contributors https://github.com/Jack-Works;>@Jack-Works made their first contribution in https://redirect.github.com/webpack/webpack/pull/16500;>webpack/webpack#16500 https://github.com/lvivski;>@lvivski made their first contribution in https://redirect.github.com/webpack/webpack/pull/16491;>webpack/webpack#16491 https://github.com/jakebailey;>@jakebailey made their first contribution in https://redirect.github.com/webpack/webpack/pull/16614;>webpack/webpack#16614 https://github.com/akhilgkrishnan;>@akhilgkrishnan made their first contribution in https://redirect.github.com/webpack/webpack/pull/16446;>webpack/webpack#16446 https://github.com/ryanwilsonperkin;>@ryanwilsonperkin made their first contribution in https://redirect.github.com/webpack/webpack/pull/16703;>webpack/webpack#16703 https://github.com/piwysocki;>@piwysocki made their first contribution in https://redirect.github.com/webpack/webpack/pull/16493;>webpack/webpack#16493 https://github.com/askoufis;>@askoufis made their first contribution in https://redirect.github.com/webpack/webpack/pull/16781;>webpack/webpack#16781 Full Changelog: https://github.com/webpack/webpack/compare/v5.75.0...v5.76.0;>https://github.com/webpack/webpack/compare/v5.75.0...v5.76.0 v5.75.0 Bugfixes experiments.* normalize to false when opt-out avoid NaN% show the correct error when using a conflicting chunk name in code HMR code tests existance of window before trying to access it fix eval-nosources-* actually exclude sources fix race condition where no module is returned from processing module fix position of standalong semicolon in runtime code Features add support for @import to extenal CSS when using experimental CSS in node add i64 support to the deprecated WASM implementation Developer Experience expose EnableWasmLoadingPlugin add more typings generate getters instead of readonly properties in typings to allow overriding them Commits https://github.com/webpack/webpack/commit/97b1718720c33f1b17302a74c5284b01e02ec001;>97b1718 Merge pull request https://redirect.github.com/webpack/webpack/issues/16781;>#16781 from askoufis/loader-context-target-type https://github.com/webpack/webpack/commit/b84efe6224b276bf72e4c5e2f4e76acddfaeef07;>b84efe6 Merge pull request https://redirect.github.com/webpack/webpack/issues/16759;>#16759 from ryanwilsonperkin/real-content-hash-regex-perf https://github.com/webpack/webpack/commit/c98e9e001441b165c7ed4845700839730b505833;>c98e9e0 Merge pull request https://redirect.github.com/webpack/webpack/issues/16493;>#16493 from piwysocki/patch-1 https://github.com/webpack/webpack/commit/5f34acfbc074da6cc09f48944d7f2b4273ffb3f8;>5f34acf feat: Add target to LoaderContext type https://github.com/webpack/webpack/commit/b7fc4d876deb958d7ee81ecc00a312e39a354a44;>b7fc4d8 Merge pull request
[GitHub] [pulsar] heesung-sn commented on a diff in pull request #19773: [improve][broker] PIP-192: Make split admin API functional
heesung-sn commented on code in PR #19773:
URL: https://github.com/apache/pulsar/pull/19773#discussion_r1136525777
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/channel/ServiceUnitStateData.java:
##
@@ -36,16 +39,22 @@ public record ServiceUnitStateData(
}
}
+public ServiceUnitStateData(ServiceUnitState state, String dstBroker,
String sourceBroker,
+Map>
splitServiceUnitToDestBroker, long versionId) {
+this(state, dstBroker, sourceBroker, splitServiceUnitToDestBroker,
false,
+System.currentTimeMillis(), versionId);
+}
+
public ServiceUnitStateData(ServiceUnitState state, String dstBroker,
String sourceBroker, long versionId) {
-this(state, dstBroker, sourceBroker, false,
System.currentTimeMillis(), versionId);
+this(state, dstBroker, sourceBroker, Map.of(), false,
System.currentTimeMillis(), versionId);
Review Comment:
Please change Map.of() to null.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] nodece commented on pull request #19793: [fix] [test] Fix flaky test `MetadataStoreStatsTest.testBatchMetadataStoreMetrics`
nodece commented on PR #19793: URL: https://github.com/apache/pulsar/pull/19793#issuecomment-1469312388 /pulsarbot rerun-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] congbobo184 commented on a diff in pull request #19816: [fix] [txn] fix race condition while update transaction state.
congbobo184 commented on code in PR #19816:
URL: https://github.com/apache/pulsar/pull/19816#discussion_r1136516511
##
pulsar-transaction/coordinator/src/main/java/org/apache/pulsar/transaction/coordinator/impl/MLTransactionMetadataStore.java:
##
@@ -400,6 +400,11 @@ public CompletableFuture updateTxnStatus(TxnID
txnID, TxnStatus newStatus,
appendLogCount.increment();
try {
synchronized (txnMetaListPair.getLeft()) {
+if (txnMetaListPair.getLeft().status() ==
newStatus) {
Review Comment:
please add a test for it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] XiaoYaoheihei added a comment to the discussion: How to start to understand and learn this technology?
GitHub user XiaoYaoheihei added a comment to the discussion: How to start to understand and learn this technology? Thank you !!! GitHub link: https://github.com/apache/pulsar/discussions/19807#discussioncomment-5316973 This is an automatically sent email for commits@pulsar.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@pulsar.apache.org
[GitHub] [pulsar] congbobo184 commented on a diff in pull request #19737: [improve] [broker] PIP-240: new public method unloadSubscription in PersistentTopic
congbobo184 commented on code in PR #19737:
URL: https://github.com/apache/pulsar/pull/19737#discussion_r1136504678
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java:
##
@@ -428,6 +430,26 @@ public AtomicLong getPendingWriteOps() {
return pendingWriteOps;
}
+public CompletableFuture unloadSubscription(String subName) {
+synchronized (ledger) {
+final PersistentSubscription sub = subscriptions.get(subName);
+if (sub == null) {
Review Comment:
if sub == null, may be we should return exception
##
pulsar-broker/src/test/java/org/apache/pulsar/client/api/UnloadSubscriptionTest.java:
##
@@ -0,0 +1,243 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.api;
+
+import static org.apache.pulsar.client.api.SubscriptionType.Shared;
+import static org.apache.pulsar.client.api.SubscriptionType.Key_Shared;
+import static org.apache.pulsar.client.api.SubscriptionType.Failover;
+import static org.apache.pulsar.client.api.SubscriptionType.Exclusive;
+import static org.testng.Assert.assertEquals;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.CompletableFuture;
+import java.util.stream.Collectors;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.pulsar.broker.service.persistent.PersistentTopic;
+import org.apache.pulsar.client.impl.BatchMessageIdImpl;
+import org.apache.pulsar.client.impl.MessageIdImpl;
+import org.apache.pulsar.client.impl.TopicMessageIdImpl;
+import org.apache.pulsar.common.util.FutureUtil;
+import org.awaitility.Awaitility;
+import org.testng.annotations.AfterClass;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.DataProvider;
+import org.testng.annotations.Test;
+
+@Slf4j
+@Test(groups = "broker-api")
+public class UnloadSubscriptionTest extends ProducerConsumerBase {
+
+@BeforeClass(alwaysRun = true)
+@Override
+protected void setup() throws Exception {
+super.internalSetup();
+super.producerBaseSetup();
+}
+
+@Override
+protected void doInitConf() throws Exception {
+super.doInitConf();
+conf.setSystemTopicEnabled(false);
+conf.setTransactionCoordinatorEnabled(false);
+}
+
+@AfterClass(alwaysRun = true)
+@Override
+protected void cleanup() throws Exception {
+super.internalCleanup();
+}
+
+@DataProvider(name = "unloadCases")
+public Object[][] unloadCases (){
+// [msgCount, enabledBatch, maxMsgPerBatch, subType, ackMsgCount]
+return new Object[][]{
+{100, false, 1, Exclusive, 0},
+{100, false, 1, Failover, 0},
+{100, false, 1, Shared, 0},
+{100, false, 1, Key_Shared, 0},
+{100, true, 5, Exclusive, 0},
+{100, true, 5, Failover, 0},
+{100, true, 5, Shared, 0},
+{100, true, 5, Key_Shared, 0},
+{100, false, 1, Exclusive, 50},
+{100, false, 1, Failover, 50},
+{100, false, 1, Shared, 50},
+{100, false, 1, Key_Shared, 50},
+{100, true, 5, Exclusive, 50},
+{100, true, 5, Failover, 50},
+{100, true, 5, Shared, 50},
+{100, true, 5, Key_Shared, 50},
+};
+}
+
+@Test(dataProvider = "unloadCases", invocationCount = 50)
+public void testSingleConsumer(int msgCount, boolean enabledBatch, int
maxMsgPerBatch, SubscriptionType subType,
+ int ackMsgCount) throws Exception {
+final String topicName = "persistent://my-property/my-ns/tp-" +
UUID.randomUUID().toString();
+final String subName = "sub";
+ListenerConsumer listenerConsumer = createListener(topicName, subName,
subType);
+ProducerAndMessageIds producerAndMessageIds =
+createProducerAndSendMessages(topicName, msgCount,
enabledBatch, maxMsgPerBatch);
+log.info("send
[pulsar] branch master updated (7a11edd4fda -> 0e96dedf66c)
This is an automated email from the ASF dual-hosted git repository. zhaocong pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git from 7a11edd4fda [fix][doc] Amend PIP voting process description (#19810) add 0e96dedf66c [fix][broker][PIP-195] Don't clean up BucketDelayedDeliveryTracker when all consumer disconnect (#19801) No new revisions were added by this update. Summary of changes: .../pulsar/broker/delayed/bucket/Bucket.java | 3 +- .../bucket/BucketDelayedDeliveryTracker.java | 4 ++ .../broker/delayed/bucket/ImmutableBucket.java | 67 -- .../broker/delayed/bucket/MutableBucket.java | 1 + .../PersistentDispatcherMultipleConsumers.java | 8 ++- .../bucket/BucketDelayedDeliveryTrackerTest.java | 18 ++ .../persistent/BucketDelayedDeliveryTest.java | 59 +++ 7 files changed, 128 insertions(+), 32 deletions(-)
[GitHub] [pulsar] coderzc merged pull request #19801: [fix][broker][PIP-195] Don't clean up BucketDelayedDeliveryTracker when all consumer disconnect
coderzc merged PR #19801: URL: https://github.com/apache/pulsar/pull/19801 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated (da3cab5289c -> 7a11edd4fda)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git from da3cab5289c [improve][broker][PIP-195] Cut off snapshot segment according to maxIndexesPerBucketSnapshotSegment (#19706) add 7a11edd4fda [fix][doc] Amend PIP voting process description (#19810) No new revisions were added by this update. Summary of changes: wiki/proposals/PIP.md | 11 ++- 1 file changed, 6 insertions(+), 5 deletions(-)
[GitHub] [pulsar] merlimat merged pull request #19810: [fix][doc] Amend PIP voting process description
merlimat merged PR #19810: URL: https://github.com/apache/pulsar/pull/19810 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] merlimat opened a new pull request, #19817: [fix][metadata-store] Fixed race condition between ResourceLock update and invalidation
merlimat opened a new pull request, #19817: URL: https://github.com/apache/pulsar/pull/19817 ### Motivation The test `LockManagerTest.updateValueWhenKeyDisappears()` can having infrequent failures, especially when running with Etcd backend. The problem is really in the logic of the `ResourceLock` itself. After a normal lock-acquire operation, the test is triggering 2 events at the same time: 1. A deletion of the lock key-value (eg: the ephemeral node might have disappeared) 2. A `lock.update()` operation to change the value of the lock. The race condition is due to the fact that operation (1) will trigger a lock revalidation in background and this now happens at the same time as operation (2). The result is that it's possible for the ResourceLock to enter into a loop of revalidations and deletions. eg: ``` 2023-03-14T20:40:40,350 - INFO - [-8-1:ResourceLockImpl@180] - Acquired resource lock on /key-9166073612375/1 2023-03-14T20:40:40,376 - INFO - [-8-1:ResourceLockImpl@201] - Lock on resource /key-9166073612375/1 was invalidated 2023-03-14T20:40:40,392 - INFO - [-8-1:ResourceLockImpl@180] - Acquired resource lock on /key-9166073612375/1 2023-03-14T20:40:40,392 - INFO - [-8-1:ResourceLockImpl@268] - Successfully re-acquired missing lock at /key-9166073612375/1 2023-03-14T20:40:40,392 - INFO - [-8-1:ResourceLockImpl@203] - Successfully revalidated the lock on /key-9166073612375/1 2023-03-14T20:40:40,402 - INFO - [-8-1:ResourceLockImpl@201] - Lock on resource /key-9166073612375/1 was invalidated 2023-03-14T20:40:40,407 - INFO - [-8-1:ResourceLockImpl@180] - Acquired resource lock on /key-9166073612375/1 2023-03-14T20:40:40,407 - INFO - [-8-1:ResourceLockImpl@325] - Successfully re-acquired lock at /key-9166073612375/1 2023-03-14T20:40:40,417 - INFO - [-8-1:ResourceLockImpl@201] - Lock on resource /key-9166073612375/1 was invalidated 2023-03-14T20:40:40,422 - INFO - [-8-1:ResourceLockImpl@180] - Acquired resource lock on /key-9166073612375/1 2023-03-14T20:40:40,422 - INFO - [-8-1:ResourceLockImpl@325] - Successfully re-acquired lock at /key-9166073612375/1 2023-03-14T20:40:40,422 - INFO - [-8-1:ResourceLockImpl@203] - Successfully revalidated the lock on /key-9166073612375/1 2023-03-14T20:40:40,433 - INFO - [-8-1:ResourceLockImpl@201] - Lock on resource /key-9166073612375/1 was invalidated 2023-03-14T20:40:40,437 - INFO - [-8-1:ResourceLockImpl@180] - Acquired resource lock on /key-9166073612375/1 2023-03-14T20:40:40,437 - INFO - [-8-1:ResourceLockImpl@325] - Successfully re-acquired lock at /key-9166073612375/1 2023-03-14T20:40:40,437 - INFO - [-8-1:ResourceLockImpl@203] - Successfully revalidated the lock on /key-9166073612375/1 2023-03-14T20:40:40,447 - INFO - [-8-1:ResourceLockImpl@201] - Lock on resource /key-9166073612375/1 was invalidated 2023-03-14T20:40:40,452 - INFO - [-8-1:ResourceLockImpl@180] - Acquired resource lock on /key-9166073612375/1 2023-03-14T20:40:40,452 - INFO - [-8-1:ResourceLockImpl@325] - Successfully re-acquired lock at /key-9166073612375/1 2023-03-14T20:40:40,452 - INFO - [-8-1:ResourceLockImpl@203] - Successfully revalidated the lock on /key-9166073612375/1 2023-03-14T20:40:40,467 - INFO - [-8-1:ResourceLockImpl@201] - Lock on resource /key-9166073612375/1 was invalidated 2023-03-14T20:40:40,472 - INFO - [-8-1:ResourceLockImpl@180] - Acquired resource lock on /key-9166073612375/1 2023-03-14T20:40:40,472 - INFO - [-8-1:ResourceLockImpl@325] - Successfully re-acquired lock at /key-9166073612375/1 2023-03-14T20:40:40,472 - INFO - [-8-1:ResourceLockImpl@203] - Successfully revalidated the lock on /key-9166073612375/1 2023-03-14T20:40:40,482 - INFO - [-8-1:ResourceLockImpl@201] - Lock on resource /key-9166073612375/1 was invalidated 2023-03-14T20:40:40,486 - INFO - [-8-1:ResourceLockImpl@180] - Acquired resource lock on /key-9166073612375/1 2023-03-14T20:40:40,486 - INFO - [-8-1:ResourceLockImpl@325] - Successfully re-acquired lock at /key-9166073612375/1 2023-03-14T20:40:40,487 - INFO - [-8-1:ResourceLockImpl@203] - Successfully revalidated the lock on /key-9166073612375/1 2023-03-14T20:40:40,497 - INFO - [-8-1:ResourceLockImpl@201] - Lock on resource /key-9166073612375/1 was invalidated ``` ### Modifications Similarly as to what we're already doing to prevent concurrent revalidate operations, we need to also wait for any existing operations to complete before attempting to do the update. The same is true if there is an update() operation in progress: the next revalidate should wait. We then use a single future to signal pending operation and to piggy back and try when it's the right time. ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following
[GitHub] [pulsar] yyj8 commented on pull request #19047: [improve][broker]PIP-214 Add broker level metrics statistics and expose to prometheus
yyj8 commented on PR #19047: URL: https://github.com/apache/pulsar/pull/19047#issuecomment-1469269932 > > @dao-jun could you share the link or PR number? > > #19793 Hi , @dao-jun ,@tisonkun When PR #19793 is merged, you will merge my PR , right? Is there anything else I need to do? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] momo-jun commented on issue #18190: [Bug] It's hard to find latest docs using search engines like Google
momo-jun commented on issue #18190: URL: https://github.com/apache/pulsar/issues/18190#issuecomment-1469269560 I encountered the same issue, and before reporting it here, I found it was filed in this thread. https://user-images.githubusercontent.com/60642177/225199843-b2dcd2a3-b375-4361-bf19-ccbd2490e814.png;> Not sure if there is a possible way to configure the canonical URLs globally across the site? @urfreespace do you have any idea? I think it would be better than configuring the front matter of each markdown file. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Nicklee007 closed pull request #19260: [fix][broker] Fix more times lookup request get the diff broker when set bundle affinity broker
Nicklee007 closed pull request #19260: [fix][broker] Fix more times lookup request get the diff broker when set bundle affinity broker URL: https://github.com/apache/pulsar/pull/19260 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-adapters] nareshv opened a new issue, #45: kafka record headers are not being handled
nareshv opened a new issue, #45: URL: https://github.com/apache/pulsar-adapters/issues/45 **Is your enhancement request related to a problem? Please describe.** Using this as a drop-in replacement for the KafkaProducer & KafkaConsumer coming from native kafka libraries. When the ProducerRecord is having any `Header` entries, they are not being sent to message, causing the consumer not able to access these headers **Describe the solution you'd like** Ideally, all the kafka record headers should be passed as-is so that the consumer can access them via KafkaConsumer interface. **Describe alternatives you've considered** None **Additional context** None -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Demogorgon314 commented on a diff in pull request #19773: [improve][broker] PIP-192: Make split admin API functional
Demogorgon314 commented on code in PR #19773:
URL: https://github.com/apache/pulsar/pull/19773#discussion_r1136487195
##
pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/extensions/channel/ServiceUnitStateChannelTest.java:
##
@@ -553,11 +552,24 @@ public void splitAndRetryTest() throws Exception {
}
// Call the real method
reset(namespaceService);
+doReturn(CompletableFuture.completedFuture(List.of("test-topic-1",
"test-topic-2")))
+
.when(namespaceService).getOwnedTopicListForNamespaceBundle(any());
return future;
}).when(namespaceService).updateNamespaceBundles(any(), any());
doReturn(namespaceService).when(pulsar1).getNamespaceService();
-
-Split split = new Split(bundle, ownerAddr1.get(), new HashMap<>());
Review Comment:
Updated.
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/channel/ServiceUnitStateData.java:
##
@@ -36,16 +39,22 @@ public record ServiceUnitStateData(
}
}
+public ServiceUnitStateData(ServiceUnitState state, String dstBroker,
String sourceBroker,
+Map>
splitServiceUnitToDestBroker, long versionId) {
+this(state, dstBroker, sourceBroker, splitServiceUnitToDestBroker,
false,
+System.currentTimeMillis(), versionId);
+}
+
public ServiceUnitStateData(ServiceUnitState state, String dstBroker,
String sourceBroker, long versionId) {
-this(state, dstBroker, sourceBroker, false,
System.currentTimeMillis(), versionId);
+this(state, dstBroker, sourceBroker, Map.of(), false,
System.currentTimeMillis(), versionId);
Review Comment:
Updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] dao-jun commented on pull request #19047: [improve][broker]PIP-214 Add broker level metrics statistics and expose to prometheus
dao-jun commented on PR #19047: URL: https://github.com/apache/pulsar/pull/19047#issuecomment-1469243968 > @dao-jun could you share the link or PR number? https://github.com/apache/pulsar/pull/19793 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] thetumbled commented on pull request #19816: [fix] [txn] fix race condition while update transaction state.
thetumbled commented on PR #19816: URL: https://github.com/apache/pulsar/pull/19816#issuecomment-1469241486 As the feature of idempotent commit need to be discussed further, I submit this PR to fix this bug solely. PTAL, thanks. @congbobo184 @poorbarcode -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] thetumbled opened a new pull request, #19816: [fix] [txn] fix race condition while update transaction state.
thetumbled opened a new pull request, #19816:
URL: https://github.com/apache/pulsar/pull/19816
### Motivation
There may be multiple commit requests arriving to brokers, but lines 385 to
388 in
`org.apache.pulsar.transaction.coordinator.impl.MLTransactionMetadataStore`
can't avoid race condition.
```
if (txnMetaListPair.getLeft().status() == newStatus) {
promise.complete(null);
return promise;
}
```
Two request may pass the check and try to update the transaction state,
which will throw exceptions like `Expect Txn (7,24726) to be in COMMITTING
status but it is in COMMITTED status`.
### Modifications
Check the transaction state in synchronized code block to avoid race
condition.
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
This change is a trivial rework / code cleanup without any test coverage.
### Does this pull request potentially affect one of the following parts:
*If the box was checked, please highlight the changes*
- [ ] Dependencies (add or upgrade a dependency)
- [ ] The public API
- [ ] The schema
- [ ] The default values of configurations
- [ ] The threading model
- [ ] The binary protocol
- [ ] The REST endpoints
- [ ] The admin CLI options
- [ ] The metrics
- [ ] Anything that affects deployment
### Documentation
- [ ] `doc`
- [ ] `doc-required`
- [ ] `doc-not-needed`
- [ ] `doc-complete`
### Matching PR in forked repository
PR in forked repository:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] congbobo184 commented on issue #19744: PIP-255: Support for Idempotent Commit and Abort.
congbobo184 commented on issue #19744: URL: https://github.com/apache/pulsar/issues/19744#issuecomment-1469217683 @thetumbled hi, I discussed with penghui, he said that because the txn in TopicTransactionBuffer has been recorded aborts txn(if transaction retention in tc, tc use memory will increase in the txn retention time), so may we should continue to discuss until the solution is perfect -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] momo-jun opened a new pull request, #470: [fix][doc] Remove CPP client lib requirements from Node.js client docs
momo-jun opened a new pull request, #470: URL: https://github.com/apache/pulsar-site/pull/470 Fixes https://github.com/apache/pulsar-client-node/issues/306. The dependency on the CPP library has been removed for the latest Node.js client library. The purpose of this fix is to: 1. set up simple and correct expectations for users and eliminate noise. 2. make sure the nitty gritty details are single-sourced in the README of the Node.js client lib. ### Modification 1. remove the out-of-date tips for requiring the CPP client lib. 2. paraphrase the backlink to the README of the Node.js client lib. ### Documentation - [x] `doc` - [ ] `doc-required` - [ ] `doc-not-needed` - [ ] `doc-complete` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] TakaHiR07 opened a new pull request, #19815: [fix] [txn] fix consumer can receive aborted txn message when readType is replay
TakaHiR07 opened a new pull request, #19815:
URL: https://github.com/apache/pulsar/pull/19815
Master Issue: #19814
### Motivation
When server try to dispatch message in
PersistentDispatcherMultipleConsumers, if consumer restart, the entries may
skip filterEntriesForConsumer() and directly go to "No consumers found with
available permits, storing {} positions for later replay".
Then when consumer replay entries, the entries would also skip
filterEntriesForConsumer(), so the aborted txn message would not be filtered.
### Modifications
also filter the aborted txn message in filterEntriesForConsumer() when
readType is replay
### Does this pull request potentially affect one of the following parts:
*If the box was checked, please highlight the changes*
- [ ] Dependencies (add or upgrade a dependency)
- [ ] The public API
- [ ] The schema
- [ ] The default values of configurations
- [ ] The threading model
- [ ] The binary protocol
- [ ] The REST endpoints
- [ ] The admin CLI options
- [ ] The metrics
- [ ] Anything that affects deployment
### Documentation
- [ ] `doc`
- [ ] `doc-required`
- [x] `doc-not-needed`
- [ ] `doc-complete`
### Matching PR in forked repository
PR in forked repository:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-client-node] shibd opened a new pull request, #307: [Fix] Wrap c++ exception when creating client.
shibd opened a new pull request, #307: URL: https://github.com/apache/pulsar-client-node/pull/307 Fixes #303 ### Motivation The `std::invalid_argument` exception causes the node process to exit directly, we should wrap it as a node exception that the user can catch. ### Modifications - Wrap c++ exception when creating client. ### Documentation - [ ] `doc-required` (Your PR needs to update docs and you will update later) - [x] `doc-not-needed` (Please explain why) - [ ] `doc` (Your PR contains doc changes) - [ ] `doc-complete` (Docs have been already added) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] BewareMyPower commented on pull request #466: Add the release note for Python client 3.1.0
BewareMyPower commented on PR #466: URL: https://github.com/apache/pulsar-site/pull/466#issuecomment-1469198814 @Anonymitaet Fixed. PTAL again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar-site] branch bewaremypower/release-note-py-3.1.0 updated (245b8fe4171 -> 2c32d570bae)
This is an automated email from the ASF dual-hosted git repository. xyz pushed a change to branch bewaremypower/release-note-py-3.1.0 in repository https://gitbox.apache.org/repos/asf/pulsar-site.git from 245b8fe4171 Add the release note for Python client 3.1.0 add 2c32d570bae Address comments No new revisions were added by this update. Summary of changes: release-notes/versioned/client-python-3.1.0.md | 8 1 file changed, 4 insertions(+), 4 deletions(-)
[GitHub] [pulsar] TakaHiR07 opened a new issue, #19814: [Bug] [txn] Consumer can receive txn aborted message when readType is replay
TakaHiR07 opened a new issue, #19814: URL: https://github.com/apache/pulsar/issues/19814 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Version server version: branch-2.9 client version: branch-2.9 ### Minimal reproduce step 1. txn produce large number of message, enabling batch. Transactions include commit and abort. 2. use shared consumer to do normal consume. 3. During the consume process, restart consumer several times. 4. See that consumer has consumed part of aborted messages. ### What did you expect to see? when enable transaction, consumer should not receive aborted message ### What did you see instead? consumer can receive aborted message , in the situation of shared subscription + batch message + consumer restart. ### Anything else? _No response_ ### Are you willing to submit a PR? - [X] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch branch-2.11 updated: [fix][meta] Fix close borrowed executor (#19761)
This is an automated email from the ASF dual-hosted git repository.
technoboy pushed a commit to branch branch-2.11
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/branch-2.11 by this push:
new 0c911dd4686 [fix][meta] Fix close borrowed executor (#19761)
0c911dd4686 is described below
commit 0c911dd4686cf9d2a483e4f3f2411db574a1e177
Author: Qiang Zhao
AuthorDate: Tue Mar 14 16:48:21 2023 +0800
[fix][meta] Fix close borrowed executor (#19761)
---
.../apache/pulsar/metadata/coordination/impl/LeaderElectionImpl.java| 2 --
1 file changed, 2 deletions(-)
diff --git
a/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/coordination/impl/LeaderElectionImpl.java
b/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/coordination/impl/LeaderElectionImpl.java
index 8e1d2cd4aa7..3dd58901f6a 100644
---
a/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/coordination/impl/LeaderElectionImpl.java
+++
b/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/coordination/impl/LeaderElectionImpl.java
@@ -253,8 +253,6 @@ class LeaderElectionImpl implements LeaderElection {
internalState = InternalState.Closed;
-executor.shutdownNow();
-
if (leaderElectionState != LeaderElectionState.Leading) {
return CompletableFuture.completedFuture(null);
}
[GitHub] [pulsar] nodece commented on pull request #19793: [fix] [test] Fix flaky test `MetadataStoreStatsTest.testBatchMetadataStoreMetrics`
nodece commented on PR #19793: URL: https://github.com/apache/pulsar/pull/19793#issuecomment-1469185569 /pulsarbot rerun-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch branch-2.11 updated: [fix][broker] Copy subscription properties during updating the topic partition number. (#19800)
This is an automated email from the ASF dual-hosted git repository.
technoboy pushed a commit to branch branch-2.11
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/branch-2.11 by this push:
new 0e32ab41161 [fix][broker] Copy subscription properties during updating
the topic partition number. (#19800)
0e32ab41161 is described below
commit 0e32ab4116126e672752ed0756cfdf27f13917b6
Author: Qiang Zhao
AuthorDate: Wed Mar 15 10:04:56 2023 +0800
[fix][broker] Copy subscription properties during updating the topic
partition number. (#19800)
---
.../broker/admin/impl/PersistentTopicsBase.java| 17 -
.../broker/admin/IncrementPartitionsTest.java | 41 +-
2 files changed, 49 insertions(+), 9 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
index 8e6f355af54..6a36ed9cff5 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
@@ -4414,17 +4414,18 @@ public class PersistentTopicsBase extends AdminResource
{
final String topicNamePartition =
topicName.getPartition(i).toString();
CompletableFuture future = new
CompletableFuture<>();
admin.topics().createSubscriptionAsync(topicNamePartition,
-subscription, MessageId.earliest,
replicated).whenComplete((__, ex) -> {
-if (ex == null) {
-future.complete(null);
-} else {
-if (ignoreConflictException
-&& ex instanceof
PulsarAdminException.ConflictException) {
+subscription, MessageId.earliest,
replicated, ss.getSubscriptionProperties())
+.whenComplete((__, ex) -> {
+if (ex == null) {
future.complete(null);
} else {
-future.completeExceptionally(ex);
+if (ignoreConflictException
+&& ex instanceof
PulsarAdminException.ConflictException) {
+future.complete(null);
+} else {
+future.completeExceptionally(ex);
+}
}
-}
});
subscriptionFutures.add(future);
}
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/IncrementPartitionsTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/IncrementPartitionsTest.java
index 0f21fe3f83b..d62ccc2ab1b 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/IncrementPartitionsTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/IncrementPartitionsTest.java
@@ -22,7 +22,10 @@ import static org.testng.Assert.assertEquals;
import com.google.common.collect.Lists;
import com.google.common.collect.Sets;
import java.util.Collections;
-
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
import lombok.Cleanup;
import org.apache.pulsar.broker.BrokerTestUtil;
import org.apache.pulsar.broker.admin.AdminApiTest.MockedPulsarService;
@@ -35,7 +38,9 @@ import org.apache.pulsar.client.api.Schema;
import org.apache.pulsar.common.naming.TopicName;
import org.apache.pulsar.common.policies.data.ClusterData;
import org.apache.pulsar.common.policies.data.TenantInfoImpl;
+import org.apache.pulsar.common.policies.data.TopicStats;
import org.awaitility.Awaitility;
+import org.testng.Assert;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;
@@ -61,6 +66,10 @@ public class IncrementPartitionsTest extends
MockedPulsarServiceBaseTest {
TenantInfoImpl tenantInfo = new
TenantInfoImpl(Sets.newHashSet("role1", "role2"), Sets.newHashSet("use"));
admin.tenants().createTenant("prop-xyz", tenantInfo);
admin.namespaces().createNamespace("prop-xyz/use/ns1");
+
+
+// Setup v2 namespaces
+setupDefaultTenantAndNamespace();
}
@AfterMethod(alwaysRun = true)
@@ -81,6 +90,36 @@ public class IncrementPartitionsTest extends
MockedPulsarServiceBaseTest {
assertEquals(admin.topics().getPartitionedTopicMetadata(partitionedTopicName).partitions,
20);
}
+
[GitHub] [pulsar] Technoboy- merged pull request #19800: [fix][broker] Copy subscription properties during updating the topic partition number.
Technoboy- merged PR #19800: URL: https://github.com/apache/pulsar/pull/19800 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] heesung-sn commented on a diff in pull request #19773: [improve][broker] PIP-192: Make split admin API functional
heesung-sn commented on code in PR #19773:
URL: https://github.com/apache/pulsar/pull/19773#discussion_r1136443035
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/channel/ServiceUnitStateData.java:
##
@@ -27,7 +29,8 @@
*/
public record ServiceUnitStateData(
-ServiceUnitState state, String dstBroker, String sourceBroker, boolean
force, long timestamp, long versionId) {
+ServiceUnitState state, String dstBroker, String sourceBroker,
+Map> splitServiceUnitToDestBroker, boolean
force, long timestamp, long versionId) {
Review Comment:
Ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] heesung-sn opened a new pull request, #19813: [improve][broker] PIP-192 Improved Auto Unload Logic
heesung-sn opened a new pull request, #19813: URL: https://github.com/apache/pulsar/pull/19813 Master Issue: Master Issue: https://github.com/apache/pulsar/issues/16691, https://github.com/apache/pulsar/issues/18099 ### Motivation Raising a PR to implement Master Issue: https://github.com/apache/pulsar/issues/16691, https://github.com/apache/pulsar/issues/18099 We want to reduce unload frequencies from flaky traffic. ### Modifications This PR - Introduced a config `loadBalancerSheddingConditionHitCountThreshold` to further restrict shedding conditions based on the hit count. - Lowered the default `loadBalanceSheddingDelayInSeconds` value from 600 to 180, as 10 mins are too long. 3 mins can be long enough to catch the new load after unloads. - Changed the config `loadBalancerBundleLoadReportPercentage` to `loadBalancerMaxNumberOfBundlesInBundleLoadReport` to make the topk bundle count absolute instead of relative. - Renamed `loadBalancerNamespaceBundleSplitConditionThreshold` to `loadBalancerNamespaceBundleSplitConditionHitCountThreshold` to be consistent with `*ConditionHitCountThreshold`. - Renamed `loadBalancerMaxNumberOfBrokerTransfersPerCycle ` to `loadBalancerMaxNumberOfBrokerSheddingPerCycle`. - Added LoadDataStore cleanup logic in BSC monitor. - Added `msgThroughputEMA` in BrokerLoadData to smooth the broker throughput info. - Updated Topk bundles sorted in a ascending order (instead of descending) - Update some info logs to only show in the debug mode. ### Verifying this change - [x] Make sure that the change passes the CI checks. This change added tests and can be verified as follows: - *Updated unit tests. ### Does this pull request potentially affect one of the following parts: *If the box was checked, please highlight the changes* - [ ] Dependencies (add or upgrade a dependency) - [ ] The public API - [ ] The schema - [ ] The default values of configurations - [ ] The threading model - [ ] The binary protocol - [ ] The REST endpoints - [ ] The admin CLI options - [ ] Anything that affects deployment ### Documentation - [ ] `doc` - [ ] `doc-required` - [x] `doc-not-needed` - [ ] `doc-complete` We will have separate PRs to update the Doc later. ### Matching PR in forked repository PR in forked repository: https://github.com/heesung-sn/pulsar/pull/39 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] Anonymitaet commented on a diff in pull request #468: [fix][doc] Add link to PIP to Contribute page
Anonymitaet commented on code in PR #468: URL: https://github.com/apache/pulsar-site/pull/468#discussion_r1136438339 ## contribute/about.md: ## @@ -26,7 +26,11 @@ If a quick [search](https://github.com/apache/pulsar/issues) doesn't turn up an If there’s an existing GitHub issue for your intended contribution, please comment about your intended work. Once the work is understood, a committer will assign the issue to you. If an issue is currently assigned, please check with the current assignee before reassigning. -For moderate or large contributions, you should not start coding or writing a design document unless there is a corresponding GitHub issue assigned to you for that work. Simple changes, like fixing typos, do not require an associated issue. +For moderate or large contributions, you should not start coding or writing a design document, a.k.a PIP (see below) unless there is a corresponding GitHub issue assigned to you for that work. Simple changes, like fixing typos, do not require an associated issue. + +### Pulsar Improvement Proposal (PIP) + +On any change which is either significant, or changes any public facing component of Pulsar (API, configuration, ...), the author should write design document known as Pulsar Improvement Proposal (PIP). There is a process for it that you should follow before starting to write one. It is [here]([url](https://github.com/apache/pulsar/blob/master/wiki/proposals/PIP.md)) Review Comment: ```suggestion On any change which is either significant (e.g., new features), or changes any public-facing component of Pulsar (API, configuration, metric ...), you need to write a design document known as a *Pulsar Improvement Proposal (PIP)*. Before starting to write a PIP, make sure you've read [Pulsar PIP Guideline](https://github.com/apache/pulsar/blob/master/wiki/proposals/PIP.md). Then you can submit one with the [Pulsar PIP template](https://github.com/apache/pulsar/issues/new?assignees==PIP=pip.yml=PIP-XYZ%3A+). ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar-site] branch main updated: [fix][doc] fix dev landing page link error (#469)
This is an automated email from the ASF dual-hosted git repository. junma pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new 57e87d69d6c [fix][doc] fix dev landing page link error (#469) 57e87d69d6c is described below commit 57e87d69d6cc07820bb858e0900e8bc18699e156 Author: Anonymitaet <50226895+anonymit...@users.noreply.github.com> AuthorDate: Wed Mar 15 09:31:03 2023 +0800 [fix][doc] fix dev landing page link error (#469) --- versioned_docs/version-2.11.x/about.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-2.11.x/about.md b/versioned_docs/version-2.11.x/about.md index e9217ad15ae..f5deb231a7f 100644 --- a/versioned_docs/version-2.11.x/about.md +++ b/versioned_docs/version-2.11.x/about.md @@ -28,7 +28,7 @@ Select one of the content blocks below to begin your Pulsar journey. If you ... - +
[GitHub] [pulsar-site] momo-jun merged pull request #469: [fix][doc] fix dev landing page link error
momo-jun merged PR #469: URL: https://github.com/apache/pulsar-site/pull/469 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] momo-jun closed issue #19809: [Doc] Developers landing page throws 404 Not Found error
momo-jun closed issue #19809: [Doc] Developers landing page throws 404 Not Found error URL: https://github.com/apache/pulsar/issues/19809 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] Anonymitaet commented on a diff in pull request #466: Add the release note for Python client 3.1.0
Anonymitaet commented on code in PR #466: URL: https://github.com/apache/pulsar-site/pull/466#discussion_r1136434184 ## release-notes/versioned/client-python-3.1.0.md: ## @@ -0,0 +1,15 @@ +## What's Changed +* Wrap the interruption to a custom exception when a blocking API is interrupted by @BewareMyPower in https://github.com/apache/pulsar-client-python/pull/99 +* Upgrade to pulsar-client-cpp 3.1.2 by @BewareMyPower in https://github.com/apache/pulsar-client-python/pull/96 +* Issue #37 : Allow passing pulsar.MessageId instance to create_reader() by @erichare in https://github.com/apache/pulsar-client-python/pull/95 Review Comment: Issue NO. should be in a PR description rather than in a PR title. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] Anonymitaet commented on a diff in pull request #466: Add the release note for Python client 3.1.0
Anonymitaet commented on code in PR #466: URL: https://github.com/apache/pulsar-site/pull/466#discussion_r1136434262 ## release-notes/versioned/client-python-3.1.0.md: ## @@ -0,0 +1,15 @@ +## What's Changed +* Wrap the interruption to a custom exception when a blocking API is interrupted by @BewareMyPower in https://github.com/apache/pulsar-client-python/pull/99 +* Upgrade to pulsar-client-cpp 3.1.2 by @BewareMyPower in https://github.com/apache/pulsar-client-python/pull/96 +* Issue #37 : Allow passing pulsar.MessageId instance to create_reader() by @erichare in https://github.com/apache/pulsar-client-python/pull/95 +* #66: Add Human-readable description of MessageId object by @erichare in https://github.com/apache/pulsar-client-python/pull/93 Review Comment: ```suggestion * Add Human-readable description of MessageId object by @erichare in https://github.com/apache/pulsar-client-python/pull/93 ``` ## release-notes/versioned/client-python-3.1.0.md: ## @@ -0,0 +1,15 @@ +## What's Changed +* Wrap the interruption to a custom exception when a blocking API is interrupted by @BewareMyPower in https://github.com/apache/pulsar-client-python/pull/99 +* Upgrade to pulsar-client-cpp 3.1.2 by @BewareMyPower in https://github.com/apache/pulsar-client-python/pull/96 +* Issue #37 : Allow passing pulsar.MessageId instance to create_reader() by @erichare in https://github.com/apache/pulsar-client-python/pull/95 +* #66: Add Human-readable description of MessageId object by @erichare in https://github.com/apache/pulsar-client-python/pull/93 +* Issue #31 - Access name attribute of any type object by @erichare in https://github.com/apache/pulsar-client-python/pull/92 +* Update the release process for versioning by @BewareMyPower in https://github.com/apache/pulsar-client-python/pull/91 +* Fixed deadlock in producer.send_async by @merlimat in https://github.com/apache/pulsar-client-python/pull/87 Review Comment: ```suggestion * Fix deadlock in producer.send_async by @merlimat in https://github.com/apache/pulsar-client-python/pull/87 ``` Keep consistent for tense ## release-notes/versioned/client-python-3.1.0.md: ## @@ -0,0 +1,15 @@ +## What's Changed +* Wrap the interruption to a custom exception when a blocking API is interrupted by @BewareMyPower in https://github.com/apache/pulsar-client-python/pull/99 +* Upgrade to pulsar-client-cpp 3.1.2 by @BewareMyPower in https://github.com/apache/pulsar-client-python/pull/96 +* Issue #37 : Allow passing pulsar.MessageId instance to create_reader() by @erichare in https://github.com/apache/pulsar-client-python/pull/95 Review Comment: Issue NO. should be in PR description instead of PR title. ## release-notes/versioned/client-python-3.1.0.md: ## @@ -0,0 +1,15 @@ +## What's Changed +* Wrap the interruption to a custom exception when a blocking API is interrupted by @BewareMyPower in https://github.com/apache/pulsar-client-python/pull/99 +* Upgrade to pulsar-client-cpp 3.1.2 by @BewareMyPower in https://github.com/apache/pulsar-client-python/pull/96 +* Issue #37 : Allow passing pulsar.MessageId instance to create_reader() by @erichare in https://github.com/apache/pulsar-client-python/pull/95 Review Comment: ```suggestion * Allow passing pulsar.MessageId instance to create_reader() by @erichare in https://github.com/apache/pulsar-client-python/pull/95 ``` ## release-notes/versioned/client-python-3.1.0.md: ## @@ -0,0 +1,15 @@ +## What's Changed +* Wrap the interruption to a custom exception when a blocking API is interrupted by @BewareMyPower in https://github.com/apache/pulsar-client-python/pull/99 +* Upgrade to pulsar-client-cpp 3.1.2 by @BewareMyPower in https://github.com/apache/pulsar-client-python/pull/96 +* Issue #37 : Allow passing pulsar.MessageId instance to create_reader() by @erichare in https://github.com/apache/pulsar-client-python/pull/95 +* #66: Add Human-readable description of MessageId object by @erichare in https://github.com/apache/pulsar-client-python/pull/93 +* Issue #31 - Access name attribute of any type object by @erichare in https://github.com/apache/pulsar-client-python/pull/92 Review Comment: ```suggestion * Access name attribute of any type object by @erichare in https://github.com/apache/pulsar-client-python/pull/92 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on issue #19809: [Doc] Developers landing page throws 404 Not Found error
Anonymitaet commented on issue #19809: URL: https://github.com/apache/pulsar/issues/19809#issuecomment-1469112881 @harikrishna553 Thanks for reporting this! I've fixed it in https://github.com/apache/pulsar-site/pull/469, PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] Anonymitaet opened a new pull request, #469: [fix][doc] fix dev landing page link error
Anonymitaet opened a new pull request, #469: URL: https://github.com/apache/pulsar-site/pull/469 Fix https://github.com/apache/pulsar/issues/19809 - [x] `doc` cc @momo-jun @DaveDuggins -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Demogorgon314 commented on a diff in pull request #19773: [improve][broker] PIP-192: Make split admin API functional
Demogorgon314 commented on code in PR #19773:
URL: https://github.com/apache/pulsar/pull/19773#discussion_r1136423905
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/channel/ServiceUnitStateData.java:
##
@@ -27,7 +29,8 @@
*/
public record ServiceUnitStateData(
-ServiceUnitState state, String dstBroker, String sourceBroker, boolean
force, long timestamp, long versionId) {
+ServiceUnitState state, String dstBroker, String sourceBroker,
+Map> splitServiceUnitToDestBroker, boolean
force, long timestamp, long versionId) {
Review Comment:
> Don't we need to keep the order of boundaries? Can it be a List?
No, we don't need to keep the order of boundaries. Yes, it can be a list,
but it cannot be duplicated.
> Can you explain why we have the dstBroker value? Can we remove the
dstBroker?
Do we need to assign the sub bundle directly to a dst broker and unload the
parent bundle? If we need this feature, it can be impl in the future.
> Currently, I think we only require split boundaries. Can we make it
List splitBoundaries?
If we need to support sub bundles direct assign, then we need to keep the
map.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-client-node] shibd commented on issue #306: Setup document is inaccurate
shibd commented on issue #306: URL: https://github.com/apache/pulsar-client-node/issues/306#issuecomment-1469075049 > I guess the setup instructions and notes should be updated? Yes > Which page plays the source role better regarding the installation? Pulsar doc page or README? I think we can use the [README#Getting Started](https://github.com/apache/pulsar-client-node#getting-started) part -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] lifepuzzlefun commented on a diff in pull request #19793: [fix] [test] Fix flaky test `MetadataStoreStatsTest.testBatchMetadataStoreMetrics`
lifepuzzlefun commented on code in PR #19793:
URL: https://github.com/apache/pulsar/pull/19793#discussion_r1136416131
##
pulsar-broker/src/test/java/org/apache/pulsar/broker/stats/MetadataStoreStatsTest.java:
##
@@ -69,6 +69,9 @@ protected void cleanup() throws Exception {
@Test
public void testMetadataStoreStats() throws Exception {
+// if this test fail see link below to find something.
Review Comment:
fixed
##
pulsar-broker/src/test/java/org/apache/pulsar/broker/zookeeper/MultiBrokerMetadataConsistencyTest.java:
##
@@ -43,6 +44,9 @@ protected int numberOfAdditionalBrokers() {
TestZKServer testZKServer;
+private final ConcurrentHashMap
needCloseStore =
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] lifepuzzlefun commented on a diff in pull request #19793: [fix] [test] Fix flaky test `MetadataStoreStatsTest.testBatchMetadataStoreMetrics`
lifepuzzlefun commented on code in PR #19793:
URL: https://github.com/apache/pulsar/pull/19793#discussion_r1136397077
##
pulsar-broker/src/test/java/org/apache/pulsar/broker/zookeeper/MultiBrokerMetadataConsistencyTest.java:
##
@@ -43,6 +44,9 @@ protected int numberOfAdditionalBrokers() {
TestZKServer testZKServer;
+private final ConcurrentHashMap
needCloseStore =
Review Comment:
will the test run concurrently？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] zymap opened a new pull request, #19812: Upgrade bookkeeper to 4.15.4
zymap opened a new pull request, #19812: URL: https://github.com/apache/pulsar/pull/19812 Fixes #xyz Master Issue: #xyz PIP: #xyz ### Motivation Upgrade bookkeeper to 4.15.4. ### Modifications Change bookkeeper version to 4.15.4 ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If the box was checked, please highlight the changes* - [ ] Dependencies (add or upgrade a dependency) - [ ] The public API - [ ] The schema - [ ] The default values of configurations - [ ] The threading model - [ ] The binary protocol - [ ] The REST endpoints - [ ] The admin CLI options - [ ] The metrics - [ ] Anything that affects deployment ### Documentation - [ ] `doc` - [ ] `doc-required` - [ ] `doc-not-needed` - [ ] `doc-complete` ### Matching PR in forked repository PR in forked repository: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] heesung-sn commented on pull request #19742: [improve][broker] PIP-192 Excluded bundles with isolation policy or anti-affinity-group policy from topk load bundles
heesung-sn commented on PR #19742: URL: https://github.com/apache/pulsar/pull/19742#issuecomment-1468857031 @Demogorgon314 @gaoran10 @Technoboy- I rebased the code. Please continue the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] heesung-sn commented on pull request #19730: [improve][broker] PIP-192 Made only the leader consume TopBundlesLoadDataStore
heesung-sn commented on PR #19730: URL: https://github.com/apache/pulsar/pull/19730#issuecomment-1468812284 @Demogorgon314 Please merge this PR if looking good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] heesung-sn commented on a diff in pull request #19773: [improve][broker] PIP-192: Make split admin API functional
heesung-sn commented on code in PR #19773:
URL: https://github.com/apache/pulsar/pull/19773#discussion_r1136042304
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/channel/ServiceUnitStateChannelImpl.java:
##
@@ -799,8 +801,29 @@ protected void
splitServiceUnitOnceAndRetry(NamespaceService namespaceService,
long startTime,
CompletableFuture
completionFuture) {
CompletableFuture> updateFuture = new
CompletableFuture<>();
+Map> bundleToDestBroker =
data.splitServiceUnitToDestBroker();
+// 1. verify the sub bundles is correct.
+if (bundleToDestBroker.size() != 2) {
Review Comment:
We should move this validation to the constructors if required.
```
public record Split(
String serviceUnit, String sourceBroker, Map> splitServiceUnitToDestBroker) {
public Split {
Objects.requireNonNull(serviceUnit);
}
}
public ServiceUnitStateData(ServiceUnitState state, String dstBroker, String
sourceBroker,
Map>
splitServiceUnitToDestBroker, long versionId) {
this(state, dstBroker, sourceBroker, splitServiceUnitToDestBroker,
false,
System.currentTimeMillis(), versionId);
}
```
Also, currently, split requests from the split scheduler do pass an empty
splitServiceUnitToDestBroker (or null), expecting split boundaries to be
auto-selected based on the configured algo.
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/channel/ServiceUnitStateChannelImpl.java:
##
@@ -799,8 +801,29 @@ protected void
splitServiceUnitOnceAndRetry(NamespaceService namespaceService,
long startTime,
CompletableFuture
completionFuture) {
CompletableFuture> updateFuture = new
CompletableFuture<>();
+Map> bundleToDestBroker =
data.splitServiceUnitToDestBroker();
+// 1. verify the sub bundles is correct.
+if (bundleToDestBroker.size() != 2) {
+String msg = format("Bundle %s should be split into 2 bundles, but
got %d bundles",
+serviceUnit, bundleToDestBroker.size());
+updateFuture.completeExceptionally(new
BrokerServiceException.ServiceUnitNotReadyException(msg));
+return;
+}
+String namespace = bundle.getNamespaceObject().toString();
+
+// 2. get the split boundary.
+Set boundaries = new HashSet<>();
+bundleToDestBroker.forEach((bundleRange, destBroker) -> {
+NamespaceBundle subBundle = bundleFactory.getBundle(namespace,
bundleRange);
+boundaries.add(subBundle.getKeyRange().lowerEndpoint());
+boundaries.add(subBundle.getKeyRange().upperEndpoint());
+});
-pulsar.getNamespaceService().getSplitBoundary(bundle,
null).thenAccept(splitBundlesPair -> {
+// 3. split the bundle use the specified boundary.
+NamespaceBundleSplitAlgorithm nsBundleSplitAlgorithm =
+NamespaceBundleSplitAlgorithm.SPECIFIED_POSITIONS_DIVIDE_ALGO;
+pulsar.getNamespaceService().getSplitBoundary(bundle,
nsBundleSplitAlgorithm, new ArrayList<>(boundaries))
Review Comment:
Again, the split scheduler does not pass the boundaries. It seems that this
change will break the split scheduler behavior.
##
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/ExtensibleLoadManagerImpl.java:
##
@@ -369,6 +378,53 @@ private CompletableFuture unloadAsync(Unload unload,
return unloadManager.waitAsync(future, unload.serviceUnit(), timeout,
timeoutUnit);
}
+public CompletableFuture splitNamespaceBundleAsync(ServiceUnitId
bundle,
+
NamespaceBundleSplitAlgorithm splitAlgorithm,
+ List
boundaries) {
+final String namespaceName =
LoadManagerShared.getNamespaceNameFromBundleName(bundle.toString());
+final String bundleRange =
LoadManagerShared.getBundleRangeFromBundleName(bundle.toString());
+NamespaceBundle namespaceBundle =
+
pulsar.getNamespaceService().getNamespaceBundleFactory().getBundle(namespaceName,
bundleRange);
+return pulsar.getNamespaceService().getSplitBoundary(namespaceBundle,
splitAlgorithm, boundaries)
+.thenCompose(splitBundlesPair -> {
+if (splitBundlesPair == null) {
+String msg = format("Bundle %s not found under
namespace", namespaceBundle);
+log.error(msg);
+return FutureUtil.failedFuture(new
IllegalStateException(msg));
+
[GitHub] [pulsar] heesung-sn commented on pull request #19730: [improve][broker] PIP-192 Made only the leader consume TopBundlesLoadDataStore
heesung-sn commented on PR #19730: URL: https://github.com/apache/pulsar/pull/19730#issuecomment-1468561722 > > @heesung-sn Please help rebase this PR to the master branch to resolve the conflicts. > > Lets merge the other PRs and rebase this PR again. @Demogorgon314 rebase is done. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] dao-jun opened a new pull request, #19811: [fix] Fix UnsupportedException for NonPersistentTopic when there are EntryFilters
dao-jun opened a new pull request, #19811: URL: https://github.com/apache/pulsar/pull/19811 ### Motivation In NonPersistentTopic, we dispatch messages to Consumers by using ``` subscription.getDispatcher().sendMessages(Collections.singletonList(entry)); ``` but Collections.SingleList.class didn't override `void set(int i, T element)` method, it extends `void set(int i, T element)` from AbstractList, and AbstractList.set(int i, T element) throws `UnsupportedException`. So the following usage will lead to `UnsupportedException`: ``` List objs = Collections.singletonList(new Object()); objs.set(0, null); ``` But in [AbstractBaseDispatcher#filterEntriesForConsumer](https://github.com/apache/pulsar/blob/master/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractBaseDispatcher.java#L153), if the filter result is REJECT/RESCHEDULE, it will filter out the entry by calling ``` entries.set(i, null); ``` and it will throw `UnsupportedException`, the exception will caught by [ServerCnx#exceptionCaught](https://github.com/apache/pulsar/blob/master/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java#L387). The producer will be closed and try to reconnect. ### Modifications ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If the box was checked, please highlight the changes* - [ ] Dependencies (add or upgrade a dependency) - [ ] The public API - [ ] The schema - [ ] The default values of configurations - [ ] The threading model - [ ] The binary protocol - [ ] The REST endpoints - [ ] The admin CLI options - [ ] The metrics - [ ] Anything that affects deployment ### Documentation - [ ] `doc` - [ ] `doc-required` - [x] `doc-not-needed` - [ ] `doc-complete` ### Matching PR in forked repository PR in forked repository: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-cpp] BewareMyPower commented on issue #209: [Flaky Test] ConnectionFailTest
BewareMyPower commented on issue #209: URL: https://github.com/apache/pulsar-client-cpp/issues/209#issuecomment-1468319279 It's still a flaky test, see [here](https://github.com/apache/pulsar-client-cpp/actions/runs/4416032164/jobs/7739779193). https://github.com/apache/pulsar-client-cpp/pull/211 fixes the race condition in `Client::close`, but there is another reason that might make this test fail. The root cause is the Boost resolver might stuck in `async_resolve`. We need to add a timeout to this method. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] asafm opened a new pull request, #19810: [fix][doc] Amend PIP voting process description
asafm opened a new pull request, #19810: URL: https://github.com/apache/pulsar/pull/19810 ### Motivation Clarify voting process ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. ### Does this pull request potentially affect one of the following parts: *If the box was checked, please highlight the changes* - [ ] Dependencies (add or upgrade a dependency) - [ ] The public API - [ ] The schema - [ ] The default values of configurations - [ ] The threading model - [ ] The binary protocol - [ ] The REST endpoints - [ ] The admin CLI options - [ ] The metrics - [ ] Anything that affects deployment ### Documentation - [ ] `doc` - [ ] `doc-required` - [ ] `doc-not-needed` - [ ] `doc-complete` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] harikrishna553 opened a new issue, #19809: [Doc] Developers landing page throws 404 Not Found error
harikrishna553 opened a new issue, #19809: URL: https://github.com/apache/pulsar/issues/19809 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### What issue do you find in Pulsar docs? Navigate to the page 'https://pulsar.apache.org/docs/2.11.x/' and click on 'Pulsar For Developers' link to check the issue ### What is your suggestion? Please point to the developers documentaiton ### Any reference? _No response_ ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-site] asafm opened a new pull request, #468: [fix][doc] Add link to PIP to Contribute page
asafm opened a new pull request, #468: URL: https://github.com/apache/pulsar-site/pull/468 Added short description of what is a PIP and liked to the page detailing it and its process ### Documentation - [ ] `doc` - [ ] `doc-required` - [ ] `doc-not-needed` - [ ] `doc-complete` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] mingyifei opened a new issue, #19808: [Bug]
mingyifei opened a new issue, #19808:
URL: https://github.com/apache/pulsar/issues/19808
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues)
and found nothing similar.
### Version
2.11
org.apache.bookkeeper.mledger.impl.OpFindNewest#readEntryComplete
### Minimal reproduce step
1.state = checkFirst, condition.apply(entry) = true
The following code is executed:
```
lastMatchedPosition = position;
// check last entry
state = State.checkLast;
PositionImpl lastPosition = ledger.getLastPosition();
searchPosition = ledger.getPositionAfterN(searchPosition,
max, PositionBound.startExcluded);
if (lastPosition.compareTo(searchPosition) < 0) {
if (log.isDebugEnabled()) {
log.debug("first position {} matches, last should be
{}, but moving to lastPos {}", position,
searchPosition, lastPosition);
}
searchPosition = lastPosition;
}
find();
```
crux of the problem：
The calculated searchPosition may be equal to the current entry position,
then, after calling find(), state=checkLast is executed the next time, this
causes condition.apply(entry) to be called twice, and with the same entry.
### What did you expect to see?
I'd like to add a check code, for example:
```
if (position.equals(searchPosition)) {
callback.findEntryComplete(position,
OpFindNewest.this.ctx);
return;
}
```
```
case checkFirst:
if (!condition.apply(entry)) {
// If no entry is found that matches the condition, it is
expected to pass null to the callback.
// Otherwise, a message before the expiration date will be
deleted due to message TTL.
// cf. https://github.com/apache/pulsar/issues/5579
callback.findEntryComplete(null, OpFindNewest.this.ctx);
return;
} else {
lastMatchedPosition = position;
// check last entry
state = State.checkLast;
PositionImpl lastPosition = ledger.getLastPosition();
searchPosition = ledger.getPositionAfterN(searchPosition,
max, PositionBound.startExcluded);
if (position.equals(searchPosition)) {
callback.findEntryComplete(position,
OpFindNewest.this.ctx);
return;
}
if (lastPosition.compareTo(searchPosition) < 0) {
if (log.isDebugEnabled()) {
log.debug("first position {} matches, last should be
{}, but moving to lastPos {}", position,
searchPosition, lastPosition);
}
searchPosition = lastPosition;
}
find();
}
break;
```
### What did you see instead?
condition.apply(entry) is executed twice
### Anything else?
no
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] nodece commented on pull request #19794: [fix][authentication][branch-2.10] Store the original authentication data
nodece commented on PR #19794: URL: https://github.com/apache/pulsar/pull/19794#issuecomment-1468276008 /pulsarbot rerun-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] nodece commented on a diff in pull request #19793: [fix] [test] Fix flaky test `MetadataStoreStatsTest.testBatchMetadataStoreMetrics`
nodece commented on code in PR #19793:
URL: https://github.com/apache/pulsar/pull/19793#discussion_r1135695448
##
pulsar-broker/src/test/java/org/apache/pulsar/broker/zookeeper/MultiBrokerMetadataConsistencyTest.java:
##
@@ -43,6 +44,9 @@ protected int numberOfAdditionalBrokers() {
TestZKServer testZKServer;
+private final ConcurrentHashMap
needCloseStore =
Review Comment:
Could you use the List?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] nodece commented on a diff in pull request #19793: [fix] [test] Fix flaky test `MetadataStoreStatsTest.testBatchMetadataStoreMetrics`
nodece commented on code in PR #19793:
URL: https://github.com/apache/pulsar/pull/19793#discussion_r1135694755
##
pulsar-broker/src/test/java/org/apache/pulsar/broker/stats/MetadataStoreStatsTest.java:
##
@@ -69,6 +69,9 @@ protected void cleanup() throws Exception {
@Test
public void testMetadataStoreStats() throws Exception {
+// if this test fail see link below to find something.
Review Comment:
It looks unnecessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] jak78 edited a comment on the discussion: How to start to understand and learn this technology?
GitHub user jak78 edited a comment on the discussion: How to start to understand and learn this technology? Hi @XiaoYaoheihei, and welcome to the community! I would recommend enrolling in courses from [StreamNative Academy](https://www.academy.streamnative.io/tracks) and you can start with this one: [Introduction to Apache Pulsar](https://www.academy.streamnative.io/courses/course-v1:streamnative+Beg-001+2022/about) You can also watch videos from their [YouTube channel](https://www.youtube.com/playlist?list=PLqRma1oIkcWjkpm1-0ox-BFJdVBsgZwPU). You can also read the [Apache Pulsar in Action book](https://streamnative.io/ebooks/get-your-free-copy-of-mannings-apache-pulsar-in-action). If you want to 'feel' Pulsar by trying it out, you can follow [one of these Getting Started guides](https://pulsar.apache.org/docs/2.11.x/getting-started-home/). Also, @visortelle has built an amazing list of curated Pulsar resources: https://github.com/tealtools/awesome-apache-pulsar Don't hesitate to reach out if you have any questions during your journey through Pulsar. I'd be happy to help GitHub link: https://github.com/apache/pulsar/discussions/19807#discussioncomment-5306215 This is an automatically sent email for commits@pulsar.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@pulsar.apache.org
[GitHub] [pulsar] jak78 edited a comment on the discussion: How to start to understand and learn this technology?
GitHub user jak78 edited a comment on the discussion: How to start to understand and learn this technology? Hi @XiaoYaoheihei, and welcome to the community! I would recommend enrolling in courses from [StreamNative Academy](https://www.academy.streamnative.io/tracks) and you can start with this one: [Introduction to Apache Pulsar](https://www.academy.streamnative.io/courses/course-v1:streamnative+Beg-001+2022/about) You can also watch videos from their [YouTube channel](https://www.youtube.com/playlist?list=PLqRma1oIkcWjkpm1-0ox-BFJdVBsgZwPU). You can also read the [Apache Pulsar in Action book](https://streamnative.io/ebooks/get-your-free-copy-of-mannings-apache-pulsar-in-action). If you want to 'feel' Pulsar by trying it out, you can follow [one of these Getting Started guides](https://pulsar.apache.org/docs/2.11.x/getting-started-home/). Also, @visortelle has built an amazing list of curated Pulsar resources: https://github.com/tealtools/awesome-apache-pulsar Don't hesitate to reach out if you have any questions during your journey through Pulsar. I'd be happy to help GitHub link: https://github.com/apache/pulsar/discussions/19807#discussioncomment-5306215 This is an automatically sent email for commits@pulsar.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@pulsar.apache.org
[GitHub] [pulsar] jak78 edited a comment on the discussion: How to start to understand and learn this technology?
GitHub user jak78 edited a comment on the discussion: How to start to understand and learn this technology? Hi @XiaoYaoheihei, and welcome to the community! I would recommend enrolling in courses from [StreamNative Academy](https://www.academy.streamnative.io/tracks) and you can start with this one: [Introduction to Apache Pulsar](https://www.academy.streamnative.io/courses/course-v1:streamnative+Beg-001+2022/about) You can also go more in-depth and watch the other videos from StreamNative or read the [Apache Pulsar in Action book](https://streamnative.io/ebooks/get-your-free-copy-of-mannings-apache-pulsar-in-action). If you want to 'feel' Pulsar by trying it out, you can follow [one of these Getting Started guides](https://pulsar.apache.org/docs/2.11.x/getting-started-home/). Also, @visortelle has built an amazing list of curated Pulsar resources: https://github.com/tealtools/awesome-apache-pulsar Don't hesitate to reach out if you have any questions during your journey through Pulsar. I'd be happy to help GitHub link: https://github.com/apache/pulsar/discussions/19807#discussioncomment-5306215 This is an automatically sent email for commits@pulsar.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@pulsar.apache.org
[GitHub] [pulsar-client-cpp] BewareMyPower closed issue #212: [Flaky Test] TableViewTest.testCreateTableView
BewareMyPower closed issue #212: [Flaky Test] TableViewTest.testCreateTableView URL: https://github.com/apache/pulsar-client-cpp/issues/212 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar-client-cpp] branch main updated: Fix testPublishEmptyValue flaky test. (#216)
This is an automated email from the ASF dual-hosted git repository.
xyz pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-client-cpp.git
The following commit(s) were added to refs/heads/main by this push:
new 81830b7 Fix testPublishEmptyValue flaky test. (#216)
81830b7 is described below
commit 81830b7ed94b9b1899be53ddff1066b8b29f2ce9
Author: Baodi Shi
AuthorDate: Tue Mar 14 22:45:30 2023 +0800
Fix testPublishEmptyValue flaky test. (#216)
---
tests/TableViewTest.cc | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tests/TableViewTest.cc b/tests/TableViewTest.cc
index 9f0ffe9..91a16bb 100644
--- a/tests/TableViewTest.cc
+++ b/tests/TableViewTest.cc
@@ -67,7 +67,7 @@ TEST(TableViewTest, testCreateTableView) {
}
TEST(TableViewTest, testSimpleTableView) {
-const std::string topic = "testTableView" + std::to_string(time(nullptr));
+const std::string topic = "testSimpleTableView" +
std::to_string(time(nullptr));
Client client(lookupUrl);
ProducerConfiguration producerConfiguration;
@@ -114,7 +114,7 @@ TEST(TableViewTest, testSimpleTableView) {
}
TEST(TableViewTest, testPublishEmptyValue) {
-const std::string topic = "testTableView" + std::to_string(time(nullptr));
+const std::string topic = "testPublishEmptyValue" +
std::to_string(time(nullptr));
Client client(lookupUrl);
ProducerConfiguration producerConfiguration;
[GitHub] [pulsar-client-cpp] BewareMyPower merged pull request #216: Fix testPublishEmptyValue flaky test.
BewareMyPower merged PR #216: URL: https://github.com/apache/pulsar-client-cpp/pull/216 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-cpp] BewareMyPower closed issue #215: [Flaky Test] TableViewTest.testPublishEmptyValue
BewareMyPower closed issue #215: [Flaky Test] TableViewTest.testPublishEmptyValue URL: https://github.com/apache/pulsar-client-cpp/issues/215 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] jak78 added a comment to the discussion: How to start to understand and learn this technology?
GitHub user jak78 added a comment to the discussion: How to start to understand and learn this technology? Hi @XiaoYaoheihei, and welcome to the community! I would recommend watching some videos from [StreamNative Academy](https://www.academy.streamnative.io/tracks) and you can start with this one: [Introduction to Apache Pulsar](https://www.academy.streamnative.io/courses/course-v1:streamnative+Beg-001+2022/about) You can also go more in-depth and watch the other videos from StreamNative or read the [Apache Pulsar in Action book](https://streamnative.io/ebooks/get-your-free-copy-of-mannings-apache-pulsar-in-action). If you want to 'feel' Pulsar by trying it out, you can follow [one of these Getting Started guides](https://pulsar.apache.org/docs/2.11.x/getting-started-home/). Also, @visortelle has built an amazing list of curated Pulsar resources: https://github.com/tealtools/awesome-apache-pulsar Don't hesitate to reach out if you have any questions during your journey through Pulsar. I'd be happy to help GitHub link: https://github.com/apache/pulsar/discussions/19807#discussioncomment-5306215 This is an automatically sent email for commits@pulsar.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@pulsar.apache.org
[pulsar] branch master updated: [improve][broker][PIP-195] Cut off snapshot segment according to maxIndexesPerBucketSnapshotSegment (#19706)
This is an automated email from the ASF dual-hosted git repository. zhaocong pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new da3cab5289c [improve][broker][PIP-195] Cut off snapshot segment according to maxIndexesPerBucketSnapshotSegment (#19706) da3cab5289c is described below commit da3cab5289c41cfe1e064b02ce4794224aeaea04 Author: Cong Zhao AuthorDate: Tue Mar 14 22:06:21 2023 +0800 [improve][broker][PIP-195] Cut off snapshot segment according to maxIndexesPerBucketSnapshotSegment (#19706) --- conf/broker.conf | 7 ++-- conf/standalone.conf | 42 ++ .../apache/pulsar/broker/ServiceConfiguration.java | 12 --- .../BucketDelayedDeliveryTrackerFactory.java | 8 +++-- .../bucket/BucketDelayedDeliveryTracker.java | 19 ++ .../broker/delayed/bucket/MutableBucket.java | 10 -- .../bucket/BucketDelayedDeliveryTrackerTest.java | 41 ++--- .../persistent/BucketDelayedDeliveryTest.java | 1 + 8 files changed, 109 insertions(+), 31 deletions(-) diff --git a/conf/broker.conf b/conf/broker.conf index 4b7c108be5f..d52adb25456 100644 --- a/conf/broker.conf +++ b/conf/broker.conf @@ -571,13 +571,14 @@ delayedDeliveryMinIndexCountPerBucket=5 # after reaching the max time step limitation, the snapshot segment will be cut off. delayedDeliveryMaxTimeStepPerBucketSnapshotSegmentSeconds=300 +# The max number of delayed message index in per bucket snapshot segment, -1 means no limitation +# after reaching the max number limitation, the snapshot segment will be cut off. +delayedDeliveryMaxIndexesPerBucketSnapshotSegment=5000 + # The max number of delayed message index bucket, # after reaching the max buckets limitation, the adjacent buckets will be merged. delayedDeliveryMaxNumBuckets=50 -# Enable share the delayed message index across subscriptions -delayedDeliverySharedIndexEnabled=false - # Size of the lookahead window to use when detecting if all the messages in the topic # have a fixed delay. # Default is 50,000. Setting the lookahead window to 0 will disable the logic to handle diff --git a/conf/standalone.conf b/conf/standalone.conf index ed883406883..f141946c29f 100644 --- a/conf/standalone.conf +++ b/conf/standalone.conf @@ -1223,3 +1223,45 @@ configurationStoreServers= # zookeeper. # Deprecated: use managedLedgerMaxUnackedRangesToPersistInMetadataStore managedLedgerMaxUnackedRangesToPersistInZooKeeper=-1 + +# Whether to enable the delayed delivery for messages. +# If disabled, messages will be immediately delivered and there will +# be no tracking overhead. +delayedDeliveryEnabled=true + +# Class name of the factory that implements the delayed deliver tracker. +# If value is "org.apache.pulsar.broker.delayed.BucketDelayedDeliveryTrackerFactory", +# will create bucket based delayed message index tracker. +delayedDeliveryTrackerFactoryClassName=org.apache.pulsar.broker.delayed.InMemoryDelayedDeliveryTrackerFactory + +# Control the tick time for when retrying on delayed delivery, +# affecting the accuracy of the delivery time compared to the scheduled time. +# Note that this time is used to configure the HashedWheelTimer's tick time for the +# InMemoryDelayedDeliveryTrackerFactory (the default DelayedDeliverTrackerFactory). +# Default is 1 second. +delayedDeliveryTickTimeMillis=1000 + +# When using the InMemoryDelayedDeliveryTrackerFactory (the default DelayedDeliverTrackerFactory), whether +# the deliverAt time is strictly followed. When false (default), messages may be sent to consumers before the deliverAt +# time by as much as the tickTimeMillis. This can reduce the overhead on the broker of maintaining the delayed index +# for a potentially very short time period. When true, messages will not be sent to consumer until the deliverAt time +# has passed, and they may be as late as the deliverAt time plus the tickTimeMillis for the topic plus the +# delayedDeliveryTickTimeMillis. +isDelayedDeliveryDeliverAtTimeStrict=false + +# The delayed message index bucket min index count. +# When the index count of the current bucket is more than this value and all message indexes of current ledger +# have already been added to the tracker we will seal the bucket. +delayedDeliveryMinIndexCountPerBucket=5 + +# The delayed message index bucket time step(in seconds) in per bucket snapshot segment, +# after reaching the max time step limitation, the snapshot segment will be cut off. +delayedDeliveryMaxTimeStepPerBucketSnapshotSegmentSeconds=300 + +# The max number of delayed message index in per bucket snapshot segment, -1 means no limitation +# after reaching the max number limitation, the snapshot segment will be cut off. +delayedDeliveryMaxIndexesPerBucketSnapshotSegment=5000 + +# The max number of delayed message
[GitHub] [pulsar] coderzc merged pull request #19706: [improve][broker][PIP-195] Cut off snapshot segment according to maxIndexesPerBucketSnapshotSegment
coderzc merged PR #19706: URL: https://github.com/apache/pulsar/pull/19706 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-cpp] shibd opened a new pull request, #216: Fix testPublishEmptyValue flaky test.
shibd opened a new pull request, #216: URL: https://github.com/apache/pulsar-client-cpp/pull/216 Fixes #215 ### Motivation The root cause is topic name is duplicated. ### Modifications - Each test uses a separate topic name. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-go] shibd commented on a diff in pull request #976: [feat]: Support auto scaled consumer receiver queue
shibd commented on code in PR #976:
URL: https://github.com/apache/pulsar-client-go/pull/976#discussion_r1135535670
##
pulsar/consumer_test.go:
##
@@ -4030,9 +4030,6 @@ func TestConsumerWithAutoScaledQueueReceive(t *testing.T)
{
_, err = c.Receive(context.Background())
assert.Nil(t, err)
- // waiting for prefetched message passing from queueCh to dispatcher()
- time.Sleep(time.Second)
Review Comment:
You can use `retryAssert` instead of sleep.
https://github.com/apache/pulsar-client-go/blob/e2ea255052e8a527091791ef368851d885ee2d45/pulsar/helper_for_test.go#L176-L185
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[pulsar-client-cpp] branch main updated: Fix Client::close might hang forever for a multi-topics consumer (#211)
This is an automated email from the ASF dual-hosted git repository.
baodi pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-client-cpp.git
The following commit(s) were added to refs/heads/main by this push:
new b7fecc3 Fix Client::close might hang forever for a multi-topics
consumer (#211)
b7fecc3 is described below
commit b7fecc3fa3de7752bf6d594b3792b0ea516b309d
Author: Yunze Xu
AuthorDate: Tue Mar 14 20:53:37 2023 +0800
Fix Client::close might hang forever for a multi-topics consumer (#211)
* Fix Client::close might hang forever for a multi-topics consumer
Fixes https://github.com/apache/pulsar-client-cpp/issues/209
### Motivation
There is a race condition when `Client::close` is called after a
`MultiTopicsConsumerImpl` instance is destructed. e.g.
```c++
{
Consumer consumer;
// subscribe multiple topics or partitions
client.subscribe(/* ... */, consumer);
} // consumer will be destructed after going out of the scope here
client.close();
```
There is a chance that when `client.close()` is called first, the
destructor of `consumer` is not called, so `client.consumers_` is
not empty and `ClientImpl::closeAsync` will wait until the
`MultiTopicsConsumerImpl::closeAsync` completes. Then, in its callback
https://github.com/apache/pulsar-client-cpp/blob/242ad770f76655d92c5c1d1abfdee1319ba2ebdc/lib/MultiTopicsConsumerImpl.cc#L482
the original callback (`ClientImpl::handleClose`) will be skipped and
`ClientImpl::closeAsync` will never complete.
It's also the root cause of the flaky `ConnectionFailTest`, we can see
each time it stuck for 120 minutes, there must be the following log:
```
ClientImpl:589 | Closing Pulsar client with 0 producers and 1 consumers
```
When the test passed, the log was:
```
ClientImpl:589 | Closing Pulsar client with 0 producers and 0 consumers
```
### Modifications
In `MultiTopicsConsumerImpl::closeAsync`, do not capture `weakSelf` in
the callback that is passed to `consumer->closeAsync`. Instead, capture
the `weakSelf` in the wrapped `callback` and modify the `state_` field
here. Since now `callback` (and `originalCallback`) will never be
skipped, `ClientImpl::close` will not be blocked forever.
### Verifications
Run the `ConnectionFailTest` inside a `ubuntu:22.04` docker container
until the following log appeared.
```
ClientImpl:589 | Closing Pulsar client with 0 producers and 1 consumers
```
We can see now the test still passed.
```
2023-03-14 06:51:39.201 INFO [140309830422656] ClientImpl:553 | Closing
Pulsar client with 0 producers and 1 consumers
2023-03-14 06:51:39.201 INFO [140309830422656] ConsumerImpl:1188 |
[persistent://public/default/test-connection-fail-51678776698-partition-4,
reader-e0617a7614, 9] Closing consumer for topic
persistent://public/default/test-connection-fail-51678776698-partition-4
2023-03-14 06:51:39.202 INFO [140309830422656] ConsumerImpl:1188 |
[persistent://public/default/test-connection-fail-51678776698-partition-3,
reader-e0617a7614, 8] Closing consumer for topic
persistent://public/default/test-connection-fail-51678776698-partition-3
2023-03-14 06:51:39.202 INFO [140309830422656] ConsumerImpl:1188 |
[persistent://public/default/test-connection-fail-51678776698-partition-2,
reader-e0617a7614, 7] Closing consumer for topic
persistent://public/default/test-connection-fail-51678776698-partition-2
2023-03-14 06:51:39.202 INFO [140309830422656] ConsumerImpl:1188 |
[persistent://public/default/test-connection-fail-51678776698-partition-1,
reader-e0617a7614, 6] Closing consumer for topic
persistent://public/default/test-connection-fail-51678776698-partition-1
2023-03-14 06:51:39.203 INFO [140309830422656] ConsumerImpl:1188 |
[persistent://public/default/test-connection-fail-51678776698-partition-0,
reader-e0617a7614, 5] Closing consumer for topic
persistent://public/default/test-connection-fail-51678776698-partition-0
2023-03-14 06:51:39.250 INFO [140309602235968] ConsumerImpl:1174 |
[persistent://public/default/test-connection-fail-51678776698-partition-4,
reader-e0617a7614, 9] Closed consumer 9
2023-03-14 06:51:39.250 INFO [140309602235968] ConsumerImpl:1174 |
[persistent://public/default/test-connection-fail-51678776698-partition-3,
reader-e0617a7614, 8] Closed consumer 8
2023-03-14 06:51:39.251 INFO [140309602235968] ConsumerImpl:1174 |
[persistent://public/default/test-connection-fail-51678776698-partition-2,
reader-e0617a7614, 7] Closed consumer 7
2023-03-14 06:51:39.251 INFO [140309602235968] ConsumerImpl:1174 |
[persistent://public/default/test-connection-fail-51678776698-partition-1,
reader-e0617a7614, 6] Closed consumer 6
2023-03-14 06:51:39.251 INFO
[GitHub] [pulsar-client-cpp] shibd closed issue #209: [Flaky Test] ConnectionFailTest
shibd closed issue #209: [Flaky Test] ConnectionFailTest URL: https://github.com/apache/pulsar-client-cpp/issues/209 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-cpp] shibd merged pull request #211: Fix Client::close might hang forever for a multi-topics consumer
shibd merged PR #211: URL: https://github.com/apache/pulsar-client-cpp/pull/211 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] 315157973 opened a new issue, #19806: PIP-XYZ: Assign topic partitions to bundles by round robin
315157973 opened a new issue, #19806: URL: https://github.com/apache/pulsar/issues/19806 ### Motivation With all existing Pulsar load balancing algorithms, it is difficult to balance the load of Pulsar cluster nodes. It often happens that some nodes are highly loaded while others are idle, and the CPU of each Broker node is very different.  There are three reasons why the existing way will make the cluster unbalanced: 1. The load managed by each Bundle is not even. Even if the number of partitions managed by each bundle is the same, there is no guarantee that the sum of the loads of these partitions will be the same. 2. Doesn't shed loads very well. The existing default policy ThresholdShedder has a relatively high usage threshold, and various traffic thresholds need to be set. Many clusters with high TPS and small message bodies may have high CPU but low traffic; And for many small-scale clusters, the threshold needs to be modified according to the actual business. 3. The removed Bundle cannot be well distributed to other Brokers. The load information of each Broker will be reported at regular intervals, so the judgment of the Leader Broker when allocating Bundles cannot be guaranteed to be completely correct. Secondly, if there are a large number of Bundles to be redistributed, the Leader may make the low-load Broker a new high-load node when the load information is not up-to-date. ### Goal We reimplemented the algorithm in 3 parts and listed our test report at the end: 1. Assigning partitions to Bundle. 2. Allocating Bundle to Broker. 3. The algorithm for the leader to decide which Bundles on the high-load Broker to offload. In this PIP, we only discuss the first one: `Assigning partitions to Bundle`. Let each bundle carry the same load as possible. ### API Changes 1. Add a configuration item `partitionAssignerClassName`, so that different partition assignment algorithms can be dynamically configured. The existing algorithm will be used as the default `partitionAssignerClassName=ConsistentHashingPartitionAssigner` 3. Implement a new partition assignment class `RoundRobinPartitionAssigner` ### Implementation The client sends a message to a multi-partition Topic, which uses polling by default. Therefore, we believe that the load of partitions of the same topic is balanced. We assign partitions of the same topic to bundle by round-robin. In this way, the difference in the number of partitions carried by the bundle will not exceed 1. Since we consider the load of each partition of the same topic to be balanced, the load carried by each bundle is also balanced. https://user-images.githubusercontent.com/9758905/224997861-9bb9e1ef-1439-4f67-a3f7-53d9b6e1e5da.png; width="500" height="500"> Operation steps: 1. Partition 0 finds a starting bundle through the consistent hash algorithm, assuming it is bundle0, we start from this bundle 2. By round-robin, assign partition 1 to the next bundle1, assign partition 2 to the next bundle2, and so on If the number of partitions is less than the number of bundles, will some bundles have a high load? Since the starting bundle is determined by consistent hashing, the starting point of each topic is different, which can prevent the earlier bundles from becoming hotspots. When the number of bundles changes, will all partitions be reassigned? Only when the number of bundles change, all partitions under the same namespace will be reassigned. Changing the number of broker or partitions, will not trigger reassignment. We usually only split when there is a hot bundle. The current partition assign method makes the load of each bundle approximately balanced, so the bundle split will not be triggered unless it is artificially split. Of course, we have also tested the time-consuming of assigning all partitions in the entire namespace in the worst case. Test scenario: 6 * 4C32GB Brokers, the CPU water level of each broker is at 60%, and 50,000 partitions under the namespace are assigned at the same time. It takes about 30s.  ### Security Considerations None ### Alternatives _No response_ ### Anything else? Test report We tested several scenarios, each using the three algorithms we mentioned above. And every node looks well balanced. Machine: 4C32G * 6 Test scenario: 200 partitions, after the cluster is stable, restart one of them  Test scenario: Restart multiple nodes in a loop,
[GitHub] [pulsar-client-go] shibd commented on a diff in pull request #976: [feat]: Support auto scaled consumer receiver queue
shibd commented on code in PR #976:
URL: https://github.com/apache/pulsar-client-go/pull/976#discussion_r1135485766
##
pulsar/consumer_test.go:
##
@@ -4030,9 +4030,6 @@ func TestConsumerWithAutoScaledQueueReceive(t *testing.T)
{
_, err = c.Receive(context.Background())
assert.Nil(t, err)
- // waiting for prefetched message passing from queueCh to dispatcher()
- time.Sleep(time.Second)
Review Comment:
CI failed, it seems we can't remove this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-client-go] panszobe commented on issue #949: Consume Performance drops when set EnableBatchIndexAcknowledgment = true
panszobe commented on issue #949: URL: https://github.com/apache/pulsar-client-go/issues/949#issuecomment-1468005099 New PR merged, I will take a test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-cpp] lalit-g-deepr closed issue #205: the C++ clients don’t have isClosed() method like the java client does for testing connection with pulsar server. Can is it implemented fo
lalit-g-deepr closed issue #205: the C++ clients don’t have isClosed() method like the java client does for testing connection with pulsar server. Can is it implemented for C++ client? URL: https://github.com/apache/pulsar-client-cpp/issues/205 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-cpp] lalit-g-deepr commented on issue #205: the C++ clients don’t have isClosed() method like the java client does for testing connection with pulsar server. Can is it implemen
lalit-g-deepr commented on issue #205: URL: https://github.com/apache/pulsar-client-cpp/issues/205#issuecomment-1467963171 ok @BewareMyPower got your point. Closing the request. Thanks for taking time out. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] coderzc commented on issue #16497: Flaky-test: ManagedLedgerCompressionTest. testRestartBrokerEnableManagedLedgerInfoCompression
coderzc commented on issue #16497: URL: https://github.com/apache/pulsar/issues/16497#issuecomment-1467928209 another one: https://github.com/apache/pulsar/actions/runs/4413610159/jobs/7734530087 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] coderzc commented on a diff in pull request #19802: [improve][broker][PIP-195] Add topicName and cursorName for ledger metadata of bucket snapshot
coderzc commented on code in PR #19802:
URL: https://github.com/apache/pulsar/pull/19802#discussion_r1135399086
##
managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/LedgerMetadataUtils.java:
##
@@ -48,7 +48,9 @@ public final class LedgerMetadataUtils {
private static final String METADATA_PROPERTY_COMPACTEDTO =
"pulsar/compactedTo";
private static final String METADATA_PROPERTY_SCHEMAID = "pulsar/schemaId";
-private static final String METADATA_PROPERTY_DELAYED_INDEX_BUCKETID =
"pulsar/delayedIndexBucketId";
+private static final String METADATA_PROPERTY_DELAYED_INDEX_BUCKET_KEY =
"pulsar/delayedIndexBucketKey";
Review Comment:
No, we don't read metadata directly, it only used to troubleshoot issues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-client-python] BewareMyPower merged pull request #105: Bumped version to 3.2.0a1
BewareMyPower merged PR #105: URL: https://github.com/apache/pulsar-client-python/pull/105 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar-client-python] branch main updated: Bumped version to 3.2.0a1 (#105)
This is an automated email from the ASF dual-hosted git repository. xyz pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-client-python.git The following commit(s) were added to refs/heads/main by this push: new a6476d9 Bumped version to 3.2.0a1 (#105) a6476d9 is described below commit a6476d9c45508f55a7af4b25001038a8e3a27489 Author: Yunze Xu AuthorDate: Tue Mar 14 19:11:45 2023 +0800 Bumped version to 3.2.0a1 (#105) --- pulsar/__about__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pulsar/__about__.py b/pulsar/__about__.py index a66d247..1c296a7 100644 --- a/pulsar/__about__.py +++ b/pulsar/__about__.py @@ -16,4 +16,4 @@ # specific language governing permissions and limitations # under the License. # -__version__='3.1.0a1' +__version__='3.2.0a1'
[GitHub] [pulsar] lifepuzzlefun commented on pull request #19793: [fix] [test] Fix flaky test `MetadataStoreStatsTest.testBatchMetadataStoreMetrics`
lifepuzzlefun commented on PR #19793: URL: https://github.com/apache/pulsar/pull/19793#issuecomment-1467894025 @nodece @coderzc please take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-cpp] BewareMyPower commented on issue #205: the C++ clients don’t have isClosed() method like the java client does for testing connection with pulsar server. Can is it implemen
BewareMyPower commented on issue #205: URL: https://github.com/apache/pulsar-client-cpp/issues/205#issuecomment-1467862420 > Right now I am getting this functionality by creating dummy producer on test topic. Having this method would save creation of dummy resources. However, the `isClosed` method is used to detect if a client is closed (by the `closeAsync`/`close` method). The client is associated with a connection pool, not a connection. The producer/consumer/reader is associated with a connection. So even if adding the `isClosed` method, you still cannot use it to achieve your goal. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Technoboy- commented on a diff in pull request #19802: [improve][broker][PIP-195] Add topicName and cursorName for ledger metadata of bucket snapshot
Technoboy- commented on code in PR #19802:
URL: https://github.com/apache/pulsar/pull/19802#discussion_r1135350236
##
managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/LedgerMetadataUtils.java:
##
@@ -48,7 +48,9 @@ public final class LedgerMetadataUtils {
private static final String METADATA_PROPERTY_COMPACTEDTO =
"pulsar/compactedTo";
private static final String METADATA_PROPERTY_SCHEMAID = "pulsar/schemaId";
-private static final String METADATA_PROPERTY_DELAYED_INDEX_BUCKETID =
"pulsar/delayedIndexBucketId";
+private static final String METADATA_PROPERTY_DELAYED_INDEX_BUCKET_KEY =
"pulsar/delayedIndexBucketKey";
Review Comment:
Does this change affect getting older values?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-client-cpp] BewareMyPower commented on issue #212: [Flaky Test] TableViewTest.testCreateTableView
BewareMyPower commented on issue #212: URL: https://github.com/apache/pulsar-client-cpp/issues/212#issuecomment-1467854535 [Another failure](https://github.com/apache/pulsar-client-cpp/actions/runs/4414239473/jobs/7735688682) ``` 2023-03-14T09:45:43.0030645Z 2023-03-14 09:45:42.992 ERROR [140135802893888] ConsumerImpl:303 | [persistent://public/default/testCreateTableView1678787142, reader-f8ccc495db, 2] Failed to create consumer: IncompatibleSchema 2023-03-14T09:45:43.0031317Z /home/runner/work/pulsar-client-cpp/pulsar-client-cpp/tests/TableViewTest.cc:63: Failure 2023-03-14T09:45:43.0031826Z Value of: result == ResultDisconnected || result == ResultAlreadyClosed 2023-03-14T09:45:43.0032125Z Actual: false 2023-03-14T09:45:43.0032388Z Expected: true 2023-03-14T09:45:43.0032907Z 2023-03-14 09:45:42.995 INFO [140135752549952] ClientConnection:1200 | [[::1]:58170 -> [::1]:6650] Connection disconnected ``` It seems that the `ResultIncompatibleSchema` is returned instead. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-cpp] BewareMyPower opened a new issue, #215: [Flaky Test] TableViewTest.testPublishEmptyValue
BewareMyPower opened a new issue, #215:
URL: https://github.com/apache/pulsar-client-cpp/issues/215
[example
failure](https://github.com/apache/pulsar-client-cpp/actions/runs/4414239473/jobs/7735688682)
```
FAILED TESTS (5/474):
149 ms: ./pulsar-tests TableViewTest.testCreateTableView (try #1)
208 ms: ./pulsar-tests TableViewTest.testPublishEmptyValue (try #1)
205 ms: ./pulsar-tests TableViewTest.testPublishEmptyValue (try #2)
253 ms: ./pulsar-tests TableViewTest.testPublishEmptyValue (try #3)
202 ms: ./pulsar-tests TableViewTest.testPublishEmptyValue (try #4)
```
logs:
```
2023-03-14T09:45:43.4435782Z 2023-03-14 09:45:43.269 INFO [140505019240192]
ClientConnection:183 | [ -> pulsar://localhost:6650] Create
ClientConnection, timeout=1
2023-03-14T09:45:43.4439221Z 2023-03-14 09:45:43.270 INFO [140505019240192]
ConnectionPool:106 | Created connection for pulsar://localhost:6650
2023-03-14T09:45:43.4442484Z 2023-03-14 09:45:43.273 INFO [140505016116800]
ClientConnection:381 | [[::1]:58212 -> [::1]:6650] Connected to broker
2023-03-14T09:45:43.4445629Z 2023-03-14 09:45:43.275 INFO [140505016116800]
HandlerBase:72 | [persistent://public/default/testTableView1678787143, ]
Getting connection from pool
2023-03-14T09:45:43.4448984Z 2023-03-14 09:45:43.276 INFO [140505016116800]
ProducerImpl:207 | [persistent://public/default/testTableView1678787143, ]
Created producer on broker [[::1]:58212 -> [::1]:6650]
2023-03-14T09:45:43.4452436Z 2023-03-14 09:45:43.333 INFO [140505016116800]
HandlerBase:72 | [persistent://public/default/testTableView1678787143,
reader-d3fb8f00b8, 0] Getting connection from pool
2023-03-14T09:45:43.4456193Z 2023-03-14 09:45:43.335 INFO [140505016116800]
ConsumerImpl:260 | [persistent://public/default/testTableView1678787143,
reader-d3fb8f00b8, 0] Created consumer on broker [[::1]:58212 -> [::1]:6650]
2023-03-14T09:45:43.4459683Z 2023-03-14 09:45:43.355 INFO [140505016116800]
TableViewImpl:150 | Started table view for
persistent://public/default/testTableView1678787143Replayed: 80 message in 20
millis
2023-03-14T09:45:43.4461563Z
/home/runner/work/pulsar-client-cpp/pulsar-client-cpp/tests/TableViewTest.cc:136:
Failure
2023-03-14T09:45:43.4466115Z Expected equality of these values:
2023-03-14T09:45:43.4466421Z tableView.size()
2023-03-14T09:45:43.4466691Z Which is: 40
2023-03-14T09:45:43.4466868Z count
2023-03-14T09:45:43.4467034Z Which is: 20
2023-03-14T09:45:43.4468276Z 2023-03-14 09:45:43.355 INFO [140505019240192]
ProducerImpl:708 | Producer -
[persistent://public/default/testTableView1678787143, standalone-0-1935] ,
[batchMessageContainer = { BatchMessageContainer [size = 0] [bytes = 0]
[maxSize = 1000] [maxBytes = 131072] [topicName =
persistent://public/default/testTableView1678787143] [numberOfBatchesSent_ =
20] [averageBatchSize_ = 1] }]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] coderzc commented on issue #19792: Flaky-test: org.apache.pulsar.broker.stats.MetadataStoreStatsTest.testBatchMetadataStoreMetrics
coderzc commented on issue #19792: URL: https://github.com/apache/pulsar/issues/19792#issuecomment-1467815667 + 1 https://github.com/apache/pulsar/actions/runs/4413158692/jobs/7736015420#step:11:2577 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] mattisonchao commented on pull request #19800: [fix][broker] Copy subscription properties during updating the topic partition number.
mattisonchao commented on PR #19800: URL: https://github.com/apache/pulsar/pull/19800#issuecomment-1467810153 @codelipenghui Let me change the original PR title. Thanks for your reminder. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] Technoboy- commented on pull request #19762: [fix][broker] Fix create cluster with empty url
Technoboy- commented on PR #19762: URL: https://github.com/apache/pulsar/pull/19762#issuecomment-1467787417 /pulsarbot rerun-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-cpp] lalit-g-deepr commented on issue #205: the C++ clients don’t have isClosed() method like the java client does for testing connection with pulsar server. Can is it implemen
lalit-g-deepr commented on issue #205: URL: https://github.com/apache/pulsar-client-cpp/issues/205#issuecomment-1467768225 @BewareMyPower I am writing API for basic operation for producer consumers. I want that user can check connection without creating producer or consumer. After making connection with pulsar client if connection is not proper they don't need to create producer or consumer. Right now I am getting this functionality by creating dummy producer on test topic. Having this method would save creation of dummy resources. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
