[GitHub] [pulsar] AnonHxy commented on a diff in pull request #16605: [improve][client]PIP-189: No batching if only one message in batch
AnonHxy commented on code in PR #16605:
URL: https://github.com/apache/pulsar/pull/16605#discussion_r950789224
##
pulsar-client-tools/src/main/java/org/apache/pulsar/client/cli/CmdProduce.java:
##
@@ -147,6 +148,7 @@ public class CmdProduce {
private ClientBuilder clientBuilder;
private Authentication authentication;
private String serviceURL;
+private List messageIds = new ArrayList<>();
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] AnonHxy commented on a diff in pull request #16605: [improve][client]PIP-189: No batching if only one message in batch
AnonHxy commented on code in PR #16605:
URL: https://github.com/apache/pulsar/pull/16605#discussion_r950789178
##
pulsar-client-tools-test/src/test/java/org/apache/pulsar/client/cli/PulsarClientToolTest.java:
##
@@ -261,7 +261,6 @@ public void testDisableBatching() throws Exception {
Assert.assertNotNull(msg);
if (i < numberOfMessages) {
Assert.assertEquals(new String(msg.getData()), "batched");
-Assert.assertTrue(msg.getMessageId() instanceof
BatchMessageIdImpl);
Review Comment:
Updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] AnonHxy commented on a diff in pull request #16605: [improve][client]PIP-189: No batching if only one message in batch
AnonHxy commented on code in PR #16605:
URL: https://github.com/apache/pulsar/pull/16605#discussion_r950789165
##
pulsar-broker/src/test/java/org/apache/pulsar/client/api/TopicReaderTest.java:
##
@@ -1096,7 +1096,6 @@ public void testHasMessageAvailableWithBatch() throws
Exception {
ReaderImpl reader =
(ReaderImpl)pulsarClient.newReader().topic(topicName)
.startMessageId(messageId).startMessageIdInclusive().create();
MessageIdImpl lastMsgId = (MessageIdImpl)
reader.getConsumer().getLastMessageId();
-assertTrue(messageId instanceof BatchMessageIdImpl);
Review Comment:
> Please change the name of the method then
>
> @codelipenghui FYI
Changed from `testHasMessageAvailableWithBatch` to `testHasMessageAvailable`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] MarvinCai commented on pull request #17182: [improve][java-client] check consumer pause status before consumer receive/batchReceive
MarvinCai commented on PR #17182:
URL: https://github.com/apache/pulsar/pull/17182#issuecomment-1221469670
the doc for `Consumer#pause` is
```
/**
* Stop requesting new messages from the broker until {@link #resume()} is
called. Note that this might cause
* {@link #receive()} to block until {@link #resume()} is called and new
messages are pushed by the broker.
*/
```
it doesn't say anything about stop receiving message instantly, I think
current behavior is acceptable?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] AnonHxy commented on pull request #16605: [improve][client]PIP-189: No batching if only one message in batch
AnonHxy commented on PR #16605: URL: https://github.com/apache/pulsar/pull/16605#issuecomment-1221468109 /pulsarbot run-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] MarvinCai commented on issue #17121: [Bug][pulsar-client-admin-api] Topics.getInternalInfo() throws 500 instead of 404 if topic does not exist
MarvinCai commented on issue #17121: URL: https://github.com/apache/pulsar/issues/17121#issuecomment-1221466277 do you see any server log that can help diagnose? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] yuruguo commented on a diff in pull request #17170: [fix][test] entry filter makes no sense
yuruguo commented on code in PR #17170:
URL: https://github.com/apache/pulsar/pull/17170#discussion_r950783646
##
pulsar-broker/src/test/java/org/apache/pulsar/broker/service/AbstractBaseDispatcherTest.java:
##
@@ -214,6 +214,9 @@ protected boolean isConsumersExceededOnSubscription() {
@Override
public boolean trackDelayedDelivery(long ledgerId, long entryId,
MessageMetadata msgMetadata) {
+if (!msgMetadata.hasDeliverAtTime()) {
Review Comment:
Minor sugesstion: use `renturn msgMetadata.hasDeliverAtTime()` directly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] mattisonchao closed pull request #17145: [fix][test] Move `PersistentStreamingDispatcherMultipleConsumers` relative test out of `flaky-group`
mattisonchao closed pull request #17145: [fix][test] Move `PersistentStreamingDispatcherMultipleConsumers` relative test out of `flaky-group` URL: https://github.com/apache/pulsar/pull/17145 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #16179: [improve][pulsar-io] Added support for generic record and raw JSON string schemas to CassandraSink
github-actions[bot] commented on PR #16179: URL: https://github.com/apache/pulsar/pull/16179#issuecomment-1221451899 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #16686: [fix][doc] Hotfix Pulsar IO docs
github-actions[bot] commented on PR #16686: URL: https://github.com/apache/pulsar/pull/16686#issuecomment-1221451833 The pr had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #16712: ZkSessionWatcher stuck for 20s after zookeeper client connected.
github-actions[bot] commented on issue #16712: URL: https://github.com/apache/pulsar/issues/16712#issuecomment-1221451816 The issue had no activity for 30 days, mark with Stale label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated: [fix][flaky-test] Fix flaky test testBacklogNoDelayedForPartitionedTopic (#17180)
This is an automated email from the ASF dual-hosted git repository.
technoboy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 1faf497ce29 [fix][flaky-test] Fix flaky test
testBacklogNoDelayedForPartitionedTopic (#17180)
1faf497ce29 is described below
commit 1faf497ce29ca573ebf354396cf73691b27d5fc3
Author: gaozhangmin
AuthorDate: Sun Aug 21 10:01:22 2022 +0800
[fix][flaky-test] Fix flaky test testBacklogNoDelayedForPartitionedTopic
(#17180)
---
.../java/org/apache/pulsar/broker/admin/AdminApi2Test.java | 12 +++-
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApi2Test.java
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApi2Test.java
index 11289119117..580aa906936 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApi2Test.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApi2Test.java
@@ -1669,11 +1669,13 @@ public class AdminApi2Test extends
MockedPulsarServiceBaseTest {
producer.send("message-1".getBytes(StandardCharsets.UTF_8));
}
}
-
-TopicStats topicStats = admin.topics().getPartitionedStats(topic,
false, true, true, true);
-
assertEquals(topicStats.getSubscriptions().get(subName).getMsgBacklog(), 10);
-
assertEquals(topicStats.getSubscriptions().get(subName).getBacklogSize(), 440);
-
assertEquals(topicStats.getSubscriptions().get(subName).getMsgBacklogNoDelayed(),
5);
+// wait until the message add to delay queue.
+Awaitility.await().untilAsserted(() -> {
+TopicStats topicStats = admin.topics().getPartitionedStats(topic,
false, true, true, true);
+
assertEquals(topicStats.getSubscriptions().get(subName).getMsgBacklog(), 10);
+
assertEquals(topicStats.getSubscriptions().get(subName).getBacklogSize(), 440);
+
assertEquals(topicStats.getSubscriptions().get(subName).getMsgBacklogNoDelayed(),
5);
+});
for (int i = 0; i < 5; i++) {
consumer.acknowledge(consumer.receive());
[GitHub] [pulsar] MarvinCai commented on a diff in pull request #17192: [fix][broker] Fix pulsarLedgerIdGenerator can't delete index path when zk metadata store config rootPath.
MarvinCai commented on code in PR #17192:
URL: https://github.com/apache/pulsar/pull/17192#discussion_r950774073
##
pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/PulsarLedgerIdGenerator.java:
##
@@ -173,7 +174,7 @@ private CompletableFuture generateLongLedgerId() {
if (log.isDebugEnabled()) {
log.debug("DELETING HIGH ORDER DIR: {}",
path);
}
-store.delete(path, Optional.of(0L));
Review Comment:
this is a relative path we build ourself and won't have the problem of
carrying root path when getting path from `stats.getPath()`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] Technoboy- merged pull request #17180: [fix][flaky-test] Fix flaky test testBacklogNoDelayedForPartitionedTopic
Technoboy- merged PR #17180: URL: https://github.com/apache/pulsar/pull/17180 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] MarvinCai commented on pull request #17193: [Improvement] Apply maven modernizer to remaining modules.
MarvinCai commented on PR #17193: URL: https://github.com/apache/pulsar/pull/17193#issuecomment-1221443070 @tisonkun please help take a look, these are remaining modules except pulsar-broker -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar-site] branch asf-site-next updated: Updated site at revision 5fac659a
This is an automated email from the ASF dual-hosted git repository.
urfree pushed a commit to branch asf-site-next
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
The following commit(s) were added to refs/heads/asf-site-next by this push:
new 74a14c77d8c Updated site at revision 5fac659a
74a14c77d8c is described below
commit 74a14c77d8c6ff257bed1fb1299da739beeac397
Author: Pulsar Site Updater
AuthorDate: Sun Aug 21 01:05:39 2022 +
Updated site at revision 5fac659a
---
content/index.html | 2 +-
content/swagger/master/swagger.json| 116 ++---
content/swagger/master/swaggerfunctions.json | 74 ++---
content/swagger/master/swaggerlookup.json | 6 +-
content/swagger/master/swaggerpackages.json| 6 +-
content/swagger/master/swaggersink.json| 6 +-
content/swagger/master/swaggersource.json | 6 +-
content/swagger/master/swaggertransactions.json| 6 +-
content/swagger/master/v2/swagger.json | 116 ++---
content/swagger/master/v2/swaggerlookup.json | 6 +-
content/swagger/master/v3/swaggerfunctions.json| 74 ++---
content/swagger/master/v3/swaggerpackages.json | 6 +-
content/swagger/master/v3/swaggersink.json | 6 +-
content/swagger/master/v3/swaggersource.json | 6 +-
content/swagger/master/v3/swaggertransactions.json | 6 +-
15 files changed, 221 insertions(+), 221 deletions(-)
diff --git a/content/index.html b/content/index.html
index a83af85bb02..cec30edc262 100644
--- a/content/index.html
+++ b/content/index.html
@@ -23,7 +23,7 @@
!function(){function
t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var
t=null;try{t=localStorage.getItem("theme")}catch(t){}return
t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())
-Skip to main
content Pulsar Summit San Francisco 2022 will
take place on August 18th, 2022. https://pulsar-summit.org/;>Register now and help us make i [...]
+Skip to main
content Pulsar Summit San Francisco 2022 will
take place on August 18th, 2022. https://pulsar-summit.org/;>Register now and help us make i [...]
http://www.apache.org/; target="_blank" rel="noopener noreferrer"
class="footer__link-item">Foundationhttps://www.apache.org/events/current- [...]
Apache Pulsar is available under the
https://www.apache.org/licenses/LICENSE-2.0; target="_blank">Apache
License, version 2.0.
diff --git a/content/swagger/master/swagger.json
b/content/swagger/master/swagger.json
index 2e10a1dd9c5..df4c4a526b7 100644
--- a/content/swagger/master/swagger.json
+++ b/content/swagger/master/swagger.json
@@ -22036,6 +22036,9 @@
"peerAddress" : {
"$ref" : "#/definitions/SocketAddress"
},
+"subscription" : {
+ "type" : "string"
+},
"tlsCertificates" : {
"type" : "array",
"items" : {
@@ -22047,9 +22050,6 @@
},
"commandData" : {
"type" : "string"
-},
-"subscription" : {
- "type" : "string"
}
}
},
@@ -22098,10 +22098,6 @@
"type" : "string",
"enum" : [ "producer_request_hold", "producer_exception",
"consumer_backlog_eviction" ]
},
-"limit" : {
- "type" : "integer",
- "format" : "int64"
-},
"limitSize" : {
"type" : "integer",
"format" : "int64"
@@ -22109,6 +22105,10 @@
"limitTime" : {
"type" : "integer",
"format" : "int32"
+},
+"limit" : {
+ "type" : "integer",
+ "format" : "int64"
}
}
},
@@ -22145,10 +22145,10 @@
"BookieInfo" : {
"type" : "object",
"properties" : {
-"hostname" : {
+"rack" : {
"type" : "string"
},
-"rack" : {
+"hostname" : {
"type" : "string"
}
}
@@ -22795,6 +22795,13 @@
"FunctionInstanceStatsData" : {
"type" : "object",
"properties" : {
+"oneMin" : {
+ "$ref" : "#/definitions/FunctionInstanceStatsDataBase"
+},
+"lastInvocation" : {
+ "type" : "integer",
+ "format" : "int64"
+},
"userMetrics" : {
"type" : "object",
"additionalProperties" : {
@@ -22802,13 +22809,6 @@
"format" : "double"
}
},
-"lastInvocation" : {
- "type" : "integer",
- "format" : "int64"
-},
-"oneMin" : {
- "$ref" : "#/definitions/FunctionInstanceStatsDataBase"
-},
[pulsar-site] branch main updated: Docs sync done from apache/pulsar(#7f69727)
This is an automated email from the ASF dual-hosted git repository. urfree pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new 5fac659ac61 Docs sync done from apache/pulsar(#7f69727) 5fac659ac61 is described below commit 5fac659ac611c8fad73431f8b9699a4c42393b5b Author: Pulsar Site Updater AuthorDate: Sun Aug 21 00:01:48 2022 + Docs sync done from apache/pulsar(#7f69727)
[GitHub] [pulsar] mans2singh opened a new pull request, #17194: [fix][kubernetes-helm] Update to Step 4.4
mans2singh opened a new pull request, #17194: URL: https://github.com/apache/pulsar/pull/17194 ### Motivation * Step 4.4 indicates ...After successfully **created** an environment.. ### Modifications * Updated Step 4.4 to ...After successfully **creating** an environment... ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. This PR is doc update only. ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): no - The public API: no - The schema: no - The default values of configurations: no - The wire protocol: yes - The rest endpoints: no - The admin cli options: no - Anything that affects deployment: no ### Documentation Check the box below or label this PR directly. Need to update docs? - [ ] `doc-required` (Your PR needs to update docs and you will update later) - [ ] `doc-not-needed` (Please explain why) - [ ] `doc` (Your PR contains doc changes) - [x] `doc-complete` This PR is doc update only. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar-site] branch main updated: Docs sync done from apache/pulsar(#7f69727)
This is an automated email from the ASF dual-hosted git repository. urfree pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new 478d97a844f Docs sync done from apache/pulsar(#7f69727) 478d97a844f is described below commit 478d97a844f05939a3108095aa8e3b71b1c25c8c Author: Pulsar Site Updater AuthorDate: Sat Aug 20 18:01:49 2022 + Docs sync done from apache/pulsar(#7f69727)
[GitHub] [pulsar] shibd commented on a diff in pull request #16969: [refactor][client c++] Delete PartitionedConsumerImpl, use MultiTopicsConsumerImpl instead
shibd commented on code in PR #16969:
URL: https://github.com/apache/pulsar/pull/16969#discussion_r950712013
##
pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc:
##
@@ -363,13 +401,12 @@ void MultiTopicsConsumerImpl::closeAsync(ResultCallback
callback) {
auto self = shared_from_this();
int numConsumers = 0;
-consumers_.forEach(
-[, , callback](const std::string& name, const
ConsumerImplPtr& consumer) {
-numConsumers++;
-consumer->closeAsync([self, name, callback](Result result) {
-self->handleSingleConsumerClose(result, name, callback);
-});
+for (const auto& item : consumers_.toPairVector()) {
Review Comment:
@RobertIndie @BewareMyPower In the current implementation, The `consumers_`
is removed or clear in the `handSingleConsumerClose` method, and
`handSingleConsumerClose` method is executed in `consumers_.forEach`. So there
is a risk that the nullptr.
https://github.com/apache/pulsar/blob/7f6972766f1f8a989f477cbb3085d0660be53bd8/pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc#L369-L375
https://github.com/apache/pulsar/blob/7f6972766f1f8a989f477cbb3085d0660be53bd8/pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc#L392
https://github.com/apache/pulsar/blob/7f6972766f1f8a989f477cbb3085d0660be53bd8/pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc#L409
So use `toPairVector` to re-copy a collection for traversal. Is there a
better way?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] shibd commented on a diff in pull request #16969: [refactor][client c++] Delete PartitionedConsumerImpl, use MultiTopicsConsumerImpl instead
shibd commented on code in PR #16969:
URL: https://github.com/apache/pulsar/pull/16969#discussion_r950712013
##
pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc:
##
@@ -363,13 +401,12 @@ void MultiTopicsConsumerImpl::closeAsync(ResultCallback
callback) {
auto self = shared_from_this();
int numConsumers = 0;
-consumers_.forEach(
-[, , callback](const std::string& name, const
ConsumerImplPtr& consumer) {
-numConsumers++;
-consumer->closeAsync([self, name, callback](Result result) {
-self->handleSingleConsumerClose(result, name, callback);
-});
+for (const auto& item : consumers_.toPairVector()) {
Review Comment:
@RobertIndie @BewareMyPower The current implementation removes or cleans up
the `consumers_` in the `handSingleConsumerClose` method, so there is a risk
that the nullptr.
https://github.com/apache/pulsar/blob/7f6972766f1f8a989f477cbb3085d0660be53bd8/pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc#L369-L375
https://github.com/apache/pulsar/blob/7f6972766f1f8a989f477cbb3085d0660be53bd8/pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc#L392
https://github.com/apache/pulsar/blob/7f6972766f1f8a989f477cbb3085d0660be53bd8/pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc#L409
So use `toPairVector` to re-copy a collection for traversal. Is there a
better way?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar] MarvinCai opened a new pull request, #17193: [Improvement] Apply maven modernizer to remaining modules.
MarvinCai opened a new pull request, #17193: URL: https://github.com/apache/pulsar/pull/17193 Master Issue: https://github.com/apache/pulsar/issues/12271 ### Motivation ### Modifications Applied to following modeules: pulsar-config-validation pulsar-functions pulsar-metadata pulsar-package-management ### Verifying this change - [x] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (no) - The public API: (no) - The schema: (no) - The default values of configurations: (no) - The wire protocol: (no) - The rest endpoints: (no) - The admin cli options: (no) - Anything that affects deployment: (no) ### Documentation Check the box below or label this PR directly. Need to update docs? - [ ] `doc-required` (Your PR needs to update docs and you will update later) - [x] `doc-not-needed` (Please explain why) - [ ] `doc` (Your PR contains doc changes) - [ ] `doc-complete` (Docs have been already added) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] AnonHxy commented on pull request #16605: [improve][client]PIP-189: No batching if only one message in batch
AnonHxy commented on PR #16605: URL: https://github.com/apache/pulsar/pull/16605#issuecomment-1221308388 /pulsarbot run-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar-site] branch main updated: Docs sync done from apache/pulsar(#7f69727)
This is an automated email from the ASF dual-hosted git repository. urfree pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new 8d9f6c3d3ba Docs sync done from apache/pulsar(#7f69727) 8d9f6c3d3ba is described below commit 8d9f6c3d3bab30fc2e9b83554644d6e9102a59ec Author: Pulsar Site Updater AuthorDate: Sat Aug 20 12:01:59 2022 + Docs sync done from apache/pulsar(#7f69727)
[GitHub] [pulsar] github-actions[bot] commented on pull request #17192: [fix][broker] Fix pulsarLedgerIdGenerator can't delete index path when zk metadata store config rootPath.
github-actions[bot] commented on PR #17192: URL: https://github.com/apache/pulsar/pull/17192#issuecomment-1221264131 @horizonzy Please provide a correct documentation label for your PR. Instructions see [Pulsar Documentation Label Guide](https://docs.google.com/document/d/1Qw7LHQdXWBW9t2-r-A7QdFDBwmZh6ytB4guwMoXHqc0). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] horizonzy opened a new pull request, #17192: [fix][broker] Fix pulsarLedgerIdGenerator can't delete index path when zk metadata store config rootPath.
horizonzy opened a new pull request, #17192: URL: https://github.com/apache/pulsar/pull/17192 *(If this PR fixes a github issue, please add `Fixes #`.)* Fixes #17191 ### Motivation Fix pulsarLedgerIdGenerator can't delete index path when zk metadata store config rootPath. ### Documentation - [ ] `doc-not-needed` (Please explain why) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] horizonzy opened a new issue, #17191: [Bug] PulsarLedgerIdGenerator can't delete the index path when zk metadata store config rootPath.
horizonzy opened a new issue, #17191: URL: https://github.com/apache/pulsar/issues/17191 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Version branch master. ### Minimal reproduce step Config zk metadata store with root path, and you will found error log info. ``` metadataStoreUrl=127.0.0.1:2181/cluster2 configurationMetadataStoreUrl=127.0.0.1:2181/cluster2 ``` Error info: ``` Aug 16 00:31:23 168-64-26-103 pulsar[10037]: 2022-08-16T00:31:23,648+0800 [main-EventThread] WARN org.apache.pulsar.metadata.bookkeeper.PulsarLedgerIdGenerator - Exception during deleting node for id generation: Aug 16 00:31:23 168-64-26-103 pulsar[10037]: java.util.concurrent.CompletionException: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /cluster2/ledgers/idgen/ID-000588 Aug 16 00:31:23 168-64-26-103 pulsar[10037]: at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331) ~[?:?] Aug 16 00:31:23 168-64-26-103 pulsar[10037]: at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346) ~[?:?] Aug 16 00:31:23 168-64-26-103 pulsar[10037]: at java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:777) ~[?:?] Aug 16 00:31:23 168-64-26-103 pulsar[10037]: at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?] Aug 16 00:31:23 168-64-26-103 pulsar[10037]: at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?] Aug 16 00:31:23 168-64-26-103 pulsar[10037]: at org.apache.pulsar.metadata.impl.ZKMetadataStore.handleDeleteResult(ZKMetadataStore.java:268) ~[io.streamnative-pulsar-metadata-2.10.1.2.jar:2.10.1.2] Aug 16 00:31:23 168-64-26-103 pulsar[10037]: at org.apache.pulsar.metadata.impl.ZKMetadataStore.lambda$batchOperation$6(ZKMetadataStore.java:180) ~[io.streamnative-pulsar-metadata-2.10.1.2.jar:2.10.1.2] Aug 16 00:31:23 168-64-26-103 pulsar[10037]: at org.apache.pulsar.metadata.impl.PulsarZooKeeperClient$3$1.processResult(PulsarZooKeeperClient.java:490) [io.streamnative-pulsar-metadata-2.10.1.2.jar:2.10.1.2] Aug 16 00:31:23 168-64-26-103 pulsar[10037]: at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:722) [io.streamnative-zookeeper-3.6.3.1.jar:3.6.3.1] Aug 16 00:31:23 168-64-26-103 pulsar[10037]: at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:563) [io.streamnative-zookeeper-3.6.3.1.jar:3.6.3.1] Aug 16 00:31:23 168-64-26-103 pulsar[10037]: Caused by: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /cluster2/ledgers/idgen/ID-000588 Aug 16 00:31:23 168-64-26-103 pulsar[10037]: at org.apache.pulsar.metadata.impl.ZKMetadataStore.getException(ZKMetadataStore.java:428) ~[io.streamnative-pulsar-metadata-2.10.1.2.jar:2.10.1.2] Aug 16 00:31:23 168-64-26-103 pulsar[10037]: ... 5 more Aug 16 00:31:23 168-64-26-103 pulsar[10037]: Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /cluster2/ledgers/idgen/ID-000588 Aug 16 00:31:23 168-64-26-103 pulsar[10037]: at org.apache.zookeeper.KeeperException.create(KeeperException.java:118) ~[io.streamnative-zookeeper-3.6.3.1.jar:3.6.3.1] Aug 16 00:31:23 168-64-26-103 pulsar[10037]: at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[io.streamnative-zookeeper-3.6.3.1.jar:3.6.3.1] Aug 16 00:31:23 168-64-26-103 pulsar[10037]: at org.apache.pulsar.metadata.impl.ZKMetadataStore.getException(ZKMetadataStore.java:422) ~[io.streamnative-pulsar-metadata-2.10.1.2.jar:2.10.1.2] Aug 16 00:31:23 168-64-26-103 pulsar[10037]: ... 5 more Aug 16 00:31:23 168-64-26-103 pulsar[10037]: 2022-08-16T00:31:23,651+0800 [metadata-store-10-1] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [168.64.26.108:3181, 168.64.26.109:3181] for ledger: 588 Aug 16 00:31:23 168-64-26-103 pulsar[10037]: 2022-08-16T00:31:23,651+0800 [metadata-store-10-1] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [self-support/43-rtdc/persistent/RTDC-TRADE-O45-FUNDFLOW-partition-0] Created new ledger 588 Aug 16 00:31:27 168-64-26-103 pulsar[10037]: 2022-08-16T00:31:27,126+0800 [pulsar-web-34-8] INFO org.eclipse.jetty.server.RequestLog - 168.64.26.102 - - [16/Aug/2022:00:31:27 +0800] "GET /metrics HTTP/1.1" 302 0 "-" "Prometheus/2.33.4" 0 Aug 16 00:31:27 168-64-26-103 pulsar[10037]: 2022-08-16T00:31:27,130+0800 [prometheus-stats-35-1] INFO org.eclipse.jetty.server.RequestLog - 168.64.26.102 - - [16/Aug/2022:00:31:27 +0800] "GET /metrics/ HTTP/1.1" 200 299526
[GitHub] [pulsar] eolivelli commented on pull request #17163: [fix][broker] Avoid messages being repeatedly replayed with SHARED subscriptions (streaming dispatcher)
eolivelli commented on PR #17163: URL: https://github.com/apache/pulsar/pull/17163#issuecomment-1221247037 @Technoboy- this change must go into 2.11 as well -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar] branch master updated: [fix][broker] Avoid messages being repeatedly replayed with SHARED subscriptions (streaming dispatcher) (#17163)
This is an automated email from the ASF dual-hosted git repository.
eolivelli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 7f6972766f1 [fix][broker] Avoid messages being repeatedly replayed
with SHARED subscriptions (streaming dispatcher) (#17163)
7f6972766f1 is described below
commit 7f6972766f1f8a989f477cbb3085d0660be53bd8
Author: Nicolò Boschi
AuthorDate: Sat Aug 20 09:03:09 2022 +0200
[fix][broker] Avoid messages being repeatedly replayed with SHARED
subscriptions (streaming dispatcher) (#17163)
* [fix][broker] Avoid messages being repeatedly replayed with SHARED
subscriptions
* move test to broker-api
---
.../persistent/PersistentStreamingDispatcherMultipleConsumers.java | 5 +
.../SimpleProducerConsumerTestStreamingDispatcherTest.java | 2 +-
.../org/apache/pulsar/client/api/SimpleProducerConsumerTest.java | 2 +-
3 files changed, 7 insertions(+), 2 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentStreamingDispatcherMultipleConsumers.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentStreamingDispatcherMultipleConsumers.java
index 3cbc43cfdfe..26647dfc1d5 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentStreamingDispatcherMultipleConsumers.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentStreamingDispatcherMultipleConsumers.java
@@ -155,6 +155,11 @@ public class
PersistentStreamingDispatcherMultipleConsumers extends PersistentDi
@Override
public synchronized void readMoreEntries() {
+if (sendInProgress) {
+// we cannot read more entries while sending the previous batch
+// otherwise we could re-read the same entries and send duplicates
+return;
+}
// totalAvailablePermits may be updated by other threads
int currentTotalAvailablePermits = totalAvailablePermits;
if (currentTotalAvailablePermits > 0 &&
isAtleastOneConsumerAvailable()) {
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/persistent/SimpleProducerConsumerTestStreamingDispatcherTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/persistent/SimpleProducerConsumerTestStreamingDispatcherTest.java
index 7ed7981c937..08b680b61b9 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/persistent/SimpleProducerConsumerTestStreamingDispatcherTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/persistent/SimpleProducerConsumerTestStreamingDispatcherTest.java
@@ -25,7 +25,7 @@ import org.testng.annotations.Test;
/**
* SimpleProducerConsumerTest with {@link StreamingDispatcher}
*/
-@Test(groups = "flaky")
+@Test(groups = "broker-api")
public class SimpleProducerConsumerTestStreamingDispatcherTest extends
SimpleProducerConsumerTest {
@Override
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/client/api/SimpleProducerConsumerTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/client/api/SimpleProducerConsumerTest.java
index 2f572a841b0..e19b712a85b 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/client/api/SimpleProducerConsumerTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/client/api/SimpleProducerConsumerTest.java
@@ -113,7 +113,7 @@ import org.testng.annotations.BeforeMethod;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
-@Test(groups = "flaky")
+@Test(groups = "broker-api")
public class SimpleProducerConsumerTest extends ProducerConsumerBase {
private static final Logger log =
LoggerFactory.getLogger(SimpleProducerConsumerTest.class);
private static final int TIMEOUT_MULTIPLIER =
Integer.getInteger("SimpleProducerConsumerTest.receive.timeout.multiplier", 1);
[GitHub] [pulsar] eolivelli closed issue #16795: Flaky-test: SimpleProducerConsumerTest.testSharedSamePriorityConsumer
eolivelli closed issue #16795: Flaky-test: SimpleProducerConsumerTest.testSharedSamePriorityConsumer URL: https://github.com/apache/pulsar/issues/16795 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] eolivelli merged pull request #17163: [fix][broker] Avoid messages being repeatedly replayed with SHARED subscriptions (streaming dispatcher)
eolivelli merged PR #17163: URL: https://github.com/apache/pulsar/pull/17163 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #16969: [refactor][client c++] Delete PartitionedConsumerImpl, use MultiTopicsConsumerImpl instead
BewareMyPower commented on PR #16969: URL: https://github.com/apache/pulsar/pull/16969#issuecomment-1221246386 Just a suggestion. You can ran the unit tests locally before pushing more commits. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] shibd commented on pull request #17125: [improve][client c++] Support KeyValue Schema.
shibd commented on PR #17125: URL: https://github.com/apache/pulsar/pull/17125#issuecomment-1221244223 /pulsarbot run-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[pulsar-site] branch asf-site-next updated: Updated site at revision 981df9f1
This is an automated email from the ASF dual-hosted git repository.
urfree pushed a commit to branch asf-site-next
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
The following commit(s) were added to refs/heads/asf-site-next by this push:
new c70fff0d3bf Updated site at revision 981df9f1
c70fff0d3bf is described below
commit c70fff0d3bf4016ed9717369696671ed16d29d34
Author: Pulsar Site Updater
AuthorDate: Sat Aug 20 06:18:14 2022 +
Updated site at revision 981df9f1
---
content/index.html | 2 +-
content/swagger/master/swagger.json| 220 ++--
content/swagger/master/swaggerfunctions.json | 130 ++--
content/swagger/master/swaggerlookup.json | 104 +-
content/swagger/master/swaggerpackages.json| 104 +-
content/swagger/master/swaggersink.json| 104 +-
content/swagger/master/swaggersource.json | 104 +-
content/swagger/master/swaggertransactions.json| 104 +-
content/swagger/master/v2/swagger.json | 226 ++---
content/swagger/master/v2/swaggerlookup.json | 104 +-
content/swagger/master/v3/swaggerfunctions.json| 130 ++--
content/swagger/master/v3/swaggerpackages.json | 104 +-
content/swagger/master/v3/swaggersink.json | 104 +-
content/swagger/master/v3/swaggersource.json | 104 +-
content/swagger/master/v3/swaggertransactions.json | 104 +-
15 files changed, 874 insertions(+), 874 deletions(-)
diff --git a/content/index.html b/content/index.html
index c3ee1cc156c..37957338952 100644
--- a/content/index.html
+++ b/content/index.html
@@ -23,7 +23,7 @@
!function(){function
t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var
t=null;try{t=localStorage.getItem("theme")}catch(t){}return
t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())
-Skip to main
content Pulsar Summit San Francisco 2022 will
take place on August 18th, 2022. https://pulsar-summit.org/;>Register now and help us make i [...]
+Skip to main
content Pulsar Summit San Francisco 2022 will
take place on August 18th, 2022. https://pulsar-summit.org/;>Register now and help us make i [...]
http://www.apache.org/; target="_blank" rel="noopener noreferrer"
class="footer__link-item">Foundationhttps://www.apache.org/events/current- [...]
Apache Pulsar is available under the
https://www.apache.org/licenses/LICENSE-2.0; target="_blank">Apache
License, version 2.0.
diff --git a/content/swagger/master/swagger.json
b/content/swagger/master/swagger.json
index 4b37e290a98..c59647c75d2 100644
--- a/content/swagger/master/swagger.json
+++ b/content/swagger/master/swagger.json
@@ -22036,6 +22036,9 @@
"peerAddress" : {
"$ref" : "#/definitions/SocketAddress"
},
+"subscription" : {
+ "type" : "string"
+},
"tlsCertificates" : {
"type" : "array",
"items" : {
@@ -22047,9 +22050,6 @@
},
"commandData" : {
"type" : "string"
-},
-"subscription" : {
- "type" : "string"
}
}
},
@@ -22098,10 +22098,6 @@
"type" : "string",
"enum" : [ "producer_request_hold", "producer_exception",
"consumer_backlog_eviction" ]
},
-"limit" : {
- "type" : "integer",
- "format" : "int64"
-},
"limitSize" : {
"type" : "integer",
"format" : "int64"
@@ -22109,6 +22105,10 @@
"limitTime" : {
"type" : "integer",
"format" : "int32"
+},
+"limit" : {
+ "type" : "integer",
+ "format" : "int64"
}
}
},
@@ -22145,10 +22145,10 @@
"BookieInfo" : {
"type" : "object",
"properties" : {
-"hostname" : {
+"rack" : {
"type" : "string"
},
-"rack" : {
+"hostname" : {
"type" : "string"
}
}
@@ -22657,12 +22657,12 @@
"rootName" : {
"type" : "string"
},
-"fullRootName" : {
- "$ref" : "#/definitions/PropertyName"
-},
"subtypeResolver" : {
"$ref" : "#/definitions/SubtypeResolver"
},
+"fullRootName" : {
+ "$ref" : "#/definitions/PropertyName"
+},
"defaultVisibilityChecker" : {
"$ref" : "#/definitions/VisibilityCheckerObject"
},
@@ -22675,12 +22675,12 @@
"typeFactory" : {
"$ref" : "#/definitions/TypeFactory"
},
-"handlerInstantiator" : {
- "$ref"
[pulsar-site] branch main updated: Docs sync done from apache/pulsar(#1d6824c)
This is an automated email from the ASF dual-hosted git repository. urfree pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new 981df9f14f8 Docs sync done from apache/pulsar(#1d6824c) 981df9f14f8 is described below commit 981df9f14f8c64f098158d2085a654ec543eba8f Author: Pulsar Site Updater AuthorDate: Sat Aug 20 06:01:48 2022 + Docs sync done from apache/pulsar(#1d6824c)
