[GitHub] [pulsar] Anonymitaet commented on issue #5173: [Doc] Update Canal source connector guide
Anonymitaet commented on issue #5173: [Doc] Update Canal source connector guide URL: https://github.com/apache/pulsar/pull/5173#issuecomment-531637046 @tuteng I've incorporated all your comments, could you PTAL? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5173: [Doc] Update Canal source connector guide
Anonymitaet commented on issue #5173: [Doc] Update Canal source connector guide URL: https://github.com/apache/pulsar/pull/5173#issuecomment-531637075 run cpp tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] massakam commented on issue #5185: [pulsar-broker] Fix bug that message delivery stops after resetting cursor for failover subscription
massakam commented on issue #5185: [pulsar-broker] Fix bug that message delivery stops after resetting cursor for failover subscription URL: https://github.com/apache/pulsar/pull/5185#issuecomment-531634199 rerun java8 tests rerun integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5183: [Doc] Update *Overview of Connector*
Anonymitaet commented on issue #5183: [Doc] Update *Overview of Connector* URL: https://github.com/apache/pulsar/pull/5183#issuecomment-531629979 @tuteng I've incorporated all your comments, could you PTAL? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5181: [Doc] Update *Develop Connectors Guide*
Anonymitaet commented on issue #5181: [Doc] Update *Develop Connectors Guide* URL: https://github.com/apache/pulsar/pull/5181#issuecomment-531624564 @tuteng I've incorporated all your comments, could u PTAL? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5192: [doc]Fix javadoc api
tuteng commented on issue #5192: [doc]Fix javadoc api URL: https://github.com/apache/pulsar/pull/5192#issuecomment-531617447 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: skip-all is clear-backlog after topics unification (#5194)
This is an automated email from the ASF dual-hosted git repository. aahmed pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 8f75f58 skip-all is clear-backlog after topics unification (#5194) 8f75f58 is described below commit 8f75f5849ca1ce9ff2bb68662bc2ca64fbffde70 Author: vicaya AuthorDate: Sun Sep 15 13:41:32 2019 -0700 skip-all is clear-backlog after topics unification (#5194) Resolves #5060 --- site2/docs/reference-pulsar-admin.md | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/site2/docs/reference-pulsar-admin.md b/site2/docs/reference-pulsar-admin.md index 2f49410..4564aa7 100644 --- a/site2/docs/reference-pulsar-admin.md +++ b/site2/docs/reference-pulsar-admin.md @@ -1505,7 +1505,7 @@ Subcommands * `info-internal` * `partitioned-stats` * `skip` -* `skip-all` +* `clear-backlog` * `expire-messages` * `expire-messages-all-subscriptions` * `peek-messages` @@ -1800,12 +1800,12 @@ Options |`-s`, `--subscription`|The subscription on which to skip messages|| -### `skip-all` -Skip all the messages for the subscription +### `clear-backlog` +Clear backlog (skip all the messages) for the subscription Usage ```bash -$ pulsar-admin topics skip-all topic options +$ pulsar-admin topics clear-backlog topic options ``` Options
[GitHub] [pulsar] aahmed-se closed issue #5060: pulsar-admin topics skip-all is missing
aahmed-se closed issue #5060: pulsar-admin topics skip-all is missing URL: https://github.com/apache/pulsar/issues/5060 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se merged pull request #5194: skip-all is clear-backlog after topics unification
aahmed-se merged pull request #5194: skip-all is clear-backlog after topics unification URL: https://github.com/apache/pulsar/pull/5194 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] BlackJohnny opened a new pull request #5196: Added missing enum value KeyShared on python wrapper
BlackJohnny opened a new pull request #5196: Added missing enum value KeyShared on python wrapper URL: https://github.com/apache/pulsar/pull/5196 The KeyShared feature is currently not available for python wrapper. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] massakam commented on issue #5185: [pulsar-broker] Fix bug that message delivery stops after resetting cursor for failover subscription
massakam commented on issue #5185: [pulsar-broker] Fix bug that message delivery stops after resetting cursor for failover subscription URL: https://github.com/apache/pulsar/pull/5185#issuecomment-531585525 rerun java8 tests rerun integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] qindongliang closed issue #5195: About pulsar single partition caching problem on a broker
qindongliang closed issue #5195: About pulsar single partition caching problem on a broker URL: https://github.com/apache/pulsar/issues/5195 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] qindongliang commented on issue #5195: About pulsar single partition caching problem on a broker
qindongliang commented on issue #5195: About pulsar single partition caching problem on a broker URL: https://github.com/apache/pulsar/issues/5195#issuecomment-531577169 I have got the answer from slack，https://app.slack.com/client/T5Z3B324U/C5Z4T36F7/thread/C5Z4T36F7-1568557002.035600 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] qindongliang opened a new issue #5195: about pulsar's broker cache problem
qindongliang opened a new issue #5195: about pulsar's broker cache problem URL: https://github.com/apache/pulsar/issues/5195 Hello, everyone，I want to ask if the pulsar's broker cache has multiple copies, or is it only in one broker. If there is only one node cache, is there a hotspot problem which include read or write? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: Fix bug that fails to search namespace bundle due to NPE (#5191)
This is an automated email from the ASF dual-hosted git repository.
zhaijia pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 412c8fd Fix bug that fails to search namespace bundle due to NPE
(#5191)
412c8fd is described below
commit 412c8fdb5ea61593017c395a7c8e291c922d6bf5
Author: Masahiro Sakamoto
AuthorDate: Sun Sep 15 21:31:05 2019 +0900
Fix bug that fails to search namespace bundle due to NPE (#5191)
Fixes #5176
### Motivation
As mentioned in #5176, NPE may occur in the load manager and fail to search
for namespace bundles. This is because multiple threads may update a map named
`brokerToNamespaceToBundleRange` at the same time.
### Modifications
- Changed `brokerToNamespaceToBundleRange` to ConcurrentOpenHashMap instead
of HashMap which is not thread safe.
- Fixed `LoadManagerShared.removeMostServicingBrokersForNamespace()` logic
so that NPE does not occur.
---
.../broker/loadbalance/impl/LoadManagerShared.java | 66 -
.../loadbalance/impl/ModularLoadManagerImpl.java | 18 +++--
.../loadbalance/impl/SimpleLoadManagerImpl.java| 15 ++--
.../AntiAffinityNamespaceGroupTest.java| 17 +++--
.../loadbalance/impl/LoadManagerSharedTest.java| 86 ++
5 files changed, 151 insertions(+), 51 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LoadManagerShared.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LoadManagerShared.java
index 6a5575c..d9cab23 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LoadManagerShared.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LoadManagerShared.java
@@ -21,6 +21,7 @@ package org.apache.pulsar.broker.loadbalance.impl;
import static com.google.common.base.Preconditions.checkArgument;
import static
org.apache.pulsar.broker.cache.ConfigurationCacheService.POLICIES;
import static org.apache.pulsar.broker.web.PulsarWebResource.path;
+import static org.apache.pulsar.common.stats.JvmMetrics.getJvmDirectMemoryUsed;
import java.io.IOException;
import java.net.MalformedURLException;
@@ -41,12 +42,13 @@ import org.apache.pulsar.broker.PulsarService;
import org.apache.pulsar.broker.ServiceConfiguration;
import org.apache.pulsar.broker.loadbalance.BrokerHostUsage;
import org.apache.pulsar.broker.loadbalance.LoadData;
-import static org.apache.pulsar.common.stats.JvmMetrics.getJvmDirectMemoryUsed;
import org.apache.pulsar.common.naming.NamespaceBundle;
import org.apache.pulsar.common.naming.NamespaceName;
import org.apache.pulsar.common.naming.ServiceUnitId;
import org.apache.pulsar.common.policies.data.Policies;
import org.apache.pulsar.common.util.FutureUtil;
+import org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap;
+import org.apache.pulsar.common.util.collections.ConcurrentOpenHashSet;
import org.apache.pulsar.policies.data.loadbalancer.SystemResourceUsage;
import org.apache.pulsar.zookeeper.ZooKeeperDataCache;
import org.slf4j.Logger;
@@ -195,11 +197,12 @@ public class LoadManagerShared {
* @param target
*Map to fill.
*/
-public static void fillNamespaceToBundlesMap(final Set bundles,
final Map> target) {
+public static void fillNamespaceToBundlesMap(final Set bundles,
+final ConcurrentOpenHashMap>
target) {
bundles.forEach(bundleName -> {
final String namespaceName =
getNamespaceNameFromBundleName(bundleName);
final String bundleRange =
getBundleRangeFromBundleName(bundleName);
-target.computeIfAbsent(namespaceName, k -> new
HashSet<>()).add(bundleRange);
+target.computeIfAbsent(namespaceName, k -> new
ConcurrentOpenHashSet<>()).add(bundleRange);
});
}
@@ -258,41 +261,32 @@ public class LoadManagerShared {
*Map from brokers to namespaces to bundle ranges.
*/
public static void removeMostServicingBrokersForNamespace(final String
assignedBundleName,
-final Set candidates, final Map>> brokerToNamespaceToBundleRange) {
+final Set candidates,
+final ConcurrentOpenHashMap>> brokerToNamespaceToBundleRange) {
if (candidates.isEmpty()) {
return;
}
+
final String namespaceName =
getNamespaceNameFromBundleName(assignedBundleName);
int leastBundles = Integer.MAX_VALUE;
+
for (final String broker : candidates) {
-if (brokerToNamespaceToBundleRange.containsKey(broker)) {
-final Set bundleRanges =
brokerToNamespaceToBundleRange.get(broker).get(namespaceName);
-if (bundleRanges == null) {
-// Assume that when the namespace is absent, there are no
bundles for this
[GitHub] [pulsar] jiazhai merged pull request #5191: [Issue #5176][pulsar-broker] Fix bug that fails to search namespace bundle due to NPE
jiazhai merged pull request #5191: [Issue #5176][pulsar-broker] Fix bug that fails to search namespace bundle due to NPE URL: https://github.com/apache/pulsar/pull/5191 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai closed issue #5176: NPE occurs while searching for namespace bundle
jiazhai closed issue #5176: NPE occurs while searching for namespace bundle URL: https://github.com/apache/pulsar/issues/5176 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] massakam commented on issue #5193: [pulsar-broker] Make some member variables of Dispatcher volatile
massakam commented on issue #5193: [pulsar-broker] Make some member variables of Dispatcher volatile URL: https://github.com/apache/pulsar/pull/5193#issuecomment-531560074 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] massakam commented on issue #5185: [pulsar-broker] Fix bug that message delivery stops after resetting cursor for failover subscription
massakam commented on issue #5185: [pulsar-broker] Fix bug that message delivery stops after resetting cursor for failover subscription URL: https://github.com/apache/pulsar/pull/5185#issuecomment-531560049 rerun java8 tests rerun integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] liketic commented on issue #5147: Deprecate getstatus subcommand for source command
liketic commented on issue #5147: Deprecate getstatus subcommand for source command URL: https://github.com/apache/pulsar/pull/5147#issuecomment-531559934 run integration tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] liketic closed pull request #5105: Add skip-all command for topic
liketic closed pull request #5105: Add skip-all command for topic URL: https://github.com/apache/pulsar/pull/5105 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] massakam commented on issue #5193: [pulsar-broker] Make some member variables of Dispatcher volatile
massakam commented on issue #5193: [pulsar-broker] Make some member variables of Dispatcher volatile URL: https://github.com/apache/pulsar/pull/5193#issuecomment-531550196 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] massakam commented on issue #5185: [pulsar-broker] Fix bug that message delivery stops after resetting cursor for failover subscription
massakam commented on issue #5185: [pulsar-broker] Fix bug that message delivery stops after resetting cursor for failover subscription URL: https://github.com/apache/pulsar/pull/5185#issuecomment-531550171 rerun java8 tests rerun integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] massakam commented on issue #5191: [Issue #5176][pulsar-broker] Fix bug that fails to search namespace bundle due to NPE
massakam commented on issue #5191: [Issue #5176][pulsar-broker] Fix bug that fails to search namespace bundle due to NPE URL: https://github.com/apache/pulsar/pull/5191#issuecomment-531550147 rerun java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5100: [bookkeeper] allow configuring region aware placement related settings
sijie commented on issue #5100: [bookkeeper] allow configuring region aware placement related settings URL: https://github.com/apache/pulsar/pull/5100#issuecomment-531537727 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5143: [Issue 5142][docs] Remove non-partitioned topic info from partitioned topic docs
sijie commented on issue #5143: [Issue 5142][docs] Remove non-partitioned topic info from partitioned topic docs URL: https://github.com/apache/pulsar/pull/5143#issuecomment-531537703 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5151: Broker fails to start with function worker enabled and broker client using TLS
sijie commented on issue #5151: Broker fails to start with function worker enabled and broker client using TLS URL: https://github.com/apache/pulsar/pull/5151#issuecomment-531537681 @jiazhai please review it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5192: [doc]Fix javadoc api
sijie commented on issue #5192: [doc]Fix javadoc api URL: https://github.com/apache/pulsar/pull/5192#issuecomment-531537609 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5105: Add skip-all command for topic
sijie commented on issue #5105: Add skip-all command for topic URL: https://github.com/apache/pulsar/pull/5105#issuecomment-531537578 @liketic #5194 has fixed the documentation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5194: skip-all is clear-backlog after topics unification
sijie commented on issue #5194: skip-all is clear-backlog after topics unification URL: https://github.com/apache/pulsar/pull/5194#issuecomment-531537545 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie merged pull request #5184: [Doc] Move Connector Admin CLI to Connector Guide
sijie merged pull request #5184: [Doc] Move Connector Admin CLI to Connector Guide URL: https://github.com/apache/pulsar/pull/5184 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (bbdf14b -> a43a933)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from bbdf14b Fix tenant typo from pubilc to public (#5190) add a43a933 Move Connector Admin CLI to Connector Guide (#5184) No new revisions were added by this update. Summary of changes: site2/website/sidebars.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
