[pulsar-manager] branch master updated: Add license file for project (#209)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-manager.git The following commit(s) were added to refs/heads/master by this push: new 852b16e Add license file for project (#209) 852b16e is described below commit 852b16e5b792e6900350d1a3bb401155d75bba28 Author: tuteng AuthorDate: Tue Nov 12 15:16:42 2019 +0800 Add license file for project (#209) * Add license File for Dependency(Include backend and frontend) --- build.gradle | 17 +- distribution/LICENSE.bin.txt | 413 + distribution/NOTICE.bin.txt| 13 + distribution/licenses/LICENSE-autoprefixer.txt | 20 + distribution/licenses/LICENSE-axios.txt| 19 + distribution/licenses/LICENSE-babel-core.txt | 22 ++ distribution/licenses/LICENSE-babel-eslint.txt | 22 ++ distribution/licenses/LICENSE-babel-loader.txt | 22 ++ .../LICENSE-babel-plugin-dynamic-import-node.txt | 21 ++ distribution/licenses/LICENSE-babel.txt| 22 ++ distribution/licenses/LICENSE-chalk.txt| 9 + distribution/licenses/LICENSE-clipboard.txt| 7 + distribution/licenses/LICENSE-codemirror.txt | 21 ++ distribution/licenses/LICENSE-connect.txt | 25 ++ distribution/licenses/LICENSE-cross-env.txt| 20 + distribution/licenses/LICENSE-css-loader.txt | 20 + distribution/licenses/LICENSE-driver.js.txt| 21 ++ distribution/licenses/LICENSE-element-ui.txt | 21 ++ .../licenses/LICENSE-eslint-friendly-formatter.txt | 21 ++ distribution/licenses/LICENSE-eslint-loader.txt| 20 + .../licenses/LICENSE-eslint-plugin-vue.txt | 21 ++ distribution/licenses/LICENSE-eslint.txt | 19 + distribution/licenses/LICENSE-file-loader.txt | 20 + distribution/licenses/LICENSE-file-saver.txt | 9 + distribution/licenses/LICENSE-font-awesome.txt | 34 ++ .../LICENSE-friendly-errors-webpack-plugin.txt | 21 ++ distribution/licenses/LICENSE-hash-sum.txt | 20 + .../licenses/LICENSE-html-webpack-plugin.txt | 20 + distribution/licenses/LICENSE-husky.txt| 21 ++ distribution/licenses/LICENSE-js-cookie.txt| 21 ++ distribution/licenses/LICENSE-jsonlint.txt | 7 + distribution/licenses/LICENSE-lint-staged.txt | 21 ++ .../licenses/LICENSE-mini-css-extract-plugin.txt | 20 + distribution/licenses/LICENSE-mockito.txt | 21 ++ distribution/licenses/LICENSE-mockjs.txt | 21 ++ distribution/licenses/LICENSE-nightwatch.txt | 9 + distribution/licenses/LICENSE-node-notifier.txt| 21 ++ distribution/licenses/LICENSE-node-sass.txt| 20 + distribution/licenses/LICENSE-normalize.css.txt| 8 + distribution/licenses/LICENSE-nprogress.txt| 7 + .../LICENSE-optimize-css-assets-webpack-plugin.txt | 21 ++ distribution/licenses/LICENSE-ora.txt | 9 + distribution/licenses/LICENSE-pagehelper.txt | 20 + distribution/licenses/LICENSE-path-to-regexp.txt | 21 ++ distribution/licenses/LICENSE-portfinder.txt | 22 ++ distribution/licenses/LICENSE-postcss-import.txt | 20 + distribution/licenses/LICENSE-postcss-loader.txt | 20 + distribution/licenses/LICENSE-postcss-url.txt | 20 + distribution/licenses/LICENSE-postgresql.txt | 23 ++ distribution/licenses/LICENSE-rimraf.txt | 15 + distribution/licenses/LICENSE-sass-loader.txt | 20 + .../LICENSE-script-ext-html-webpack-plugin.txt | 21 ++ distribution/licenses/LICENSE-script-loader.txt| 21 ++ distribution/licenses/LICENSE-semver.txt | 15 + distribution/licenses/LICENSE-serve-static.txt | 25 ++ distribution/licenses/LICENSE-shelljs.txt | 26 ++ distribution/licenses/LICENSE-showdown.txt | 21 ++ distribution/licenses/LICENSE-sortablejs.txt | 21 ++ .../licenses/LICENSE-svg-sprite-loader.txt | 21 ++ distribution/licenses/LICENSE-svgo.txt | 55 +++ .../licenses/LICENSE-uglifyjs-webpack-plugin.txt | 20 + distribution/licenses/LICENSE-url-loader.txt | 20 + distribution/licenses/LICENSE-vue-draggable.txt| 21 ++ distribution/licenses/LICENSE-vue-i18n.txt | 20 + distribution/licenses/LICENSE-vue-loader.txt | 21 ++ distribution/licenses/LICENSE-vue-router.txt | 21 ++ distribution/licenses/LICENSE-vue-style-loader.txt | 20 + distribution/licenses/LICENSE-vue.txt | 21 ++ distribution/licenses/LICENSE-vuex.txt | 21 ++ distribution/licenses/LICENSE-webpack-cli.txt | 20 + .../licenses/LICENSE-webpack-dev-server.txt| 20 + distribution/licenses/LICENSE-webpack-merge.txt| 21 ++ distribution/licenses/LICENSE-webpack-plugin.txt | 20
[pulsar] branch master updated (f2f801a -> 0279431)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from f2f801a [Issue 5585][pulsar-client] Fix producer Semaphore release error (#5587) add 0279431 [Docs] Add admin api docs of Pulsar Functions (#5593) No new revisions were added by this update. Summary of changes: node_modules/.yarn-integrity | 10 + site2/docs/admin-api-functions.md | 545 ++ site2/website/sidebars.json | 3 +- 3 files changed, 557 insertions(+), 1 deletion(-) create mode 100644 node_modules/.yarn-integrity create mode 100644 site2/docs/admin-api-functions.md
[pulsar] branch master updated (0d0af45 -> e8adb0a)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git.
from 0d0af45 [Docs] Add putstate, set-replicator-dispatch-rate and
get-replicator-dispatch-rate cli docs for pulsar admin (#5338)
add e8adb0a Build python 37 artifacts and use them for the docker image
(#5335)
No new revisions were added by this update.
Summary of changes:
docker/pulsar/Dockerfile | 16 +---
docker/pulsar/pom.xml| 4 ++--
...l-pulsar-client-35.sh => install-pulsar-client-37.sh} | 4 ++--
3 files changed, 9 insertions(+), 15 deletions(-)
rename docker/pulsar/scripts/{install-pulsar-client-35.sh =>
install-pulsar-client-37.sh} (88%)
[pulsar] branch master updated (e8adb0a -> dbd48ab)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from e8adb0a Build python 37 artifacts and use them for the docker image (#5335) add dbd48ab Fix bad_weak_ptr error when closing producer (#5315) No new revisions were added by this update. Summary of changes: pulsar-client-cpp/lib/ConsumerImpl.cc | 10 +++--- pulsar-client-cpp/lib/ConsumerImpl.h | 2 +- pulsar-client-cpp/lib/ProducerImpl.cc | 25 + pulsar-client-cpp/lib/ProducerImpl.h | 2 +- 4 files changed, 26 insertions(+), 13 deletions(-)
[pulsar] branch master updated (1611470 -> 0d0af45)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 1611470 [Doc] Add *HDFS2 sink connector guide* (#5226) add 0d0af45 [Docs] Add putstate, set-replicator-dispatch-rate and get-replicator-dispatch-rate cli docs for pulsar admin (#5338) No new revisions were added by this update. Summary of changes: site2/docs/reference-pulsar-admin.md | 43 1 file changed, 43 insertions(+)
[pulsar] branch master updated: [pulsar-proxy] add debug log + fix thread-factory name (#5302)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new ab8db2d [pulsar-proxy] add debug log + fix thread-factory name (#5302)
ab8db2d is described below
commit ab8db2d48f1358ffab1a1df6b35a29bf858267ff
Author: Rajan Dhabalia
AuthorDate: Thu Oct 3 09:31:15 2019 -0700
[pulsar-proxy] add debug log + fix thread-factory name (#5302)
### Motivation
We are debugging issue where client is timing out lookup request and it
helps to have debug log for successful lookup response. and fix thread-factory
name for proxy.
---
.../java/org/apache/pulsar/proxy/server/LookupProxyHandler.java| 7 +--
.../src/main/java/org/apache/pulsar/proxy/server/ProxyService.java | 4 ++--
2 files changed, 7 insertions(+), 4 deletions(-)
diff --git
a/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/LookupProxyHandler.java
b/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/LookupProxyHandler.java
index 9e3893d..ef4fc07 100644
---
a/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/LookupProxyHandler.java
+++
b/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/LookupProxyHandler.java
@@ -27,7 +27,6 @@ import java.net.URISyntaxException;
import java.util.Optional;
import org.apache.commons.lang3.StringUtils;
-import org.apache.pulsar.common.api.proto.PulsarApi.CommandGetSchemaResponse;
import org.apache.pulsar.common.protocol.Commands;
import
org.apache.pulsar.common.api.proto.PulsarApi.CommandGetTopicsOfNamespace;
import org.apache.pulsar.common.api.proto.PulsarApi.CommandGetSchema;
@@ -37,7 +36,6 @@ import
org.apache.pulsar.common.api.proto.PulsarApi.CommandPartitionedTopicMetad
import org.apache.pulsar.common.api.proto.PulsarApi.ServerError;
import org.apache.pulsar.common.naming.TopicName;
import org.apache.pulsar.common.protocol.schema.BytesSchemaVersion;
-import org.apache.pulsar.common.schema.SchemaInfo;
import org.apache.pulsar.policies.data.loadbalancer.ServiceLookupData;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -172,6 +170,11 @@ public class LookupProxyHandler {
// client
// to use the appropriate target broker (and port) when it
// will connect back.
+if (log.isDebugEnabled()) {
+log.debug(
+"Successfully perform lookup '{}' for topic
'{}' with clientReq Id '{}' and lookup-broker {}",
+addr, topic, clientRequestId, brokerUrl);
+}
proxyConnection.ctx().writeAndFlush(Commands.newLookupResponse(brokerUrl,
brokerUrl, true,
LookupType.Connect, clientRequestId, true /* this
is coming from proxy */));
}
diff --git
a/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/ProxyService.java
b/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/ProxyService.java
index a634485..6aa163b 100644
---
a/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/ProxyService.java
+++
b/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/ProxyService.java
@@ -63,8 +63,8 @@ public class ProxyService implements Closeable {
private final EventLoopGroup acceptorGroup;
private final EventLoopGroup workerGroup;
-private final DefaultThreadFactory acceptorThreadFactory = new
DefaultThreadFactory("pulsar-discovery-acceptor");
-private final DefaultThreadFactory workersThreadFactory = new
DefaultThreadFactory("pulsar-discovery-io");
+private final DefaultThreadFactory acceptorThreadFactory = new
DefaultThreadFactory("pulsar-proxy-acceptor");
+private final DefaultThreadFactory workersThreadFactory = new
DefaultThreadFactory("pulsar-proxy-io");
private BrokerDiscoveryProvider discoveryProvider;
[pulsar] branch master updated (0ec1abf -> bdb1f4c)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 0ec1abf [blog] Refine format issues for Pulsar 2.4.2 release blog (#5893) add bdb1f4c [blog] Add blog on Pulsar Summit CFP (#5888) No new revisions were added by this update. Summary of changes: site2/website/blog/2019-12-18-Pulsar-summit-cfp.md | 43 ++ 1 file changed, 43 insertions(+) create mode 100644 site2/website/blog/2019-12-18-Pulsar-summit-cfp.md
[pulsar.wiki] branch master updated: Updated Home (markdown)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new 305bc0c Updated Home (markdown) 305bc0c is described below commit 305bc0c29baeff28c756465d7bf803c453991334 Author: Sijie Guo AuthorDate: Mon Dec 23 13:45:57 2019 +0800 Updated Home (markdown) --- Home.md | 1 + 1 file changed, 1 insertion(+) diff --git a/Home.md b/Home.md index 9866939..7e0ad4e 100644 --- a/Home.md +++ b/Home.md @@ -7,6 +7,7 @@ We encourage to document any big change or feature or any addition to public use ### Proposed +* [[PIP 53: Contribute DotPulsar to Apache Pulsar]] * [[PIP 52: Message dispatch throttling relative to publish rate]] * [[PIP 51: Tenant policy support]] * [[PIP 50: Package Management]]
[pulsar.wiki] branch master updated: Created PIP 53: Contribute DotPulsar to Apache Pulsar (markdown)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new be7f555 Created PIP 53: Contribute DotPulsar to Apache Pulsar (markdown) be7f555 is described below commit be7f5558220ff83f0da96d7db8bb07d2250f0105 Author: Sijie Guo AuthorDate: Mon Dec 23 13:47:55 2019 +0800 Created PIP 53: Contribute DotPulsar to Apache Pulsar (markdown) --- PIP-53:-Contribute-DotPulsar-to-Apache-Pulsar.md | 44 1 file changed, 44 insertions(+) diff --git a/PIP-53:-Contribute-DotPulsar-to-Apache-Pulsar.md b/PIP-53:-Contribute-DotPulsar-to-Apache-Pulsar.md new file mode 100644 index 000..24633e0 --- /dev/null +++ b/PIP-53:-Contribute-DotPulsar-to-Apache-Pulsar.md @@ -0,0 +1,44 @@ +* **Status**: Proposed +* **Author**: Daniel Blankensteiner +* **Pull Request**: +* **Mailing List discussion**: https://lists.apache.org/thread.html/8ebe35027d818e595eec322da26a3f392324ec3d86c4c1b12b1ff498%40%3Cdev.pulsar.apache.org%3E +* **Release**: + +At [Danske Commodities A/S](https://github.com/danske-commodities/dotpulsar/wiki#who-we-are) we have developed a Pulsar client library for .NET - DotPulsar. +The repository is here: https://github.com/danske-commodities/dotpulsar +and the NuGet package is here: https://www.nuget.org/packages/DotPulsar/ +We would like to contribute the project back to the Pulsar community. + +--- + +# Motivation + +Currently, there are no official client libraries for .NET. Therefore we have written one entirely in C# (implementing the binary protocol), using the latest constructs like IAsyncDisposable, IAsyncEnumerable, ValueTask, nullable, pipelines and ReadOnlyMemory/ReadOnlySequence, for optimal developer experience and performance. + +# Features + +DotPulsar is by no means feature complete, but the basic use cases (for Danske Commodities A/S) are supported. + +- Service discovery (LOOKUP) +- Automatic reconnect/retry +- [TLS connections](https://github.com/danske-commodities/dotpulsar/wiki/Client#tls-connection) +- [TLS Authentication](https://github.com/danske-commodities/dotpulsar/wiki/Client#tls-authentication) +- [JSON Web Token Authentication](https://github.com/danske-commodities/dotpulsar/wiki/Client#json-web-token-authentication) +- [Producing message with/without metadata](https://github.com/danske-commodities/dotpulsar/wiki/Producer) +- [Consuming messages using all subscription types and seeking](https://github.com/danske-commodities/dotpulsar/wiki/Consumer) +- [Reading messages](https://github.com/danske-commodities/dotpulsar/wiki/Reader) +- Read/Consume/Acknowledge batched messages (but currently not producing batched messages) + +# Licenses + +DotPulsar is under the Apache License Version 2.0 and only has two dependencies for the .NET Standard 2.1 version and four dependencies for the .NET Standard 2.0 version (which can be used from .NET Framework clients). + +## System.IO.Pipelines, Microsoft.Bcl.AsyncInterfaces and Microsoft.Bcl.HashCode + +Developed by Microsoft and under the MIT license. + +## Protobuf-net + +Developed by Marc Gravell and under the Apache License Version 2.0. + +We are looking forward to any feedback.
[pulsar] branch master updated (bdb1f4c -> b9aa4ce)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from bdb1f4c [blog] Add blog on Pulsar Summit CFP (#5888) add b9aa4ce Fix Typo (#5864) No new revisions were added by this update. Summary of changes: site2/docs/schema-get-started.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[pulsar] branch master updated (e18463f -> 5ffca81)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from e18463f [Issue 5857][Helm Chart] - Support to existing Storage Class with StorageClassName (#5860) add 5ffca81 fix some typos (#5889) No new revisions were added by this update. Summary of changes: .../java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java| 2 +- .../apache/pulsar/broker/service/persistent/PersistentReplicator.java | 2 +- .../java/org/apache/pulsar/client/api/BrokerServiceLookupTest.java| 4 ++-- .../org/apache/pulsar/common/util/SslContextAutoRefreshBuilder.java | 2 +- .../jdbc/src/test/java/org/apache/pulsar/io/jdbc/JdbcSinkTest.java| 2 +- 5 files changed, 6 insertions(+), 6 deletions(-)
[pulsar] branch master updated (5ffca81 -> 17e22d1)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 5ffca81 fix some typos (#5889) add 17e22d1 Added catch for NoClassDefFoundError wherever there was a ClassNotFoundException (#5870) No new revisions were added by this update. Summary of changes: .../org/apache/pulsar/broker/service/BrokerService.java| 2 +- .../org/apache/pulsar/client/internal/ReflectionUtils.java | 2 +- .../apache/kafka/clients/producer/PulsarKafkaProducer.java | 2 +- .../java/org/apache/pulsar/common/nar/NarClassLoader.java | 4 ++-- .../org/apache/pulsar/functions/source/TopicSchema.java| 2 +- .../pulsar/functions/windowing/WindowFunctionExecutor.java | 2 +- .../apache/pulsar/functions/instance/JavaInstanceMain.java | 2 +- .../org/apache/pulsar/functions/utils/FunctionCommon.java | 8 .../apache/pulsar/functions/utils/FunctionConfigUtils.java | 6 +++--- .../org/apache/pulsar/functions/utils/Reflections.java | 14 +++--- .../org/apache/pulsar/functions/utils/SinkConfigUtils.java | 8 .../apache/pulsar/functions/utils/SourceConfigUtils.java | 8 .../org/apache/pulsar/functions/utils/ValidatorUtils.java | 8 .../io/flume/node/PropertiesFileConfigurationProvider.java | 2 +- .../org/apache/pulsar/io/hdfs2/AbstractHdfsConnector.java | 2 +- .../org/apache/pulsar/io/hdfs3/AbstractHdfsConnector.java | 2 +- .../test/java/org/apache/pulsar/io/jdbc/SqliteUtils.java | 2 +- .../org/apache/pulsar/sql/presto/PulsarConnectorUtils.java | 2 +- 18 files changed, 39 insertions(+), 39 deletions(-)
[pulsar] branch master updated (b9aa4ce -> a958098)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from b9aa4ce Fix Typo (#5864) add a958098 fix 5825: make PulsarClusterMetadataSetup idempotent (#5879) No new revisions were added by this update. Summary of changes: .../apache/pulsar/PulsarClusterMetadataSetup.java | 112 + .../broker/zookeeper/ClusterMetadataSetupTest.java | 60 +++ .../zookeeper/ZooKeeperClientAspectJTest.java | 2 +- 3 files changed, 107 insertions(+), 67 deletions(-) create mode 100644 pulsar-broker/src/test/java/org/apache/pulsar/broker/zookeeper/ClusterMetadataSetupTest.java
[pulsar] branch master updated (a958098 -> e18463f)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from a958098 fix 5825: make PulsarClusterMetadataSetup idempotent (#5879) add e18463f [Issue 5857][Helm Chart] - Support to existing Storage Class with StorageClassName (#5860) No new revisions were added by this update. Summary of changes: .../pulsar/templates/bookkeeper-statefulset.yaml | 8 -- .../helm/pulsar/templates/prometheus-pvc.yaml | 4 ++- .../pulsar/templates/zookeeper-statefulset.yaml| 4 ++- deployment/kubernetes/helm/pulsar/values.yaml | 32 -- 4 files changed, 36 insertions(+), 12 deletions(-)
[pulsar] branch master updated (17e22d1 -> 805a421)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 17e22d1 Added catch for NoClassDefFoundError wherever there was a ClassNotFoundException (#5870) add 805a421 [Pulsar Client Tools]Support generate documentation of pulsar admin cli automatically (#5738) No new revisions were added by this update. Summary of changes: .../pulsar/admin/cli/CmdGenerateDocument.java | 129 + .../apache/pulsar/admin/cli/PulsarAdminTool.java | 3 + .../pulsar/tests/integration/cli/CLITest.java | 27 + 3 files changed, 159 insertions(+) create mode 100644 pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdGenerateDocument.java
[pulsar] branch master updated (f7f2184 -> 0607600)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from f7f2184 [Issue 5612] Use Reactivestreams client for Mongo Sink (#5635) add 0607600 [Issue 5832][Pulsar IO]Fix npe of debezium delete event (#5876) No new revisions were added by this update. Summary of changes: .../io/kafka/connect/KafkaConnectSource.java | 8 ++- .../integration/functions/PulsarFunctionsTest.java | 60 -- .../io/DebeziumMongoDbSourceTester.java| 37 + .../integration/io/DebeziumMySqlSourceTester.java | 43 +--- .../io/DebeziumPostgreSqlSourceTester.java | 34 ++-- .../tests/integration/io/KafkaSourceTester.java| 15 ++ .../pulsar/tests/integration/io/SourceTester.java | 27 +- 7 files changed, 208 insertions(+), 16 deletions(-)
[pulsar] branch master updated (805a421 -> 880fb56)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 805a421 [Pulsar Client Tools]Support generate documentation of pulsar admin cli automatically (#5738) add 880fb56 avoid unsafe split when validate hostname which might be ipv6 address (#5713) No new revisions were added by this update. Summary of changes: .../org/apache/pulsar/common/net/ServiceURI.java | 27 +++-- .../apache/pulsar/common/net/ServiceURITest.java | 28 -- 2 files changed, 40 insertions(+), 15 deletions(-)
[pulsar] branch master updated (880fb56 -> f7f2184)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 880fb56 avoid unsafe split when validate hostname which might be ipv6 address (#5713) add f7f2184 [Issue 5612] Use Reactivestreams client for Mongo Sink (#5635) No new revisions were added by this update. Summary of changes: .../org/apache/pulsar/io/mongodb/MongoSink.java| 79 ++ .../apache/pulsar/io/mongodb/MongoSinkTest.java| 60 .../apache/pulsar/io/mongodb/MongoSourceTest.java | 9 ++- 3 files changed, 93 insertions(+), 55 deletions(-)
[pulsar] branch master updated (0607600 -> 5a9d35c)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 0607600 [Issue 5832][Pulsar IO]Fix npe of debezium delete event (#5876) add 5a9d35c Use BOOKIE_MEM and BOOKIE_GC env vars if set. (#5892) No new revisions were added by this update. Summary of changes: conf/bkenv.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
[pulsar-manager] branch master updated (7f97223 -> 5d8ff3f)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-manager.git. from 7f97223 Ugrade HerdDB to 0.13.1 (#235) add 5d8ff3f Add github pull request template (#237) No new revisions were added by this update. Summary of changes: .github/PULL_REQUEST_TEMPLATE.md | 23 +++ 1 file changed, 23 insertions(+) create mode 100644 .github/PULL_REQUEST_TEMPLATE.md
[pulsar-manager] branch master updated: Add a configuration file for backend service of pulsar manager (#236)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-manager.git The following commit(s) were added to refs/heads/master by this push: new 61526ce Add a configuration file for backend service of pulsar manager (#236) 61526ce is described below commit 61526ced603e34d0d723a6fdb1c0b96eb72af4dc Author: tuteng AuthorDate: Wed Dec 25 23:51:45 2019 +0800 Add a configuration file for backend service of pulsar manager (#236) ### Motivation Currently, there are more and more configurable options for pulsar-manger backend services. We need to start the service by specifying a configuration file, which is very useful in a production environment. ### Modification * Add a `supervisord-configuration-file.conf` profile and start the service through the profile becomes an option. --- docker/Dockerfile | 2 ++ docker/entrypoint.sh | 4 docker/supervisord-configuration-file.conf | 32 ++ 3 files changed, 38 insertions(+) diff --git a/docker/Dockerfile b/docker/Dockerfile index f863f37..0fb3f8a 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -57,6 +57,8 @@ COPY docker/supervisord-private-key.conf /etc/ COPY docker/supervisord-secret-key.conf /etc/ +COPY docker/supervisord-configuration-file.conf /etc/ + COPY docker/default.conf /etc/nginx/conf.d/ COPY docker/startup.sh /pulsar-manager/ diff --git a/docker/entrypoint.sh b/docker/entrypoint.sh index 7a5b1ae..d5bc46c 100755 --- a/docker/entrypoint.sh +++ b/docker/entrypoint.sh @@ -51,6 +51,10 @@ elif [[ -n "$JWT_TOKEN" ]] then echo "Enable JWT auth." supervisord -c /etc/supervisord-token.conf -n +elif [[ -n "$SPRING_CONFIGURATION_FILE" ]] +then + echo "Start Pulsar Manager by specifying a configuration file." + supervisord -c /etc/supervisord-configuration-file.conf -n else echo "Start servie no enable JWT." supervisord -c /etc/supervisord.conf -n diff --git a/docker/supervisord-configuration-file.conf b/docker/supervisord-configuration-file.conf new file mode 100644 index 000..ac94c47 --- /dev/null +++ b/docker/supervisord-configuration-file.conf @@ -0,0 +1,32 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +[supervisorctl] +serverurl=unix:///pulsar-manager/supervisor.sock + +[supervisord] +logfile_maxbytes=50MB +logfile_backups=10 +nodaemon=false +minfds=1024 +minprocs=200 + +[program:pulsar-manager-backend] +command = /pulsar-manager/pulsar-manager/bin/pulsar-manager --spring.config.location=%(ENV_SPRING_CONFIGURATION_FILE)s +user = root
[pulsar-manager] branch master updated (61526ce -> 81d0101)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-manager.git. from 61526ce Add a configuration file for backend service of pulsar manager (#236) add 81d0101 Support display stats for tenant and namespaces (#239) No new revisions were added by this update. Summary of changes: front-end/src/views/management/tenants/index.vue | 47 - front-end/src/views/management/tenants/tenant.vue | 46 - .../manager/dao/TopicsStatsRepositoryImpl.java | 19 ++ .../manager/entity/TopicsStatsRepository.java | 13 .../pulsar/manager/mapper/TopicsStatsMapper.java | 40 .../pulsar/manager/service/BrokerStatsService.java | 19 ++ .../service/impl/BrokerStatsServiceImpl.java | 19 ++ .../service/impl/NamespacesServiceImpl.java| 37 ++- .../manager/service/impl/TenantsServiceImpl.java | 38 +++ .../service/BrokerStatsServiceImplTest.java| 76 ++ 10 files changed, 347 insertions(+), 7 deletions(-)
[pulsar-manager] branch master updated: Support sub and unsub (#240)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar-manager.git
The following commit(s) were added to refs/heads/master by this push:
new 1a74888 Support sub and unsub (#240)
1a74888 is described below
commit 1a748880f8135c8efe26fa4a5cbc4772bd8abd60
Author: tuteng
AuthorDate: Wed Dec 25 23:53:56 2019 +0800
Support sub and unsub (#240)
Fix #167
### Motivation
Add operation for sub and unsub
### Modifications
* Add sub and unsub operations
* Support topic and partition topic
### Verifying this change
Local test pass
---
front-end/src/api/subscriptions.js | 22 +
front-end/src/lang/en.js | 9 +-
front-end/src/lang/zh.js | 9 +-
front-end/src/utils/request.js | 2 +
.../views/management/topics/partitionedTopic.vue | 98 +++---
front-end/src/views/management/topics/topic.vue| 71 +++-
6 files changed, 193 insertions(+), 18 deletions(-)
diff --git a/front-end/src/api/subscriptions.js
b/front-end/src/api/subscriptions.js
index e2f5b56..5a002f2 100644
--- a/front-end/src/api/subscriptions.js
+++ b/front-end/src/api/subscriptions.js
@@ -29,9 +29,31 @@ export function putSubscription(tenant, namespace, topic,
subscription) {
})
}
+export function putSubscriptionOnCluster(cluster, persistent,
tenantNamespaceTopic, subscription) {
+ return request({
+headers: {
+ 'Content-Type': 'application/json',
+ 'x-pulsar-cluster': cluster
+},
+url: BASE_URL_V2 +
`/${persistent}/${tenantNamespaceTopic}/subscription/${subscription}`,
+method: 'put'
+ })
+}
+
export function deleteSubscription(tenant, namespace, topic, subscription) {
return request({
url: BASE_URL_V2 +
`/persistent/${tenant}/${namespace}/${topic}/subscription/${subscription}`,
method: 'delete'
})
}
+
+export function deleteSubscriptionOnCluster(cluster, persistent,
tenantNamespaceTopic, subscription) {
+ return request({
+headers: {
+ 'Content-Type': 'application/json',
+ 'x-pulsar-cluster': cluster
+},
+url: BASE_URL_V2 +
`/${persistent}/${tenantNamespaceTopic}/subscription/${subscription}`,
+method: 'delete'
+ })
+}
diff --git a/front-end/src/lang/en.js b/front-end/src/lang/en.js
index 69d1327..8264b31 100644
--- a/front-end/src/lang/en.js
+++ b/front-end/src/lang/en.js
@@ -557,7 +557,14 @@ export default {
storage: 'STORAGE',
storageSize: 'Storage Size',
entries: 'Entries',
- segments: 'Segments'
+ segments: 'Segments',
+ newSub: 'New Subscription',
+ sub: 'Subscribe',
+ unsub: 'Unsubscribe',
+ subNotification: 'Please input correct sub name',
+ createSubSuccess: 'Create subscription successfully',
+ deleteSubSuccess: 'Delete subscription successfully',
+ deleteSubConfirm: 'Do you want to delete this subscription?'
},
segment: {
label: 'Segment',
diff --git a/front-end/src/lang/zh.js b/front-end/src/lang/zh.js
index 0c1e983..d09b9b3 100644
--- a/front-end/src/lang/zh.js
+++ b/front-end/src/lang/zh.js
@@ -557,7 +557,14 @@ export default {
storage: 'STORAGE',
storageSize: 'Storage Size',
entries: 'Entries',
- segments: 'Segments'
+ segments: 'Segments',
+ newSub: 'New Subscription',
+ sub: 'Subscribe',
+ unsub: 'Unsubscribe',
+ subNotification: 'Please input correct sub name',
+ createSubSuccess: 'Create subscription successfully',
+ deleteSubSuccess: 'Delete subscription successfully',
+ deleteSubConfirm: 'Do you want to delete this subscription?'
},
segment: {
label: 'Segment',
diff --git a/front-end/src/utils/request.js b/front-end/src/utils/request.js
index 92cc701..07dcca3 100644
--- a/front-end/src/utils/request.js
+++ b/front-end/src/utils/request.js
@@ -75,6 +75,8 @@ service.interceptors.response.use(
})
return
}
+} else if (error.response.data.hasOwnProperty('reason')) {
+ message = error.response.data.reason
} else {
message = error.response.data
if (message.indexOf('Trying to subscribe with incompatible') >= 0) {
diff --git a/front-end/src/views/management/topics/partitionedTopic.vue
b/front-end/src/views/management/topics/partitionedTopic.vue
index 7ad9e58..1601773 100644
--- a/front-end/src/views/management/topics/partitionedTopic.vue
+++ b/front-end/src/views/management/topics/partitionedTopic.vue
@@ -56,6 +56,13 @@
{{ $t('topic.subscription.subscriptions') }}
+
+ New Sub
+
{{ $t('topic.subscription.clear') }}
+
+{{ $t('topic.subscription.un
[pulsar-manager] branch master updated: Switch from field injection to constructor injection for beans (#230)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar-manager.git
The following commit(s) were added to refs/heads/master by this push:
new 55693a2 Switch from field injection to constructor injection for
beans (#230)
55693a2 is described below
commit 55693a2531d82c2466ebc972ae1d6599a9d34a92
Author: Sergii Zhevzhyk
AuthorDate: Tue Dec 10 05:23:58 2019 +0100
Switch from field injection to constructor injection for beans (#230)
## Motivation
Field injection is not recommended in Spring (for example, IntelliJ shows a
warning). There are a couple of articles and discussions showing the advantages
of Constructor-based dependency injection and Setter-based dependency
injection. For this project, I recommend switching from Field-based dependency
injection to Constructor-based injection. It should bring better testability,
immutability, and show problems in design.
## Change
Initialization of all fields which were injected with the `@Autowired`
annotation was moved to constructors.
---
.../manager/controller/BookiesController.java | 6 ++-
.../manager/controller/BrokerStatsController.java | 17 ---
.../manager/controller/BrokerTokensController.java | 9 ++--
.../manager/controller/BrokersController.java | 17 ---
.../manager/controller/ClustersController.java | 17 ---
.../manager/controller/EnvironmentsController.java | 13 +++--
.../pulsar/manager/controller/LoginController.java | 6 ++-
.../manager/controller/NamespacesController.java | 17 ---
.../manager/controller/TenantsController.java | 17 ---
.../ThirdPartyLoginCallbackController.java | 6 ++-
.../manager/controller/TopicsController.java | 17 ---
.../interceptor/AdminHandlerInterceptor.java | 9 ++--
.../service/impl/BrokerStatsServiceImpl.java | 56 +++---
.../manager/service/impl/ClustersServiceImpl.java | 6 ++-
.../service/impl/EnvironmentCacheServiceImpl.java | 8 ++--
.../service/impl/NamespacesServiceImpl.java| 11 +++--
.../manager/service/impl/TopicsServiceImpl.java| 12 ++---
.../pulsar/manager/zuul/EnvironmentForward.java| 9 +++-
.../zuul/LocationHeaderRewritingFilter.java| 11 +++--
19 files changed, 169 insertions(+), 95 deletions(-)
diff --git
a/src/main/java/org/apache/pulsar/manager/controller/BookiesController.java
b/src/main/java/org/apache/pulsar/manager/controller/BookiesController.java
index 7794659..c42f9ac 100644
--- a/src/main/java/org/apache/pulsar/manager/controller/BookiesController.java
+++ b/src/main/java/org/apache/pulsar/manager/controller/BookiesController.java
@@ -34,8 +34,12 @@ import java.util.Map;
@Validated
public class BookiesController {
+private final BookiesService bookiesService;
+
@Autowired
-private BookiesService bookiesService;
+public BookiesController(BookiesService bookiesService) {
+this.bookiesService = bookiesService;
+}
@ApiOperation(value = "Get the list of existing bookies, support paging,
the default is 10 per page")
@ApiResponses({
diff --git
a/src/main/java/org/apache/pulsar/manager/controller/BrokerStatsController.java
b/src/main/java/org/apache/pulsar/manager/controller/BrokerStatsController.java
index 0f75bf3..5f06089 100644
---
a/src/main/java/org/apache/pulsar/manager/controller/BrokerStatsController.java
+++
b/src/main/java/org/apache/pulsar/manager/controller/BrokerStatsController.java
@@ -38,14 +38,19 @@ import javax.servlet.http.HttpServletRequest;
@RestController
public class BrokerStatsController {
-@Autowired
-private BrokerStatsService brokerStatsService;
-
-@Autowired
-private EnvironmentCacheService environmentCacheService;
+private final BrokerStatsService brokerStatsService;
+private final EnvironmentCacheService environmentCacheService;
+private final HttpServletRequest request;
@Autowired
-private HttpServletRequest request;
+public BrokerStatsController(
+BrokerStatsService brokerStatsService,
+EnvironmentCacheService environmentCacheService,
+HttpServletRequest request) {
+this.brokerStatsService = brokerStatsService;
+this.environmentCacheService = environmentCacheService;
+this.request = request;
+}
@ApiOperation(value = "Get the broker stats metrics")
@ApiResponses({
diff --git
a/src/main/java/org/apache/pulsar/manager/controller/BrokerTokensController.java
b/src/main/java/org/apache/pulsar/manager/controller/BrokerTokensController.java
index ab3296d..7cb973b 100644
---
a/src/main/java/org/apache/pulsar/manager/controller/BrokerTokensController.java
+++
b/src/main/java/org/apache/pulsar/manager/controller/BrokerTokensController.java
@@ -38,11 +38,14 @@ import java.util.Optional;
@Re
[pulsar] branch master updated: updated link to website (#5803)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 4190cfe updated link to website (#5803) 4190cfe is described below commit 4190cfe4d5b27d4ef762e627c4afd80af1f2cc8d Author: Gil Yehuda AuthorDate: Sun Dec 8 11:23:11 2019 -0500 updated link to website (#5803) ### Motivation The current docs pointed to yahoo.github.io, which we decommissioned a while ago. I propose pointing this to the new webpage at opensource.yahoo.com. --- site2/website/versioned_docs/version-2.3.1/concepts-overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site2/website/versioned_docs/version-2.3.1/concepts-overview.md b/site2/website/versioned_docs/version-2.3.1/concepts-overview.md index ad44397..4b33ef9 100644 --- a/site2/website/versioned_docs/version-2.3.1/concepts-overview.md +++ b/site2/website/versioned_docs/version-2.3.1/concepts-overview.md @@ -5,7 +5,7 @@ sidebar_label: Overview original_id: concepts-overview --- -Pulsar is a multi-tenant, high-performance solution for server-to-server messaging. Pulsar was originally developed by [Yahoo](http://yahoo.github.io/), it is under the stewardship of the [Apache Software Foundation](https://www.apache.org/). +Pulsar is a multi-tenant, high-performance solution for server-to-server messaging. Pulsar was originally developed by [Yahoo](http://opensource.yahoo.com/), it is under the stewardship of the [Apache Software Foundation](https://www.apache.org/). Key features of Pulsar are listed below:
svn commit: r37160 [1/2] - in /dev/pulsar/pulsar-2.5.0-candidate-1: ./ DEB/ RPMS/ RPMS/repodata/ connectors/
Author: sijie Date: Tue Dec 10 07:52:09 2019 New Revision: 37160 Log: Staging artifacts and signature for Pulsar release 2.5.0 Added: dev/pulsar/pulsar-2.5.0-candidate-1/ dev/pulsar/pulsar-2.5.0-candidate-1/DEB/ dev/pulsar/pulsar-2.5.0-candidate-1/DEB/Packages.gz (with props) dev/pulsar/pulsar-2.5.0-candidate-1/DEB/Packages.gz.asc dev/pulsar/pulsar-2.5.0-candidate-1/DEB/Packages.gz.sha512 dev/pulsar/pulsar-2.5.0-candidate-1/DEB/apache-pulsar-client-dev.deb (with props) dev/pulsar/pulsar-2.5.0-candidate-1/DEB/apache-pulsar-client-dev.deb.asc dev/pulsar/pulsar-2.5.0-candidate-1/DEB/apache-pulsar-client-dev.deb.sha512 dev/pulsar/pulsar-2.5.0-candidate-1/DEB/apache-pulsar-client.deb (with props) dev/pulsar/pulsar-2.5.0-candidate-1/DEB/apache-pulsar-client.deb.asc dev/pulsar/pulsar-2.5.0-candidate-1/DEB/apache-pulsar-client.deb.sha512 dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/ dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/apache-pulsar-client-2.5.0-1.x86_64.rpm (with props) dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/apache-pulsar-client-2.5.0-1.x86_64.rpm.asc dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/apache-pulsar-client-2.5.0-1.x86_64.rpm.sha512 dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/apache-pulsar-client-debuginfo-2.5.0-1.x86_64.rpm (with props) dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/apache-pulsar-client-debuginfo-2.5.0-1.x86_64.rpm.asc dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/apache-pulsar-client-debuginfo-2.5.0-1.x86_64.rpm.sha512 dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/apache-pulsar-client-devel-2.5.0-1.x86_64.rpm (with props) dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/apache-pulsar-client-devel-2.5.0-1.x86_64.rpm.asc dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/apache-pulsar-client-devel-2.5.0-1.x86_64.rpm.sha512 dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/ dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/4dbc2e5e6909ad5c90aa3056b5b6e5d3e2ea05684dc7de9dfe899408db4a89e9-filelists.sqlite.bz2 (with props) dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/4dbc2e5e6909ad5c90aa3056b5b6e5d3e2ea05684dc7de9dfe899408db4a89e9-filelists.sqlite.bz2.asc dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/4dbc2e5e6909ad5c90aa3056b5b6e5d3e2ea05684dc7de9dfe899408db4a89e9-filelists.sqlite.bz2.sha512 dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/923534688e8790db826d40479f16f1f41e7cbc74493bdac340a3febfb2c7fc4f-filelists.xml.gz (with props) dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/923534688e8790db826d40479f16f1f41e7cbc74493bdac340a3febfb2c7fc4f-filelists.xml.gz.asc dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/923534688e8790db826d40479f16f1f41e7cbc74493bdac340a3febfb2c7fc4f-filelists.xml.gz.sha512 dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/961c8e56d4fd25890d343e1a6d650b627c85c62612bf1ec148b8fce91a70b520-other.xml.gz (with props) dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/961c8e56d4fd25890d343e1a6d650b627c85c62612bf1ec148b8fce91a70b520-other.xml.gz.asc dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/961c8e56d4fd25890d343e1a6d650b627c85c62612bf1ec148b8fce91a70b520-other.xml.gz.sha512 dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/a27edd82873af28ae34efa055fd9a90111bc868bdcf6a8e17f2b942ac4749671-primary.xml.gz (with props) dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/a27edd82873af28ae34efa055fd9a90111bc868bdcf6a8e17f2b942ac4749671-primary.xml.gz.asc dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/a27edd82873af28ae34efa055fd9a90111bc868bdcf6a8e17f2b942ac4749671-primary.xml.gz.sha512 dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/ee61df6431d6802db8988375904e561e77af1800114f9501b1805b7f3d802077-primary.sqlite.bz2 (with props) dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/ee61df6431d6802db8988375904e561e77af1800114f9501b1805b7f3d802077-primary.sqlite.bz2.asc dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/ee61df6431d6802db8988375904e561e77af1800114f9501b1805b7f3d802077-primary.sqlite.bz2.sha512 dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/feba6e9835863cfc562f0965c1216568e53dab93adbac2b58192b3aceba5ce02-other.sqlite.bz2 (with props) dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/feba6e9835863cfc562f0965c1216568e53dab93adbac2b58192b3aceba5ce02-other.sqlite.bz2.asc dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/feba6e9835863cfc562f0965c1216568e53dab93adbac2b58192b3aceba5ce02-other.sqlite.bz2.sha512 dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/repomd.xml dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/repomd.xml.asc dev/pulsar/pulsar-2.5.0-candidate-1/RPMS/repodata/repomd.xml.sha512 dev/pulsar/pulsar-2.5.0-candidate-1/apache-pulsar-2.5.0-bin.tar.gz (with props) dev/pulsar/pulsar-2.5.0-candidate-1/apache-pulsar-2.5.0-bin.tar.gz.asc dev/pulsar/pulsar-2.5.0-candidate-1/apache-pulsar-2.5.0-bin.tar.gz.sha512 dev/pulsar/pulsar
svn commit: r37160 [2/2] - in /dev/pulsar/pulsar-2.5.0-candidate-1: ./ DEB/ RPMS/ RPMS/repodata/ connectors/
Added: dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-file-2.5.0.nar.sha512 == --- dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-file-2.5.0.nar.sha512 (added) +++ dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-file-2.5.0.nar.sha512 Tue Dec 10 07:52:09 2019 @@ -0,0 +1 @@ +5c1ff31c066c114243798c1b641503cb859a7dd85150f3d383623b06c023ba00eb941f789e5404836a2058734cd3376c8c21843d422ef65be9982113d93a25bd ./connectors/pulsar-io-file-2.5.0.nar Added: dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-flume-2.5.0.nar == Binary file - no diff available. Propchange: dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-flume-2.5.0.nar -- svn:mime-type = application/octet-stream Added: dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-flume-2.5.0.nar.asc == --- dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-flume-2.5.0.nar.asc (added) +++ dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-flume-2.5.0.nar.asc Tue Dec 10 07:52:09 2019 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEE2k3oIDhnJWi8g07gWz+9tP10QCwFAl3vRHQACgkQWz+9tP10 +QCz9cg//fY60JvHIVrHJLKKOuH7Ikc7aLzPH1tX1trWh3XWZ2vJpqy3WJJS47/UX +rzA0CjN80t4XJugelpOylS6hhebxyca4cwCt1GL51hVDOg6dKqkVXzN9AfiBo+8l +YzA6j0rdrn8oMnZffbNbKQQXpycShlNmaNMUlETt9WSZcvQUkB8PREE6jLQw2x8H +f9fESWeYrKj+uWsBbonbmacXjqwIk0h9SL40C0quKN2RKN/nQOA2EarTm+me/KQ0 +bBzxH6p4sP1CnF9gq8qgHfu6t3mXZAPPA3AfyeIlVxoZawV4lmn78vmvu9DAFfqQ +jZ2GHFYJRlkLfOX1Gk/6v1xJBzuebtMSsl32hjZrI7ytDREd0pX7GqmyKBfb9sig +KSNqK6BzI8ua1V+mVad/zdQAFbb0eTYuG+oAATYB1hoPJzkHNl0O8qkTMSILsDv5 +GDtaR0KoSyTZhRDQdpSXRztP6TKv+X29TKRFURczSqZA7RnUNqlkfcnyzhTHUMDz +chgIYhDlsfnvvuPCYQxav6V70tNHNSZ3uPi96Rgm+pGMSvH4RVC2QFPt7xp/hi53 +gtvdq4U7EsQyrpCeMKZ/Po4m+89ntRE3XejrQ0XnGaUQSPm+c0+kiyDnK1AZd0+J +0cCU7/vEpxNaj79Jbu5L7aNo9heBHYdkcRT0vB6YXOLr7ctvsWw= +=GluW +-END PGP SIGNATURE- Added: dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-flume-2.5.0.nar.sha512 == --- dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-flume-2.5.0.nar.sha512 (added) +++ dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-flume-2.5.0.nar.sha512 Tue Dec 10 07:52:09 2019 @@ -0,0 +1 @@ +69a77289c1ca3b9001364bda072835e39147867be44b24830d4b1ef83b58b6fa003a3a118582fffddca92928749ac0f4dbfea304f6c448df54848671042fe42c ./connectors/pulsar-io-flume-2.5.0.nar Added: dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-hbase-2.5.0.nar == Binary file - no diff available. Propchange: dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-hbase-2.5.0.nar -- svn:mime-type = application/octet-stream Added: dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-hbase-2.5.0.nar.asc == --- dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-hbase-2.5.0.nar.asc (added) +++ dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-hbase-2.5.0.nar.asc Tue Dec 10 07:52:09 2019 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEE2k3oIDhnJWi8g07gWz+9tP10QCwFAl3vRHYACgkQWz+9tP10 +QCxlQQ/+PJTGTNAGa606Dy3jb2CrsrLkekU3UREyxf9UEEc99aSh7iNkp1U1Qfg3 +4bGOV5wJ7vM0kvcn36mwS87cn2VaJ/T2Mo29b6w7Kng3c9MltJrUfIJCu50G4f/4 +iFAY7ProvtDwoovpp/QJpCkofaVdIjyBZ7Xs3cb8kHrYFTjxcmXu4cqWjO8QHY7k +9o+zwYYSQ+KaC730MStpxUF5qhY55rQGzwUL/+tDd2f7tYakWcuVXnVEFFxu3eWF +CAJYA0B9KHboCKN/LpFp66NHwcqonkZwXfK2KJemesskbvG9ieFQ9wHdET0Mkoyb +li98mKakq4wR01rBolpIBoJKcCZ3O1YkQc7AJ7fKifFRhHsjUkrM74d+5SEFaqRA +oGkCagYIzDqgE64negsJxriTh6ci+Gi3zi2wHgHoK6KZbpX9Q5A8LwxK7QUIRsIK +7eomY5Wo6Cw1URdaDMX8eONdvOOUZh/gI8Z6x+DqooombpVn/aF9Ct+Ax4fTi7SG +JfmR2rz7OlCxsb60wVVwWK3urFuTZxgS9bXis8B1v4SO+KWXrVGgf6VKU7JPVksS +TlNWNTBFvcZ+SVBF8L3TqkzVxig2xWxqC39yHpyeSCnmola3myyCdmT6aJrpFSjt +E9ZKyuXiy7k7m94EU1Q0AvxSHf3qQM4NmULGH2JJw13JK26m5OA= +=yT4E +-END PGP SIGNATURE- Added: dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-hbase-2.5.0.nar.sha512 == --- dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-hbase-2.5.0.nar.sha512 (added) +++ dev/pulsar/pulsar-2.5.0-candidate-1/connectors/pulsar-io-hbase-2.5.0.nar.sha512 Tue Dec 10 07:52:09 2019 @@ -0,0 +1 @@ +684ba872bc0323a50d8a9cd6b5e85c8e07a22801f7e8676b89b8b191afc6df5f743f5ba954a057255605120442cd1efa8df404db8cd05196713d93d972d75fa6 ./connectors/pulsar-io-hbase-2.5.0.nar Added:
[pulsar] branch master updated: add passSourceMessageProperty switch to pulsar functions (#6318)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new bce14ed add passSourceMessageProperty switch to pulsar functions
(#6318)
bce14ed is described below
commit bce14edd617300605c1212bf0ec27425c1dab2e7
Author: Neng Lu
AuthorDate: Thu Feb 13 13:33:20 2020 -0800
add passSourceMessageProperty switch to pulsar functions (#6318)
Fixes #5116
### Motivation
Based on the request in #5116, adding this flag can help reduce ambiguity
and increase flexibility. By default, the function passes source message
properties.
### Modifications
- add flags in pulsar admin client tools
- update functions proto and functionConfig to contain the flag
- update the pulsar sink to pass source message property if the flag is set
---
.../main/java/org/apache/pulsar/broker/admin/v1/Functions.java| 2 --
.../src/main/java/org/apache/pulsar/admin/cli/CmdFunctions.java | 8 +++-
.../java/org/apache/pulsar/common/functions/FunctionConfig.java | 1 +
.../apache/pulsar/functions/instance/JavaInstanceRunnable.java| 4 ++--
.../main/java/org/apache/pulsar/functions/sink/PulsarSink.java| 2 +-
.../java/org/apache/pulsar/functions/sink/PulsarSinkConfig.java | 1 +
pulsar-functions/proto/src/main/proto/Function.proto | 2 ++
.../apache/pulsar/functions/runtime/process/ProcessRuntime.java | 2 +-
.../org/apache/pulsar/functions/utils/FunctionConfigUtils.java| 4
.../apache/pulsar/functions/utils/FunctionConfigUtilsTest.java| 3 +++
10 files changed, 22 insertions(+), 7 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v1/Functions.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v1/Functions.java
index 0591279..39cd9d6 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v1/Functions.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v1/Functions.java
@@ -25,8 +25,6 @@ import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
-import org.apache.pulsar.broker.admin.impl.FunctionsBase;
-
@Path("/functions")
@Api(value = "/functions", description = "Functions admin apis", tags =
"functions", hidden = true)
@Produces(MediaType.APPLICATION_JSON)
diff --git
a/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdFunctions.java
b/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdFunctions.java
index 2787bc3..bd063d2 100644
---
a/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdFunctions.java
+++
b/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdFunctions.java
@@ -257,6 +257,8 @@ public class CmdFunctions extends CmdBase {
protected Boolean DEPRECATED_retainOrdering;
@Parameter(names = "--retain-ordering", description = "Function
consumes and processes messages in order")
protected Boolean retainOrdering;
+@Parameter(names = "--forward-source-message-property", description =
"Forwarding input message's properties to output topic when processing")
+protected Boolean forwardSourceMessageProperty = true;
@Parameter(names = "--subs-name", description = "Pulsar source
subscription name if user wants a specific subscription-name for input-topic
consumer")
protected String subsName;
@Parameter(names = "--parallelism", description = "The parallelism
factor of a Pulsar Function (i.e. the number of function instances to run)")
@@ -389,10 +391,14 @@ public class CmdFunctions extends CmdBase {
functionConfig.setProcessingGuarantees(processingGuarantees);
}
-if (retainOrdering != null) {
+if (null != retainOrdering) {
functionConfig.setRetainOrdering(retainOrdering);
}
+if (null != forwardSourceMessageProperty) {
+
functionConfig.setForwardSourceMessageProperty(forwardSourceMessageProperty);
+}
+
if (isNotBlank(subsName)) {
functionConfig.setSubName(subsName);
}
diff --git
a/pulsar-common/src/main/java/org/apache/pulsar/common/functions/FunctionConfig.java
b/pulsar-common/src/main/java/org/apache/pulsar/common/functions/FunctionConfig.java
index 68ee624..4e1bfe1 100644
---
a/pulsar-common/src/main/java/org/apache/pulsar/common/functions/FunctionConfig.java
+++
b/pulsar-common/src/main/java/org/apache/pulsar/common/functions/FunctionConfig.java
@@ -86,6 +86,7 @@ public class FunctionConfig {
private String logTopic;
private ProcessingGuarantees processingGuarantees;
private Boolean retainOrdering;
[pulsar] branch master updated (bce14ed -> d3ec620)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git.
from bce14ed add passSourceMessageProperty switch to pulsar functions
(#6318)
add d3ec620 [Helm] Pulsar Manager do not work if Pulsar authentication is
enabled (#6315)
No new revisions were added by this update.
Summary of changes:
.../kubernetes/helm/pulsar/templates/grafana-service.yaml | 8
...keeper-configmap.yaml => pulsar-manager-configmap.yaml} | 6 +++---
.../helm/pulsar/templates/pulsar-manager-deployment.yaml | 14 +++---
deployment/kubernetes/helm/pulsar/values-mini.yaml | 9 +
deployment/kubernetes/helm/pulsar/values.yaml | 9 +
5 files changed, 28 insertions(+), 18 deletions(-)
copy deployment/kubernetes/helm/pulsar/templates/{zookeeper-configmap.yaml =>
pulsar-manager-configmap.yaml} (84%)
[pulsar] branch master updated (33232ca -> bc1f078)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git.
from 33232ca [Issue 6111][Doc] Add documentation for Debezium Mongodb
connector (#6112)
add bc1f078 [Issue-6143][helm]: Add the rbac policy to give the brokers
permissions to deploy functions (#6191)
No new revisions were added by this update.
Summary of changes:
.../helm/pulsar/templates/broker-configmap.yaml| 7 -
.../helm/pulsar/templates/broker-deployment.yaml | 3 +++
.../{prometheus-rbac.yaml => broker-rbac.yaml} | 31 +++---
deployment/kubernetes/helm/pulsar/values-mini.yaml | 7 +
deployment/kubernetes/helm/pulsar/values.yaml | 7 +
5 files changed, 39 insertions(+), 16 deletions(-)
copy deployment/kubernetes/helm/pulsar/templates/{prometheus-rbac.yaml =>
broker-rbac.yaml} (66%)
[pulsar] branch master updated (7cade48 -> 257d1c5)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 7cade48 Fixed casting in ZooKeeperCache.getDataIfPresent() (#6313) add 257d1c5 [Fix doc issue 6242] Add a tip about testng in function debug (#6322) No new revisions were added by this update. Summary of changes: site2/docs/functions-debug.md | 6 ++ 1 file changed, 6 insertions(+)
[pulsar] branch master updated (d3ec620 -> 33232ca)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from d3ec620 [Helm] Pulsar Manager do not work if Pulsar authentication is enabled (#6315) add 33232ca [Issue 6111][Doc] Add documentation for Debezium Mongodb connector (#6112) No new revisions were added by this update. Summary of changes: site2/docs/io-connectors.md | 8 +++ site2/docs/io-debezium-source.md | 134 ++- 2 files changed, 139 insertions(+), 3 deletions(-)
[pulsar] branch master updated (2f42077 -> 739c0dd)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 2f42077 Fixed enum package not found (#6401) add 739c0dd removed comma from yaml config (#6402) No new revisions were added by this update. Summary of changes: site2/docs/io-rabbitmq-source.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[pulsar] branch master updated (739c0dd -> 1fcccd6)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 739c0dd removed comma from yaml config (#6402) add 1fcccd6 Fix broker client tls settings error (#6128) No new revisions were added by this update. Summary of changes: .../src/main/java/org/apache/pulsar/broker/PulsarService.java | 8 pulsar-client-cpp/python/pulsar_test.py | 8 2 files changed, 12 insertions(+), 4 deletions(-)
[pulsar] branch master updated (49c573c -> 848ad30)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git.
from 49c573c Remove duplicated lombok annotations in the tests module
(#6412)
add 848ad30 Add verification for SchemaDefinitionBuilderImpl.java (#6405)
No new revisions were added by this update.
Summary of changes:
.../impl/schema/SchemaDefinitionBuilderImpl.java | 9 ++
...dTest.java => SchemaDefinitionBuilderTest.java} | 33 +++---
2 files changed, 25 insertions(+), 17 deletions(-)
copy
pulsar-client/src/test/java/org/apache/pulsar/client/api/{MessageIdTest.java =>
SchemaDefinitionBuilderTest.java} (57%)
[pulsar] branch master updated (fa46693 -> 792ab17)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from fa46693 Cleanup pom files in the tests module (#6421) add 792ab17 Update BatchReceivePolicy.java (#6423) No new revisions were added by this update. Summary of changes: .../main/java/org/apache/pulsar/client/api/BatchReceivePolicy.java | 5 - 1 file changed, 4 insertions(+), 1 deletion(-)
[pulsar] branch master updated (848ad30 -> fa46693)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 848ad30 Add verification for SchemaDefinitionBuilderImpl.java (#6405) add fa46693 Cleanup pom files in the tests module (#6421) No new revisions were added by this update. Summary of changes: tests/bc_2_0_0/pom.xml | 6 -- tests/bc_2_0_1/pom.xml | 6 -- tests/docker-images/java-test-functions/pom.xml | 1 - tests/docker-images/latest-version-image/pom.xml | 1 - tests/docker-images/pom.xml | 1 - tests/integration/pom.xml| 5 - tests/pulsar-spark-test/pom.xml | 6 -- 7 files changed, 26 deletions(-)
[pulsar] branch master updated: Add a message on how to make log refresh immediately when starting a component (#6078)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 4f461c3 Add a message on how to make log refresh immediately when starting a component (#6078) 4f461c3 is described below commit 4f461c3bd36ad832789a6f9186f77da151aba5a5 Author: Fangbin Sun AuthorDate: Sun Mar 1 04:41:55 2020 +0800 Add a message on how to make log refresh immediately when starting a component (#6078) ### Motivation Some users may confuse by pulsar/bookie log without flushing immediately. ### Modifications Add a message in `bin/pulsar-daemon` when starting a component. --- bin/pulsar-daemon | 1 + 1 file changed, 1 insertion(+) diff --git a/bin/pulsar-daemon b/bin/pulsar-daemon index 3d555dd..196aaa7 100755 --- a/bin/pulsar-daemon +++ b/bin/pulsar-daemon @@ -151,6 +151,7 @@ case $startStop in rotate_out_log $out echo starting $command, logging to $logfile +echo Note: Set immediateFlush to true in conf/log4j2.yaml will guarantee the logging event is flushing to disk immediately. The default behavior is switched off due to performance considerations. pulsar=$PULSAR_HOME/bin/pulsar nohup $pulsar $command "$@" > "$out" 2>&1 < /dev/null & echo $! > $pid
[pulsar] branch master updated (b8f0ca0 -> 7f25add)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from b8f0ca0 fix the bug of authenticationData is't initialized. (#6440) add 7f25add Remove duplicated test libraries in POM dependencies (#6430) No new revisions were added by this update. Summary of changes: managed-ledger/pom.xml | 12 pulsar-broker/pom.xml| 7 --- pulsar-client-tools-test/pom.xml | 5 - pulsar-discovery-service/pom.xml | 7 --- pulsar-io/hdfs2/pom.xml | 5 - pulsar-io/hdfs3/pom.xml | 5 - pulsar-io/mongo/pom.xml | 6 -- pulsar-io/redis/pom.xml | 6 -- pulsar-proxy/pom.xml | 6 -- pulsar-sql/presto-pulsar/pom.xml | 13 - 10 files changed, 72 deletions(-)
[pulsar] branch master updated: Close ZK before canceling future with exception (#6228) (#6399)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new e6a631d Close ZK before canceling future with exception (#6228)
(#6399)
e6a631d is described below
commit e6a631d491948119c07a30339c7daf71002c5c0a
Author: Paweł Łoziński
AuthorDate: Sat Feb 29 21:53:15 2020 +0100
Close ZK before canceling future with exception (#6228) (#6399)
Fixes #6228
---
.../java/org/apache/pulsar/zookeeper/ZookeeperBkClientFactoryImpl.java | 1 +
1 file changed, 1 insertion(+)
diff --git
a/pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZookeeperBkClientFactoryImpl.java
b/pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZookeeperBkClientFactoryImpl.java
index aac96ce..8cd674d 100644
---
a/pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZookeeperBkClientFactoryImpl.java
+++
b/pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZookeeperBkClientFactoryImpl.java
@@ -55,6 +55,7 @@ public class ZookeeperBkClientFactoryImpl implements
ZooKeeperClientFactory {
.build();
if (zk.getState() == States.CONNECTEDREADONLY && sessionType
!= SessionType.AllowReadOnly) {
+zk.close();
future.completeExceptionally(new
IllegalStateException("Cannot use a read-only session"));
}
[pulsar] branch master updated (e4099a2 -> f8a7386)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from e4099a2 [docs] Remove Dashboard from sidebar (#6097) add f8a7386 Remove duplicated lombok annotations in the pulsar-functions modules (#6063) No new revisions were added by this update. Summary of changes: .../java/org/apache/pulsar/functions/api/utils/JavaSerDeTest.java | 4 .../apache/pulsar/functions/instance/AuthenticationConfig.java| 8 .../java/org/apache/pulsar/functions/instance/InstanceConfig.java | 8 .../org/apache/pulsar/functions/instance/JavaExecutionResult.java | 6 +- .../java/org/apache/pulsar/functions/api/examples/pojo/Tick.java | 4 .../runtime/kubernetes/KubernetesRuntimeFactoryConfig.java| 2 -- .../functions/runtime/process/ProcessRuntimeFactoryConfig.java| 2 -- .../functions/runtime/thread/ThreadRuntimeFactoryConfig.java | 2 -- .../java/org/apache/pulsar/functions/worker/WorkerConfig.java | 8 .../java/org/apache/pulsar/functions/worker/FunctionActioner.java | 8 .../org/apache/pulsar/functions/worker/FunctionRuntimeInfo.java | 4 +--- .../pulsar/functions/worker/request/ServiceRequestInfo.java | 6 -- .../pulsar/functions/worker/executor/MockExecutorController.java | 1 - 13 files changed, 2 insertions(+), 61 deletions(-)
[pulsar] branch master updated (f8a7386 -> d1afdf9)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from f8a7386 Remove duplicated lombok annotations in the pulsar-functions modules (#6063) add d1afdf9 Apply correct language identifiers to the code blocks in the documentation on how to work with schemas (#6089) No new revisions were added by this update. Summary of changes: site2/docs/schema-understand.md | 23 +++ 1 file changed, 11 insertions(+), 12 deletions(-)
[pulsar] branch master updated (6ff87ee -> 4284e39)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 6ff87ee Fixed the max backoff configuration for lookups (#6444) add 4284e39 upgrade scala-maven-plugin to 4.1.0 (#6469) No new revisions were added by this update. Summary of changes: examples/flink/pom.xml | 2 +- examples/spark/pom.xml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
[pulsar] branch master updated: Make tests more stable by using JSONAssert equals (#6435)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 17f71d3 Make tests more stable by using JSONAssert equals (#6435)
17f71d3 is described below
commit 17f71d3f73fe1bcb6723c8b0718d9a6e05574146
Author: Sijie Guo
AuthorDate: Wed Mar 4 17:34:41 2020 -0800
Make tests more stable by using JSONAssert equals (#6435)
Similar to the change you already merged for AvroSchemaTest.java(#6247):
`jsonSchema.getSchemaInfo().getSchema()` in
`pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/JSONSchemaTest.java`
returns a JSON object. `schemaJson` compares with hard-coded JSON String.
However, the order of entries in `schemaJson` is not guaranteed. Similarly,
test `testKeyValueSchemaInfoToString` in
`pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/KeyValueSchemaInfoTest.java`
returns a JSON object. `havePrimitiveType` compares with hard-coded JSON [...]
This PR proposes to use JSONAssert and modify the corresponding JSON test
assertions so that the test is more stable.
### Motivation
Using JSONAssert and modifying the corresponding JSON test assertions so
that the test is more stable.
### Modifications
Adding `assertJSONEqual` method and replacing `assertEquals` with it in
tests `testAllowNullSchema`, `testNotAllowNullSchema` and
`testKeyValueSchemaInfoToString`.
---
.../apache/pulsar/client/impl/schema/JSONSchemaTest.java| 13 +
.../pulsar/client/impl/schema/KeyValueSchemaInfoTest.java | 7 ---
2 files changed, 13 insertions(+), 7 deletions(-)
diff --git
a/pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/JSONSchemaTest.java
b/pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/JSONSchemaTest.java
index c303ce9..d17bbf0 100644
---
a/pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/JSONSchemaTest.java
+++
b/pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/JSONSchemaTest.java
@@ -33,8 +33,10 @@ import
org.apache.pulsar.client.impl.schema.SchemaTestUtils.Foo;
import org.apache.pulsar.client.impl.schema.SchemaTestUtils.NestedBar;
import org.apache.pulsar.client.impl.schema.SchemaTestUtils.NestedBarList;
import org.apache.pulsar.common.schema.SchemaType;
+import org.skyscreamer.jsonassert.JSONAssert;
import org.testng.Assert;
import org.testng.annotations.Test;
+import org.json.JSONException;
import static org.apache.pulsar.client.impl.schema.SchemaTestUtils.FOO_FIELDS;
import static
org.apache.pulsar.client.impl.schema.SchemaTestUtils.SCHEMA_JSON_NOT_ALLOW_NULL;
@@ -44,13 +46,16 @@ import static org.testng.Assert.assertEquals;
@Slf4j
public class JSONSchemaTest {
+public static void assertJSONEqual(String s1, String s2) throws
JSONException{
+JSONAssert.assertEquals(s1, s2, false);
+}
@Test
-public void testNotAllowNullSchema() {
+public void testNotAllowNullSchema() throws JSONException {
JSONSchema jsonSchema =
JSONSchema.of(SchemaDefinition.builder().withPojo(Foo.class).withAlwaysAllowNull(false).build());
Assert.assertEquals(jsonSchema.getSchemaInfo().getType(),
SchemaType.JSON);
Schema.Parser parser = new Schema.Parser();
String schemaJson = new String(jsonSchema.getSchemaInfo().getSchema());
-Assert.assertEquals(schemaJson, SCHEMA_JSON_NOT_ALLOW_NULL);
+assertJSONEqual(schemaJson, SCHEMA_JSON_NOT_ALLOW_NULL);
Schema schema = parser.parse(schemaJson);
for (String fieldName : FOO_FIELDS) {
@@ -67,13 +72,13 @@ public class JSONSchemaTest {
}
@Test
-public void testAllowNullSchema() {
+public void testAllowNullSchema() throws JSONException {
JSONSchema jsonSchema =
JSONSchema.of(SchemaDefinition.builder().withPojo(Foo.class).build());
Assert.assertEquals(jsonSchema.getSchemaInfo().getType(),
SchemaType.JSON);
Schema.Parser parser = new Schema.Parser();
parser.setValidateDefaults(false);
String schemaJson = new String(jsonSchema.getSchemaInfo().getSchema());
-Assert.assertEquals(schemaJson, SCHEMA_JSON_ALLOW_NULL);
+assertJSONEqual(schemaJson, SCHEMA_JSON_ALLOW_NULL);
Schema schema = parser.parse(schemaJson);
for (String fieldName : FOO_FIELDS) {
diff --git
a/pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/KeyValueSchemaInfoTest.java
b/pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/KeyValueSchemaInfoTest.java
index 99a1724..994f013 100644
---
a/pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/KeyValueSchemaInfoTest.java
+++
b/pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/KeyValueSchemaInfoTest.java
@@ -38,6 +38,7 @@ import
[pulsar] branch master updated (65cc303 -> c3292a6)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 65cc303 [pulsar-proxy] fix logging for published messages (#6474) add c3292a6 [Issue 6394] Add configuration to disable auto creation of subscriptions (#6456) No new revisions were added by this update. Summary of changes: conf/broker.conf | 3 ++ conf/standalone.conf | 3 ++ .../apache/pulsar/broker/ServiceConfiguration.java | 5 +++ .../broker/service/BrokerServiceException.java | 6 +++ .../apache/pulsar/broker/service/ServerCnx.java| 13 ++- .../BrokerServiceAutoTopicCreationTest.java| 43 ++ .../pulsar/functions/worker/WorkerService.java | 5 +++ site2/docs/reference-configuration.md | 1 + 8 files changed, 77 insertions(+), 2 deletions(-)
[pulsar] branch master updated (4284e39 -> 65cc303)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 4284e39 upgrade scala-maven-plugin to 4.1.0 (#6469) add 65cc303 [pulsar-proxy] fix logging for published messages (#6474) No new revisions were added by this update. Summary of changes: .../main/java/org/apache/pulsar/proxy/server/ParserProxyHandler.java| 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[pulsar.wiki] branch master updated: Updated Home (markdown)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new 65df05d Updated Home (markdown) 65df05d is described below commit 65df05d162d926edbdeb428c27d3c69b08bf3a9b Author: Sijie Guo AuthorDate: Thu Mar 5 13:30:10 2020 -0800 Updated Home (markdown) --- Home.md | 1 + 1 file changed, 1 insertion(+) diff --git a/Home.md b/Home.md index 7f321a5..b34b672 100644 --- a/Home.md +++ b/Home.md @@ -7,6 +7,7 @@ We encourage to document any big change or feature or any addition to public use ### Proposed +* [[PIP 59: gPRC Protocol Handler]] * [[PIP 58 : Support Consumers Set Custom Retry Delay]] * [[PIP 57: Improve Broker's Zookeeper Session Timeout Handling]] * [PIP 56: Python3 Migration](https://github.com/apache/pulsar/wiki/PIP-56:-Python3-Migration)
[pulsar.wiki] branch master updated: Created PIP 59: gPRC Protocol Handler (markdown)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git
The following commit(s) were added to refs/heads/master by this push:
new 10aba23 Created PIP 59: gPRC Protocol Handler (markdown)
10aba23 is described below
commit 10aba23acf6c6802d61cab9f0af4a620b4524d9c
Author: Sijie Guo
AuthorDate: Thu Mar 5 13:30:33 2020 -0800
Created PIP 59: gPRC Protocol Handler (markdown)
---
PIP-59:-gPRC-Protocol-Handler.md | 88
1 file changed, 88 insertions(+)
diff --git a/PIP-59:-gPRC-Protocol-Handler.md b/PIP-59:-gPRC-Protocol-Handler.md
new file mode 100644
index 000..1ecc589
--- /dev/null
+++ b/PIP-59:-gPRC-Protocol-Handler.md
@@ -0,0 +1,88 @@
+# PIP-59: gRPC Protocol Handler
+
+* Status: Drafting
+* Author: Christophe Bornet
+* Pull Request:
+* Mailing List discussion:
+* Release:
+
+# Motivation
+
+Implemented since v2.5.0, [PIP
41](https://github.com/apache/pulsar/wiki/PIP-41%3A-Pluggable-Protocol-Handler)
allows implementing custom protocols natively inside a Pulsar Broker.
+This PIP proposes to implement a protocol based on gRPC as an alternative to
the Pulsar binary protocol.
+Contrary to Kafka-on-Pulsar or MQTT-on-Pulsar, this would only be a
replacement of the lower transport/session protocol while keeping Pulsar
Producer/Consumer logic.
+This will allow clients in languages that have a gRPC implementation but not
yet a Pulsar driver to communicate with Pulsar without going through a proxy
that limits the performance.
+
+## Pros
+
+* gRPC being a higher level protocol than TCP it handles natively some of the
features that are needed in the binary protocol:
+ * Framing
+ * Session establishment through bidirectional streaming endpoints
+ * Keep-alive, reconnect
+ * Flow control
+ * Metadata/headers transmission
+ * Errors
+ * Authentication
+* Benefit of the gRPC eco-system (proxies, routers, load balancers, ...)
+* Simpler writing of drivers in languages that have a gRPC client
implementation
+
+## Cons
+
+* gRPC sits on HTTP2 so there will probably be a small overhead compared to
the pure TCP binary protocol.
+* The internals of the gRPC Java server implementation are often not exposed
(on purpose) which means fewer performance optimizations possible.
+
+Performance of the gRPC protocol compared to the binary one should be assessed
once developped.
+
+# Changes
+
+## Protocol handler
+
+A protocol handler will be created with the name `grpc`.
+When this protocol handler is started it will launch a gRPC server on
`grpcServicePort`and a TLS protected one on `grpcServicePortTls` if these
configuration properties are set.
+
+The gRPC service loaded by the server will have the form:
+```protobuf
+service PulsarGrpcService {
+rpc produce(stream Send) returns (stream SendResult) {}
+rpc consume(stream Ack) returns (stream Message) {}
+}
+```
+We can note that contrary to the binary protocol, producers and consumers are
handled by distinct RPCs. However they will still share the same HTTP2
connection on which they are multiplexed.
+
+## RPC Metadata/headers
+
+A gRPC `ServerInterceptor` will be used to populate the gRPC Context with
values sent in the RPC metadata.
+For instance for the produce RPC, the metadata will be the topic name, the
producer name, the schema, etc...
+The `SocketAddress` (IP /port) of the client is also put in context.
+The RPC implementation can then read these data from the context.
+
+## Producers/consumers
+
+At the start of the RPC call, the produce (resp. consume) RPCs will create a
producer (resp. consumer) using the parameters passed in the metadata.
+This producer will be used as long as the bidirectional stream is alive and
will be closed once it completes.
+The current `ServerCnx`, `Producer` and `Consumer` classes have a direct
dependency on the Netty channel and the binary protocol.
+So they must be abstracted to interface or abstract class with distinct
implementations for the Pulsar binary protocol and for the gRPC protocol.
+By making `Producer` and `Consumer` abstract, most of the producer/consumer
logic is reused.
+Only the receiving/sending of messages is done in the implementation.
+Later other protocol handlers (eg. for RSocket) that would be only changing of
the transport will just have to implement those classes.
+
+## Rate limiting
+
+The binary protocol does rate limiting on the publishing by stopping the
automatic read of the Netty channel.
+For gRPC, it's possible to do something similar by disabling the automatic
flow control and stop requesting new data when the rate limit is reached.
+
+## Authentication
+
+If authentication is enabled, a gRPC `ServerInterceptor` will extract
authentication data from metadata headers and use the `AuthenticationService`
to perform authentication.
+If authentication fails, the call will be ended with status
[pulsar] branch master updated: Independent schema is set for each consumer generated by topic (#6356)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 8003d08 Independent schema is set for each consumer generated by
topic (#6356)
8003d08 is described below
commit 8003d08e5ca325867d2e825921f18ddda8d4e1d4
Author: congbo <39078850+congbobo...@users.noreply.github.com>
AuthorDate: Fri Mar 6 14:28:30 2020 +0800
Independent schema is set for each consumer generated by topic (#6356)
### Motivation
Master Issue: #5454
When one Consumer subscribe multi topic, setSchemaInfoPorvider() will be
covered by the consumer generated by the last topic.
### Modification
clone schema for each consumer generated by topic.
### Verifying this change
Add the schemaTest for it.
---
.../schema/JsonSchemaCompatibilityCheckTest.java | 5 +
.../java/org/apache/pulsar/schema/SchemaTest.java | 136 +
.../pulsar/schema/{compatibility => }/Schemas.java | 2 +-
.../SchemaCompatibilityCheckTest.java | 22 ++--
.../java/org/apache/pulsar/client/api/Schema.java | 9 +-
.../client/kafka/compat/PulsarKafkaSchema.java | 5 +
.../client/kafka/compat/PulsarKafkaSchema.java | 5 +
.../client/impl/MultiTopicsConsumerImpl.java | 7 +-
.../pulsar/client/impl/PulsarClientImpl.java | 21 ++--
.../pulsar/client/impl/schema/AbstractSchema.java | 5 +
.../client/impl/schema/AutoConsumeSchema.java | 14 +++
.../client/impl/schema/AutoProduceBytesSchema.java | 5 +
.../pulsar/client/impl/schema/AvroSchema.java | 10 ++
.../pulsar/client/impl/schema/KeyValueSchema.java | 5 +
.../client/impl/schema/KeyValueSchemaInfo.java | 5 +
.../impl/schema/generic/GenericAvroSchema.java | 10 ++
.../impl/schema/generic/GenericSchemaImpl.java | 1 -
.../pulsar/functions/source/SerDeSchema.java | 4 +
18 files changed, 244 insertions(+), 27 deletions(-)
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/schema/JsonSchemaCompatibilityCheckTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/schema/JsonSchemaCompatibilityCheckTest.java
index 09a0c44..7befab0 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/schema/JsonSchemaCompatibilityCheckTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/schema/JsonSchemaCompatibilityCheckTest.java
@@ -124,5 +124,10 @@ public class JsonSchemaCompatibilityCheckTest extends
BaseAvroSchemaCompatibilit
info.setSchema(mapper.writeValueAsBytes(schema));
return new OldJSONSchema<>(info, pojo, mapper);
}
+
+@Override
+public Schema clone() {
+return this;
+}
}
}
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/schema/SchemaTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/schema/SchemaTest.java
new file mode 100644
index 000..eba8ec0
--- /dev/null
+++ b/pulsar-broker/src/test/java/org/apache/pulsar/schema/SchemaTest.java
@@ -0,0 +1,136 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.schema;
+
+import com.google.common.collect.Sets;
+import org.apache.pulsar.broker.auth.MockedPulsarServiceBaseTest;
+import org.apache.pulsar.client.api.Consumer;
+import org.apache.pulsar.client.api.Producer;
+import org.apache.pulsar.client.api.Schema;
+import org.apache.pulsar.client.api.schema.SchemaDefinition;
+import org.apache.pulsar.common.naming.TopicDomain;
+import org.apache.pulsar.common.naming.TopicName;
+import org.apache.pulsar.common.policies.data.ClusterData;
+import org.apache.pulsar.common.policies.data.TenantInfo;
+import org.testng.annotations.AfterMethod;
+import org.testng.annotations.BeforeMethod;
+import org.testng.annotations.Test;
+
+import java.util.Collections;
+
+import static org.apache.pulsar.common.naming.TopicName.PUBLIC_TENANT;
+import static
org.apache.pulsar.schema.compatibility.SchemaCompatibilityCheckTest.randomName;
+import s
[pulsar] branch master updated: Fix memory leak when running topic compaction. (#6485)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new f2ec1b4 Fix memory leak when running topic compaction. (#6485)
f2ec1b4 is described below
commit f2ec1b4e2836859b0a6beb9b5a12656e4bcaf8f9
Author: Rolf Arne Corneliussen
AuthorDate: Fri Mar 6 07:32:28 2020 +0100
Fix memory leak when running topic compaction. (#6485)
Fixes #6482
### Motivation
Prevent topic compaction from leaking direct memory
### Modifications
Several leaks were discovered using Netty leak detection and code review.
* `CompactedTopicImpl.readOneMessageId` would get an `Enumeration` of
`LedgerEntry`, but did not release the underlying buffers. Fix: iterate though
the `Enumeration` and release underlying buffer. Instead of logging the case
where the `Enumeration` did not contain any elements, complete the future
exceptionally with the message (will be logged by Caffeine).
* Two main sources of leak in `TwoPhaseCompactor`. The
`RawBacthConverter.rebatchMessage` method failed to close/release a `ByteBuf`
(uncompressedPayload). Also, the return ByteBuf of
`RawBacthConverter.rebatchMessage` was not closed. The first one was easy to
fix (release buffer), to fix the second one and make the code easier to read, I
decided to not let `RawBacthConverter.rebatchMessage` close the message read
from the topic, instead the message read from the topic can be closed [...]
### Verifying this change
Modified `RawReaderTest.testBatchingRebatch` to show new contract.
One can run the test described to reproduce the issue, to verify no leak is
detected.
---
.../pulsar/client/impl/RawBatchConverter.java | 5 +-
.../pulsar/compaction/CompactedTopicImpl.java | 19 ++--
.../pulsar/compaction/TwoPhaseCompactor.java | 119 +++--
.../apache/pulsar/client/impl/RawReaderTest.java | 4 +-
4 files changed, 82 insertions(+), 65 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/client/impl/RawBatchConverter.java
b/pulsar-broker/src/main/java/org/apache/pulsar/client/impl/RawBatchConverter.java
index e252426..8c21a73 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/client/impl/RawBatchConverter.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/client/impl/RawBatchConverter.java
@@ -91,8 +91,7 @@ public class RawBatchConverter {
* Take a batched message and a filter, and returns a message with the
only the sub-messages
* which match the filter. Returns an empty optional if no messages match.
*
- * This takes ownership of the passes in message, and if the returned
optional is not empty,
- * the ownership of that message is returned also.
+ * NOTE: this message does not alter the reference count of the
RawMessage argument.
*/
public static Optional rebatchMessage(RawMessage msg,
BiPredicate filter)
@@ -161,9 +160,9 @@ public class RawBatchConverter {
return Optional.empty();
}
} finally {
+uncompressedPayload.release();
batchBuffer.release();
metadata.recycle();
-msg.close();
}
}
}
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/compaction/CompactedTopicImpl.java
b/pulsar-broker/src/main/java/org/apache/pulsar/compaction/CompactedTopicImpl.java
index b1378b6..22efe8e 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/compaction/CompactedTopicImpl.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/compaction/CompactedTopicImpl.java
@@ -164,12 +164,19 @@ public class CompactedTopicImpl implements CompactedTopic
{
if (rc != BKException.Code.OK) {
promise.completeExceptionally(BKException.create(rc));
} else {
-try (RawMessage m =
RawMessageImpl.deserializeFrom(
-
seq.nextElement().getEntryBuffer())) {
-promise.complete(m.getMessageIdData());
-} catch (NoSuchElementException e) {
-log.error("No such entry {} in ledger
{}", entryId, lh.getId());
-promise.completeExceptionally(e);
+// Need to release buffers for all entries
in the sequence
+if (seq.hasMoreElements()) {
+LedgerEntry entry = seq.nextElement();
+try (RawMessage m =
RawMessageImpl.deserializeFrom(entry.getE
[pulsar] branch master updated (19ccfd5 -> a3e1efc)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 19ccfd5 Fix create partitioned topic with a substring of an existing topic name. (#6478) add a3e1efc Bump jcloud version to 2.2.0 and remove jcloud-shade module (#6494) No new revisions were added by this update. Summary of changes: jclouds-shaded/pom.xml | 141 - pom.xml| 11 +- tiered-storage/jcloud/pom.xml | 90 ++--- .../jcloud/impl/BlobStoreBackedReadHandleImpl.java | 1 - .../impl/BlobStoreManagedLedgerOffloader.java | 2 +- .../jcloud/BlobStoreBackedInputStreamTest.java | 3 - .../impl/BlobStoreManagedLedgerOffloaderTest.java | 2 +- 7 files changed, 20 insertions(+), 230 deletions(-) delete mode 100644 jclouds-shaded/pom.xml
[pulsar] branch master updated (336e971 -> f862961)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 336e971 Creating a topic does not wait for creating cursor of replicators (#6364) add f862961 [Reader] Should set either start message id or start message from roll back duration. (#6392) No new revisions were added by this update. Summary of changes: .../org/apache/pulsar/client/impl/ReaderTest.java | 2 +- .../pulsar/client/impl/ReaderBuilderImpl.java | 10 +++-- .../apache/pulsar/client/impl/BuildersTest.java| 25 ++ 3 files changed, 34 insertions(+), 3 deletions(-)
[pulsar] branch master updated (81f8afd -> 7fb9aff)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 81f8afd Seek to the first one >= timestamp (#6393) add 7fb9aff [Minor] Fix java code errors reported by lgtm. (#6398) No new revisions were added by this update. Summary of changes: .../bookkeeper/mledger/impl/ManagedLedgerOfflineBacklog.java | 2 +- .../java/org/apache/bookkeeper/mledger/util/CallbackMutex.java | 2 +- .../java/org/apache/pulsar/broker/admin/impl/BrokersBase.java | 2 +- .../org/apache/pulsar/client/admin/internal/BaseResource.java | 2 +- .../main/java/org/apache/pulsar/client/impl/ConsumerImpl.java | 10 +- .../org/apache/pulsar/client/impl/MultiTopicsConsumerImpl.java | 2 +- .../java/org/apache/pulsar/common/policies/data/Policies.java | 2 +- .../main/java/org/apache/pulsar/common/sasl/KerberosName.java | 2 +- 8 files changed, 16 insertions(+), 8 deletions(-)
[pulsar] branch master updated (f862961 -> 81f8afd)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from f862961 [Reader] Should set either start message id or start message from roll back duration. (#6392) add 81f8afd Seek to the first one >= timestamp (#6393) No new revisions were added by this update. Summary of changes: .../persistent/PersistentMessageFinder.java| 2 +- .../service/persistent/PersistentSubscription.java | 2 +- .../apache/pulsar/broker/admin/AdminApiTest.java | 22 +++--- .../pulsar/broker/admin/v1/V1_AdminApiTest.java| 18 +- .../broker/service/SubscriptionSeekTest.java | 4 ++-- .../apache/pulsar/client/api/TopicReaderTest.java | 2 +- .../org/apache/pulsar/client/impl/ReaderTest.java | 4 ++-- 7 files changed, 27 insertions(+), 27 deletions(-)
[pulsar] branch master updated (7fb9aff -> 63ccd43)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 7fb9aff [Minor] Fix java code errors reported by lgtm. (#6398) add 63ccd43 [Java Reader Client] Start reader inside batch result in read first message in batch. (#6345) No new revisions were added by this update. Summary of changes: .../client/api/SimpleProducerConsumerTest.java | 38 --- .../apache/pulsar/client/api/TopicReaderTest.java | 74 -- .../pulsar/client/api/BatchReceivePolicy.java | 2 +- .../apache/pulsar/client/impl/ConsumerBase.java| 2 +- .../apache/pulsar/client/impl/ConsumerImpl.java| 31 +++-- 5 files changed, 123 insertions(+), 24 deletions(-)
[pulsar] branch master updated (c3672a2 -> 3e1b8f6)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from c3672a2 Update sidebars.json (#6434) add 3e1b8f6 [Broker] Create namespace failed when TLS is enabled in PulsarStandalone (#6457) No new revisions were added by this update. Summary of changes: .../java/org/apache/pulsar/PulsarStandalone.java | 32 ++ 1 file changed, 21 insertions(+), 11 deletions(-)
[pulsar] branch master updated (3e1b8f6 -> 330e782)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 3e1b8f6 [Broker] Create namespace failed when TLS is enabled in PulsarStandalone (#6457) add 330e782 Update version-2.5.0-sidebars.json (#6455) No new revisions were added by this update. Summary of changes: site2/website/versioned_sidebars/version-2.5.0-sidebars.json | 4 1 file changed, 4 deletions(-)
[pulsar] branch master updated: [ClientAPI]Fix hasMessageAvailable() (#6362)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new baf155f [ClientAPI]Fix hasMessageAvailable() (#6362)
baf155f is described below
commit baf155f4a41de959145841e4cd922e89363e9292
Author: Yijie Shen
AuthorDate: Tue Mar 3 15:04:49 2020 +0800
[ClientAPI]Fix hasMessageAvailable() (#6362)
Fixes #6333
Previously, `hasMoreMessages` is test against:
```
return lastMessageIdInBroker.compareTo(lastDequeuedMessage) == 0
&& incomingMessages.size() > 0;
```
However, the `incomingMessages` could be 0 when the consumer/reader has
just started and hasn't received any messages yet.
In this PR, the last entry is retrieved and decoded to get message
metadata. for the batchIndex field population.
---
.../apache/pulsar/broker/service/ServerCnx.java| 86 ---
.../pulsar/compaction/TwoPhaseCompactor.java | 6 +-
.../apache/pulsar/client/api/TopicReaderTest.java | 75 -
.../org/apache/pulsar/client/impl/ReaderTest.java | 4 +-
.../apache/pulsar/client/impl/ConsumerImpl.java| 98 --
.../pulsar/client/impl/ZeroQueueConsumerImpl.java | 2 +-
.../pulsar/common/api/raw/RawMessageImpl.java | 3 +
7 files changed, 233 insertions(+), 41 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
index bef63fc..4497c97 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
@@ -46,7 +46,11 @@ import java.util.stream.Collectors;
import javax.naming.AuthenticationException;
import javax.net.ssl.SSLSession;
+import org.apache.bookkeeper.mledger.AsyncCallbacks;
+import org.apache.bookkeeper.mledger.Entry;
+import org.apache.bookkeeper.mledger.ManagedLedgerException;
import org.apache.bookkeeper.mledger.Position;
+import org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl;
import org.apache.bookkeeper.mledger.impl.PositionImpl;
import org.apache.bookkeeper.mledger.util.SafeRun;
import org.apache.commons.lang3.StringUtils;
@@ -59,6 +63,7 @@ import
org.apache.pulsar.broker.service.BrokerServiceException.ConsumerBusyExcep
import
org.apache.pulsar.broker.service.BrokerServiceException.ServerMetadataException;
import
org.apache.pulsar.broker.service.BrokerServiceException.ServiceUnitNotReadyException;
import
org.apache.pulsar.broker.service.BrokerServiceException.TopicNotFoundException;
+import org.apache.pulsar.broker.service.persistent.PersistentTopic;
import
org.apache.pulsar.broker.service.schema.exceptions.IncompatibleSchemaException;
import org.apache.pulsar.broker.service.schema.SchemaRegistryService;
import org.apache.pulsar.broker.web.RestException;
@@ -1396,22 +1401,83 @@ public class ServerCnx extends PulsarHandler {
Topic topic = consumer.getSubscription().getTopic();
Position position = topic.getLastMessageId();
int partitionIndex = TopicName.getPartitionIndex(topic.getName());
-if (log.isDebugEnabled()) {
-log.debug("[{}] [{}][{}] Get LastMessageId {} partitionIndex
{}", remoteAddress,
-topic.getName(), consumer.getSubscription().getName(),
position, partitionIndex);
-}
-MessageIdData messageId = MessageIdData.newBuilder()
-.setLedgerId(((PositionImpl)position).getLedgerId())
-.setEntryId(((PositionImpl)position).getEntryId())
-.setPartition(partitionIndex)
-.build();
-ctx.writeAndFlush(Commands.newGetLastMessageIdResponse(requestId,
messageId));
+getLargestBatchIndexWhenPossible(
+topic,
+(PositionImpl) position,
+partitionIndex,
+requestId,
+consumer.getSubscription().getName());
+
} else {
ctx.writeAndFlush(Commands.newError(getLastMessageId.getRequestId(),
ServerError.MetadataError, "Consumer not found"));
}
}
+private void getLargestBatchIndexWhenPossible(
+Topic topic,
+PositionImpl position,
+int partitionIndex,
+long requestId,
+String subscriptionName) {
+
+PersistentTopic persistentTopic = (PersistentTopic) topic;
+ManagedLedgerImpl ml = (ManagedLedgerImpl)
persistentTopic.getManagedLedger();
+
+// If it's not pointing to a valid entry, respond messageId of the
current position.
+if (position.getEntryId() == -1) {
+MessageIdData messa
[pulsar] branch master updated: Use System.nanoTime() instead of System.currentTimeMillis() (#6454)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 459ec6e Use System.nanoTime() instead of System.currentTimeMillis()
(#6454)
459ec6e is described below
commit 459ec6e8386f6d67e966321dcd0558d15c96205b
Author: Rolf Arne Corneliussen
AuthorDate: Tue Mar 3 08:21:58 2020 +0100
Use System.nanoTime() instead of System.currentTimeMillis() (#6454)
Fixes #6453
### Motivation
`ConsumerBase` and `ProducerImpl` use `System.currentTimeMillis()` to
measure the elapsed time in the 'operations' inner classes
(`ConsumerBase$OpBatchReceive` and `ProducerImpl$OpSendMsg`).
An instance variable `createdAt` is initialized with
`System.currentTimeMills()`, but it is not used for reading wall clock time,
the variable is only used for computing elapsed time (e.g. timeout for a batch).
When the variable is used to compute elapsed time, it would more sense to
use `System.nanoTime()`.
### Modifications
The instance variable `createdAt` in `ConsumerBase$OpBatchReceive` and
`ProducerImpl$OpSendMsg` is initialized with `System.nanoTime()`. Usage of the
variable is updated to reflect that the variable holds nano time; computations
of elapsed time takes the difference between the current system nano time and
the `createdAt` variable.
The `createdAt` field is package protected, and is currently only used in
the declaring class and outer class, limiting the chances for unwanted side
effects.
---
.../java/org/apache/pulsar/client/impl/ConsumerBase.java | 9 +
.../java/org/apache/pulsar/client/impl/ProducerImpl.java | 15 ---
2 files changed, 13 insertions(+), 11 deletions(-)
diff --git
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerBase.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerBase.java
index 0f5219b..de9ec85 100644
---
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerBase.java
+++
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerBase.java
@@ -287,7 +287,7 @@ public abstract class ConsumerBase extends HandlerState
implements TimerTask,
public void negativeAcknowledge(Message message) {
negativeAcknowledge(message.getMessageId());
}
-
+
protected CompletableFuture doAcknowledgeWithTxn(MessageId
messageId, AckType ackType,
Map
properties,
TransactionImpl
txn) {
@@ -507,7 +507,7 @@ public abstract class ConsumerBase extends HandlerState
implements TimerTask,
private OpBatchReceive(CompletableFuture> future) {
this.future = future;
-this.createdAt = System.currentTimeMillis();
+this.createdAt = System.nanoTime();
}
static OpBatchReceive of(CompletableFuture> future)
{
@@ -566,8 +566,9 @@ public abstract class ConsumerBase extends HandlerState
implements TimerTask,
while (firstOpBatchReceive != null) {
// If there is at least one batch receive, calculate the diff
between the batch receive timeout
-// and the current time.
-long diff = (firstOpBatchReceive.createdAt +
batchReceivePolicy.getTimeoutMs()) - System.currentTimeMillis();
+// and the elapsed time since the operation was created.
+long diff = batchReceivePolicy.getTimeoutMs()
+- TimeUnit.NANOSECONDS.toMillis(System.nanoTime() -
firstOpBatchReceive.createdAt);
if (diff <= 0) {
// The diff is less than or equal to zero, meaning that
the batch receive has been timed out.
// complete the OpBatchReceive and continue to check the
next OpBatchReceive in pendingBatchReceives.
diff --git
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
index 776300d..eb05909 100644
---
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
+++
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
@@ -987,7 +987,7 @@ public class ProducerImpl extends ProducerBase
implements TimerTask, Conne
op.cmd = cmd;
op.callback = callback;
op.sequenceId = sequenceId;
-op.createdAt = System.currentTimeMillis();
+op.createdAt = System.nanoTime();
return op;
}
@@ -997,7 +997,7 @@ public class ProducerImpl extends ProducerBase
implements TimerTask, Conne
op.cmd = cmd;
op.callback = callback;
op.sequenceId = sequenceId;
-op.
[pulsar] branch master updated: Update sidebars.json (#6434)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new c3672a2 Update sidebars.json (#6434) c3672a2 is described below commit c3672a238dd698bfcf0a4831b9bb050e620fad6d Author: Ryan Slominski AuthorDate: Mon Mar 2 02:52:45 2020 -0500 Update sidebars.json (#6434) The referenced markdown files do not exist and so the "Next" and "Previous" buttons on the bottom of pages surrounding them result in 404 Not Found errors --- site2/website/sidebars.json | 4 1 file changed, 4 deletions(-) diff --git a/site2/website/sidebars.json b/site2/website/sidebars.json index 828f5ea..6ebbea4 100644 --- a/site2/website/sidebars.json +++ b/site2/website/sidebars.json @@ -30,10 +30,6 @@ "functions-develop", "functions-debug", "functions-deploy", - "functions-configure", - "functions-monitor", - "functions-secure", - "functions-troubleshoot", "functions-cli" ], "Pulsar IO": [
[pulsar.wiki] branch master updated: Updated PIP 41: Pluggable Protocol Handler (markdown)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new 0407d39 Updated PIP 41: Pluggable Protocol Handler (markdown) 0407d39 is described below commit 0407d39ab1fd916d454769b545224c667b128c56 Author: Sijie Guo AuthorDate: Sun Mar 1 23:19:13 2020 -0800 Updated PIP 41: Pluggable Protocol Handler (markdown) --- PIP-41:-Pluggable-Protocol-Handler.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/PIP-41:-Pluggable-Protocol-Handler.md b/PIP-41:-Pluggable-Protocol-Handler.md index abf45bc..68b5a72 100644 --- a/PIP-41:-Pluggable-Protocol-Handler.md +++ b/PIP-41:-Pluggable-Protocol-Handler.md @@ -1,8 +1,8 @@ -* **Status**: Drafting +* **Status**: Accepted * **Author**: Jia Zhai, Sijie Guo * **Pull Request**: * **Mailing List discussion**: -* **Release**: +* **Release**: 2.5.0 Design Proposal:
[pulsar.wiki] branch master updated: Updated Home (markdown)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new 73b3608 Updated Home (markdown) 73b3608 is described below commit 73b360814633236e5f29566ac457ddfad2c02b2b Author: Sijie Guo AuthorDate: Sun Mar 1 23:18:43 2020 -0800 Updated Home (markdown) --- Home.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Home.md b/Home.md index 114275f..7f321a5 100644 --- a/Home.md +++ b/Home.md @@ -7,6 +7,7 @@ We encourage to document any big change or feature or any addition to public use ### Proposed +* [[PIP 58 : Support Consumers Set Custom Retry Delay]] * [[PIP 57: Improve Broker's Zookeeper Session Timeout Handling]] * [PIP 56: Python3 Migration](https://github.com/apache/pulsar/wiki/PIP-56:-Python3-Migration) * [[PIP 55: Refresh Authentication Credentials]] @@ -20,7 +21,6 @@ We encourage to document any big change or feature or any addition to public use * [[PIP 46: Next-gen Proxy]] * [[PIP 45: Pluggable metadata interface]] * [[PIP 42: KoP - Kafka on Pulsar]] -* [[PIP 41: Pluggable Protocol Handler]] * [[PIP 39: Namespace Change Events]] * [[PIP 37: Large message size handling in Pulsar]] * [[PIP 35: Improve topic lookup for topics that have high number of partitions]] @@ -31,6 +31,7 @@ We encourage to document any big change or feature or any addition to public use * [[PIP 47: Time Based Release Plan]] * [[PIP 44: Separate schema compatibility checker for producer and consumer]] * [[PIP 43: producer send message with different schema]] +* [[PIP 41: Pluggable Protocol Handler]] * [[PIP 40: Pulsar Manager]] * [[PIP 38: Batch Receiving Messages]] * [[PIP 33: Replicated subscriptions]]
[pulsar.wiki] branch master updated: Created PIP 58 : Support Consumers Set Custom Retry Delay (markdown)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git
The following commit(s) were added to refs/heads/master by this push:
new d5268ee Created PIP 58 : Support Consumers Set Custom Retry Delay
(markdown)
d5268ee is described below
commit d5268ee41b71402ea598a645c032ccaf43022eba
Author: Sijie Guo
AuthorDate: Sun Mar 1 23:21:58 2020 -0800
Created PIP 58 : Support Consumers Set Custom Retry Delay (markdown)
---
...-:-Support-Consumers--Set-Custom-Retry-Delay.md | 73 ++
1 file changed, 73 insertions(+)
diff --git a/PIP-58-:-Support-Consumers--Set-Custom-Retry-Delay.md
b/PIP-58-:-Support-Consumers--Set-Custom-Retry-Delay.md
new file mode 100644
index 000..f81e354
--- /dev/null
+++ b/PIP-58-:-Support-Consumers--Set-Custom-Retry-Delay.md
@@ -0,0 +1,73 @@
+- Status: Proposal
+- Author: Dezhi Liu
+- Pull Request:
+- Mailing List discussion:
https://lists.apache.org/thread.html/r5ab7533b6461f29b8ff18e4c51e058226a1b3081fddc5649b234b7fc%40%3Cdev.pulsar.apache.org%3E
+- Release:
+
+## Motivation
+
+For many online business systems, various exceptions usually occur in
+business logic processing, so the message needs to be re-consumed, but
+users hope that this delay time can be controlled flexibly. The current
+user's processing method is usually to send this message to a special retry
+topic because production can specify any delay, so consumers subscribe to the
+business topic and retry topic at the same time. I think this logic can be
+supported by pulsar itself, making it easier for users to use, and it looks
+like this is a very common requirement.
+
+## Proposed changes
+
+This change can be supported on the client side, need to add a set of
+interfaces to org.apache.pulsar.client.api.Consumer
+
+```java
+void reconsumeLater(Message message, long delayTime, TimeUnit unit)
+throws PulsarClientException;
+CompletableFuture reconsumeLaterAsync(Message message, long
+delayTime, TimeUnit unit);
+CompletableFuture reconsumeLaterAsync(Messages messages, int
+delayLevel);
+ CompletableFuture reconsumeLaterCumulativeAsync(Message message,
+long delayTime, TimeUnit unit);
+```
+
+DeadLetterPolicy add retry topic
+```java
+public class DeadLetterPolicy {
+
+/**
+ * Maximum number of times that a message will be redelivered before
+being sent to the dead letter queue.
+ */
+private int maxRedeliverCount;
+
+/**
+ * Name of the retry topic where the failing messages will be sent.
+ */
+private String retryLetterTopic;
+
+/**
+ * Name of the dead topic where the failing messages will be sent.
+ */
+private String deadLetterTopic;
+
+}
+
+```
+
+org.apache.pulsar.client.impl.ConsumerImpl add a retry producer
+```java
+ private volatile Producer deadLetterProducer;
+
+ private volatile Producer retryLetterProducer;
+```
+
+Can specify whether to enable retry when creating a consumer，default
+is disabled.
+```java
+@Override
+public ConsumerBuilder enableRetry(boolean retryEnable) {
+conf.setRetryEnable(retryEnable);
+return this;
+}
+```
\ No newline at end of file
[pulsar] branch master updated (ec122ed -> 49a9897)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from ec122ed Check for the presence of presto libraries only when sql commands are used (#5883) add 49a9897 [Websocket] Websocket doesn't set the correct cluster data (#6102) No new revisions were added by this update. Summary of changes: .../src/main/java/org/apache/pulsar/broker/PulsarService.java | 10 +++--- .../java/org/apache/pulsar/websocket/WebSocketService.java | 2 ++ 2 files changed, 9 insertions(+), 3 deletions(-)
[pulsar] branch master updated (648ed0e -> ec122ed)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 648ed0e [pulsar-broker] Fix some comment typos (#5926) add ec122ed Check for the presence of presto libraries only when sql commands are used (#5883) No new revisions were added by this update. Summary of changes: bin/pulsar | 22 +- 1 file changed, 13 insertions(+), 9 deletions(-)
[pulsar] branch master updated (8f62eb5 -> 648ed0e)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 8f62eb5 Support CLI pulsar tokens for concise description. (#6037) add 648ed0e [pulsar-broker] Fix some comment typos (#5926) No new revisions were added by this update. Summary of changes: .../loadbalance/impl/ModularLoadManagerImpl.java | 4 ++-- .../apache/pulsar/broker/namespace/OwnedBundle.java| 2 +- .../apache/pulsar/broker/service/BrokerService.java| 6 +++--- .../service/persistent/PersistentSubscription.java | 4 ++-- .../service/schema/JsonSchemaCompatibilityCheck.java | 2 +- .../org/apache/pulsar/broker/admin/AdminApiTest.java | 2 +- .../org/apache/pulsar/broker/admin/AdminApiTest2.java | 4 ++-- .../loadbalance/AntiAffinityNamespaceGroupTest.java| 2 +- .../pulsar/broker/loadbalance/LoadBalancerTest.java| 4 ++-- .../pulsar/broker/namespace/NamespaceServiceTest.java | 2 +- .../broker/service/BrokerServiceThrottlingTest.java| 2 +- .../pulsar/client/api/BrokerServiceLookupTest.java | 18 +- .../org/apache/pulsar/client/api/ClientErrorsTest.java | 4 ++-- .../pulsar/client/api/ConsumerRedeliveryTest.java | 4 ++-- .../client/impl/BrokerClientIntegrationTest.java | 2 +- 15 files changed, 31 insertions(+), 31 deletions(-)
[pulsar] branch master updated (66ea4c3 -> 68c33a7)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 66ea4c3 enable zh-TW website translation (#5956) add 68c33a7 Fix typo in Reader documentation. (#6103) No new revisions were added by this update. Summary of changes: site2/docs/client-libraries-node.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[pulsar] branch master updated (9db9605 -> c09314c)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 9db9605 [website]Fixed website error (#6125) add c09314c Fix zeroQueueConsumer using listener (#6106) No new revisions were added by this update. Summary of changes: .../java/org/apache/pulsar/client/impl/ZeroQueueConsumerImpl.java| 5 - 1 file changed, 4 insertions(+), 1 deletion(-)
[pulsar] branch master updated (c09314c -> 66ea4c3)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from c09314c Fix zeroQueueConsumer using listener (#6106) add 66ea4c3 enable zh-TW website translation (#5956) No new revisions were added by this update. Summary of changes: site2/website/languages.js | 5 - 1 file changed, 4 insertions(+), 1 deletion(-)
[pulsar] branch master updated (68c33a7 -> 8f62eb5)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 68c33a7 Fix typo in Reader documentation. (#6103) add 8f62eb5 Support CLI pulsar tokens for concise description. (#6037) No new revisions were added by this update. Summary of changes: .../main/java/org/apache/pulsar/utils/auth/tokens/TokensCliUtils.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
[pulsar] branch master updated (c078a55 -> 9db9605)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git.
from c078a55 [CPP] Compile fixes for ARM cpus (#6129)
add 9db9605 [website]Fixed website error (#6125)
No new revisions were added by this update.
Summary of changes:
site2/docs/io-quickstart.md| 2 +-
site2/docs/io-use.md | 44 ++--
site2/website/pages/en/download.js | 4 +-
site2/website/sidebars.json| 1 -
.../versioned_docs/version-2.4.0/io-quickstart.md | 2 +-
.../versioned_docs/version-2.5.0/io-quickstart.md | 2 +-
.../versioned_docs/version-2.5.0}/io-use.md| 48 +++---
.../versioned_sidebars/version-2.5.0-sidebars.json | 1 -
8 files changed, 52 insertions(+), 52 deletions(-)
copy site2/{docs => website/versioned_docs/version-2.5.0}/io-use.md (95%)
[pulsar] branch master updated: [doc] Invalid code tab markdown for TW translation (#6142)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 91fbc32 [doc] Invalid code tab markdown for TW translation (#6142) 91fbc32 is described below commit 91fbc32966fe2dc4cd3b650cbe04b87366f1fd62 Author: Sijie Guo AuthorDate: Sun Jan 26 00:55:49 2020 -0600 [doc] Invalid code tab markdown for TW translation (#6142) *Motivation* Mitigate the issue described in #5816 (crowdin removed code tab when generating translated files) --- site2/tools/build-site.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/site2/tools/build-site.sh b/site2/tools/build-site.sh index 3fe1f42..41ced3d 100755 --- a/site2/tools/build-site.sh +++ b/site2/tools/build-site.sh @@ -41,6 +41,9 @@ cp versioned_docs/version-2.5.0/io-overview.md translated_docs/ja/version-2.5.0/ cp versioned_docs/version-2.4.2/functions-develop.md translated_docs/fr/version-2.4.2/functions-develop.md cp versioned_docs/version-2.5.0/functions-develop.md translated_docs/fr/version-2.5.0/functions-develop.md cp versioned_docs/version-2.5.0/io-overview.md translated_docs/fr/version-2.5.0/io-overview.md +cp versioned_docs/version-2.4.2/functions-develop.md translated_docs/zh-TW/version-2.4.2/functions-develop.md +cp versioned_docs/version-2.5.0/functions-develop.md translated_docs/zh-TW/version-2.5.0/functions-develop.md +cp versioned_docs/version-2.5.0/io-overview.md translated_docs/zh-TW/version-2.5.0/io-overview.md yarn build
[pulsar] branch master updated (472f448 -> c90854a)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 472f448 [Issue 6056][docs] Add example on how to choose partition when using a key (#6065) add c90854a make acker transient (#6064) No new revisions were added by this update. Summary of changes: .../src/main/java/org/apache/pulsar/client/impl/BatchMessageIdImpl.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[pulsar] branch master updated (3035aa6 -> 472f448)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 3035aa6 [Issue 5996][docs] Update broken links in 2.4.1 and 2.4.2 releases (#6080) add 472f448 [Issue 6056][docs] Add example on how to choose partition when using a key (#6065) No new revisions were added by this update. Summary of changes: site2/docs/cookbooks-partitioned.md | 41 + 1 file changed, 28 insertions(+), 13 deletions(-)
[pulsar] branch master updated (f4ad1e8 -> 138f662)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from f4ad1e8 Improve documentation about batching (#5989) add 138f662 [Issue 5983][docs] promote assign the advertisedAddress when startup Pulsar standalone (#5998) No new revisions were added by this update. Summary of changes: .../website/versioned_docs/version-2.4.2/getting-started-standalone.md | 2 ++ 1 file changed, 2 insertions(+)
[pulsar] branch master updated (c90854a -> f4ad1e8)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from c90854a make acker transient (#6064) add f4ad1e8 Improve documentation about batching (#5989) No new revisions were added by this update. Summary of changes: site2/docs/concepts-messaging.md | 17 +++-- 1 file changed, 15 insertions(+), 2 deletions(-)
[pulsar] branch master updated (0aee35a -> 3035aa6)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 0aee35a Support for python native logging from python wrapper (#6113) add 3035aa6 [Issue 5996][docs] Update broken links in 2.4.1 and 2.4.2 releases (#6080) No new revisions were added by this update. Summary of changes: site2/website/versioned_docs/version-2.4.1/functions-overview.md | 2 +- site2/website/versioned_docs/version-2.4.2/functions-overview.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
[pulsar] branch master updated (56280ea -> 0aee35a)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 56280ea Fix issue 5505 (#6060) add 0aee35a Support for python native logging from python wrapper (#6113) No new revisions were added by this update. Summary of changes: pulsar-client-cpp/python/pulsar/__init__.py | 7 ++ pulsar-client-cpp/python/src/config.cc | 103 2 files changed, 110 insertions(+)
[pulsar] branch master updated (138f662 -> e23c6a8)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 138f662 [Issue 5983][docs] promote assign the advertisedAddress when startup Pulsar standalone (#5998) add e23c6a8 [Issue 5690][pulsar-io] Make type name of Elasticsearch sink connector configurable. (#6028) No new revisions were added by this update. Summary of changes: .../io/elasticsearch/ElasticSearchConfig.java | 9 ++ .../pulsar/io/elasticsearch/ElasticSearchSink.java | 4 +- .../io/elasticsearch/ElasticSearchConfigTests.java | 20 - .../src/test/resources/sinkConfig.yaml | 3 +- site2/docs/io-elasticsearch-sink.md| 100 +++-- 5 files changed, 126 insertions(+), 10 deletions(-)
[pulsar] branch master updated (2725132 -> 2976542)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 2725132 [docs] Correct language idenitifiers in IO quickstart documentation (#6145) add 2976542 [Issue 3289][docs 2.5.0] Append / to metrics url to fix documentation (#6144) No new revisions were added by this update. Summary of changes: site2/website/versioned_docs/version-2.5.0/deploy-monitoring.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[pulsar] branch master updated (91fbc32 -> 2725132)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 91fbc32 [doc] Invalid code tab markdown for TW translation (#6142) add 2725132 [docs] Correct language idenitifiers in IO quickstart documentation (#6145) No new revisions were added by this update. Summary of changes: site2/docs/io-quickstart.md | 32 1 file changed, 16 insertions(+), 16 deletions(-)
[pulsar] branch master updated (13d8ecd -> a5647ef)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 13d8ecd Add copy file for cli tools (#6210) add a5647ef Add acknowledgement explanation (#6212) No new revisions were added by this update. Summary of changes: site2/docs/concepts-messaging.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[pulsar] branch master updated (0d981fb -> 846e470)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 0d981fb [Doc] Fix issue 6186 - Add explanations for methods in producer and consumer are thread-safe in 5 language clients (#6214) add 846e470 [Github actions]Improve integration test (#6184) No new revisions were added by this update. Summary of changes: .github/workflows/ci-integration-messaging.yaml| 18 +- .github/workflows/ci-integration-schema.yaml | 18 +- .github/workflows/ci-integration-standalone.yaml | 18 +- .../workflows/ci-integration-tiered-filesystem.yaml| 18 +- .github/workflows/ci-integration-tiered-jcloud.yaml| 18 +- 5 files changed, 85 insertions(+), 5 deletions(-)
[pulsar] branch master updated (a5647ef -> 0d981fb)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from a5647ef Add acknowledgement explanation (#6212) add 0d981fb [Doc] Fix issue 6186 - Add explanations for methods in producer and consumer are thread-safe in 5 language clients (#6214) No new revisions were added by this update. Summary of changes: site2/docs/client-libraries-cpp.md| 4 site2/docs/client-libraries-go.md | 2 ++ site2/docs/client-libraries-java.md | 4 +++- site2/docs/client-libraries-node.md | 2 ++ site2/docs/client-libraries-python.md | 2 ++ 5 files changed, 13 insertions(+), 1 deletion(-)
[pulsar-test-infra] branch master created (now 01759a9)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-test-infra.git. at 01759a9 Initialize the project This branch includes the following new commits: new 01759a9 Initialize the project The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[pulsar-test-infra] 01/01: Initialize the project
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-test-infra.git commit 01759a9886bbef113dea3ef609d33fc1d65166cd Author: Sijie Guo AuthorDate: Tue Feb 4 12:49:04 2020 -0800 Initialize the project --- .asf.yaml | 21 + README.md | 3 +++ 2 files changed, 24 insertions(+) diff --git a/.asf.yaml b/.asf.yaml new file mode 100644 index 000..4ae4db7 --- /dev/null +++ b/.asf.yaml @@ -0,0 +1,21 @@ +github: + description: "Testing infrastructure for Apache Pulsar" + homepage: https://pulsar.apache.org/ + labels: +- pulsar +- pubsub +- test-infra + features: +# Enable wiki for documentation +wiki: true +# Enable issues management +issues: true +# Enable projects for project management boards +projects: true + enabled_merge_buttons: +# enable squash button: +squash: true +# disable merge button: +merge: false +# disable rebase button: +rebase: false \ No newline at end of file diff --git a/README.md b/README.md new file mode 100644 index 000..916446c --- /dev/null +++ b/README.md @@ -0,0 +1,3 @@ +# Apache Pulsar Test Infrastructure + +This repo is used for keeping all the scripts and github actions for running Pulsar test infrastructure. \ No newline at end of file
[pulsar] branch master updated (0b06dba -> af6b5f6)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git.
from 0b06dba Remove duplicated lombok annotations in the pulsar-client
module (#6147)
add af6b5f6 [Issue 6024][pulsar_storm] PulsarSpout emit to multiple
streams (#6039)
No new revisions were added by this update.
Summary of changes:
.../apache/pulsar/storm/MessageToValuesMapper.java | 4 +--
.../java/org/apache/pulsar/storm/PulsarSpout.java | 6 +++-
...MessageToValuesMapper.java => PulsarTuple.java} | 31 +--
.../org/apache/pulsar/storm/PulsarSpoutTest.java | 35 --
4 files changed, 56 insertions(+), 20 deletions(-)
copy
pulsar-storm/src/main/java/org/apache/pulsar/storm/{MessageToValuesMapper.java
=> PulsarTuple.java} (64%)
[pulsar] branch master updated (0e8d533 -> 3b46f50)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 0e8d533 Replace client api use correct version (#6240) add 3b46f50 [Doc] Fix issue 6177 - Correct explanation for routing mode (#6244) No new revisions were added by this update. Summary of changes: site2/docs/concepts-messaging.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[pulsar] branch master updated (3b46f50 -> d631156)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git.
from 3b46f50 [Doc] Fix issue 6177 - Correct explanation for routing mode
(#6244)
add d631156 [deployement] make kubernetes yamls for aws operational
(#6192)
No new revisions were added by this update.
Summary of changes:
deployment/kubernetes/aws/bookkeeper.yaml | 27 +++---
deployment/kubernetes/aws/broker.yaml | 18 +++
.../cluster-metadata.yaml | 6 ++---
deployment/kubernetes/aws/proxy.yaml | 9
4 files changed, 30 insertions(+), 30 deletions(-)
copy deployment/kubernetes/{generic/k8s-1-9-and-above =>
aws}/cluster-metadata.yaml (86%)
[pulsar] branch master updated (d631156 -> 44dd412)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from d631156 [deployement] make kubernetes yamls for aws operational (#6192) add 44dd412 Fix get schema version in HttpLookupService. (#6193) No new revisions were added by this update. Summary of changes: .../apache/pulsar/client/api/SimpleSchemaTest.java | 33 ++ .../pulsar/client/impl/HttpLookupService.java | 7 ++--- 2 files changed, 36 insertions(+), 4 deletions(-)
[pulsar] branch master updated (4a2bceb -> 92d7102)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 4a2bceb [CI] Add GITHUB_TOKEN as the environment variable for running pulsarbot (#6225) add 92d7102 Revert "Support for python native logging from python wrapper (#6113)" (#6174) No new revisions were added by this update. Summary of changes: .github/workflows/ci-integration-sql.yaml | 12 +++ .github/workflows/ci-unit-adaptors.yml | 15 +++ .github/workflows/ci-unit-broker-sasl.yml | 15 +++ .github/workflows/ci-unit-broker.yml | 14 +++ .github/workflows/ci-unit-flaky.yaml | 18 +++- .github/workflows/ci-unit-proxy.yaml | 19 +++- .github/workflows/ci-unit.yaml | 3 +- .../pulsar/broker/service/ServerCnxTest.java | 2 +- .../pulsar/client/api/PartitionCreationTest.java | 2 +- .../client/api/v1/V1_ProducerConsumerTest.java | 1 + .../client/impl/BrokerClientIntegrationTest.java | 9 +- pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc | 1 - pulsar-client-cpp/python/pulsar/__init__.py| 7 -- pulsar-client-cpp/python/src/config.cc | 103 - pulsar-client-cpp/tests/BasicEndToEndTest.cc | 7 +- .../rest/api/v2/FunctionApiV2ResourceTest.java | 24 +++-- .../rest/api/v3/FunctionApiV3ResourceTest.java | 24 +++-- .../org/apache/pulsar/proxy/server/ProxyTest.java | 49 +- .../tests/integration/cli/FunctionsCLITest.java| 3 +- 19 files changed, 166 insertions(+), 162 deletions(-)
[pulsar] branch master updated (92d7102 -> 0e8d533)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 92d7102 Revert "Support for python native logging from python wrapper (#6113)" (#6174) add 0e8d533 Replace client api use correct version (#6240) No new revisions were added by this update. Summary of changes: site2/website/pages/en/pulsar-admin-cli.js | 23 site2/website/scripts/replace.js | 34 +++--- site2/website/sidebars.json| 1 - site2/website/siteConfig.js| 3 ++- site2/website/static/css/custom.css| 4 site2/website/static/js/custom.js | 25 ++ site2/website/static/js/getCliByVersion.js | 34 ++ 7 files changed, 119 insertions(+), 5 deletions(-) create mode 100644 site2/website/pages/en/pulsar-admin-cli.js create mode 100644 site2/website/static/js/getCliByVersion.js
[pulsar] branch master updated (125a588 -> 54b39e6)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 125a588 Fixed static linking on C++ lib on MacOS (#5581) add 54b39e6 Fix bug that backlog message that has not yet expired could be deleted due to TTL (#6211) No new revisions were added by this update. Summary of changes: .../main/java/org/apache/bookkeeper/mledger/impl/OpFindNewest.java | 5 - .../java/org/apache/bookkeeper/mledger/impl/ManagedCursorTest.java | 6 +++--- .../apache/pulsar/broker/service/PersistentMessageFinderTest.java | 2 +- 3 files changed, 8 insertions(+), 5 deletions(-)
[pulsar] branch master updated (75c145a -> 6511763)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 75c145a Fix mispelling of tolarations. Correctly spelled at tolerations. (#6265) add 6511763 add Korean code base. (#6261) No new revisions were added by this update. Summary of changes: site2/website/languages.js | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-)
[pulsar] branch master updated (98cf15f -> fed8c30)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 98cf15f Allow path to bash scripts to contain symbolic links from anywhere (#6245) add fed8c30 [Issue 5927][Pulsar Client] Change the time unit of `patternAutoDiscoveryPeriod` to SECONDS (#5950) No new revisions were added by this update. Summary of changes: .../pulsar/client/impl/PatternTopicsConsumerImplTest.java | 2 +- .../org/apache/pulsar/client/api/ConsumerBuilder.java | 15 +++ .../apache/pulsar/client/impl/ConsumerBuilderImpl.java| 10 +- .../client/impl/PatternMultiTopicsConsumerImpl.java | 8 .../client/impl/conf/ConsumerConfigurationData.java | 2 +- .../pulsar/client/impl/ConsumerBuilderImplTest.java | 8 +++- 6 files changed, 37 insertions(+), 8 deletions(-)
[pulsar] branch master updated (857d63b -> 9b296d8)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 857d63b Pin the netty-transport-native-epoll to avoid conflicts (#6194) add 9b296d8 Output resource usage rate to log on broker (#6152) No new revisions were added by this update. Summary of changes: .../pulsar/broker/loadbalance/impl/OverloadShedder.java| 8 +--- .../broker/loadbalance/impl/OverloadShedderTest.java | 14 ++ .../pulsar/policies/data/loadbalancer/LocalBrokerData.java | 7 +++ 3 files changed, 26 insertions(+), 3 deletions(-)
[pulsar] branch master updated (6511763 -> 28875d5)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 6511763 add Korean code base. (#6261) add 28875d5 [ISSUE-6131]: Ensure JVM memory and GC options are set for bookie (#6201) No new revisions were added by this update. Summary of changes: conf/bkenv.sh | 2 +- .../kubernetes/helm/pulsar/templates/bookkeeper-statefulset.yaml | 1 + deployment/kubernetes/helm/pulsar/values.yaml | 3 ++- 3 files changed, 4 insertions(+), 2 deletions(-)
[pulsar] branch master updated (43d89f2 -> 857d63b)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 43d89f2 Create managed ledger path on local zookeeper when create partitions (#6189) add 857d63b Pin the netty-transport-native-epoll to avoid conflicts (#6194) No new revisions were added by this update. Summary of changes: distribution/server/src/assemble/LICENSE.bin.txt | 3 ++- pom.xml | 14 ++ pulsar-common/pom.xml| 2 ++ pulsar-sql/presto-distribution/LICENSE | 3 ++- 4 files changed, 20 insertions(+), 2 deletions(-)
[pulsar] branch master updated: [docs] improve statement on ordering guarantees for multi-topic subscriptions (#6132)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 73c013e [docs] improve statement on ordering guarantees for
multi-topic subscriptions (#6132)
73c013e is described below
commit 73c013eba4e452aaeb811991733e9b1620ae4093
Author: EugenDueck
AuthorDate: Sat Jan 25 12:31:21 2020 +0900
[docs] improve statement on ordering guarantees for multi-topic
subscriptions (#6132)
### Motivation
*The statement "ordering guarantees [..] on single topics do not hold" on
https://pulsar.apache.org/docs/en/concepts-messaging/#no-ordering-guarantees
can lead readers to believe that messages on the same topic are not guaranteed
to be delivered in order, which I believe to be false, as discussed in this
thread https://apache-pulsar.slack.com/archives/C5Z4T36F7/p1578264274257300*
### Modifications
*I've applied the same change of wording to the current
`site2/docs/concepts-messaging.md` and all the various versions under
`site2/website/versioned_docs/`, as this is independent of the version.*
#5995
---
site2/docs/concepts-messaging.md| 6 +++---
.../versioned_docs/version-2.1.0-incubating/concepts-messaging.md | 4 ++--
site2/website/versioned_docs/version-2.3.0/concepts-messaging.md| 4 ++--
site2/website/versioned_docs/version-2.3.1/concepts-messaging.md| 4 ++--
site2/website/versioned_docs/version-2.3.2/concepts-messaging.md| 4 ++--
site2/website/versioned_docs/version-2.4.0/concepts-messaging.md| 4 ++--
site2/website/versioned_docs/version-2.4.1/concepts-messaging.md| 4 ++--
site2/website/versioned_docs/version-2.4.2/concepts-messaging.md| 4 ++--
site2/website/versioned_docs/version-2.5.0/concepts-messaging.md| 4 ++--
9 files changed, 19 insertions(+), 19 deletions(-)
diff --git a/site2/docs/concepts-messaging.md b/site2/docs/concepts-messaging.md
index f950701..b6f49c3 100644
--- a/site2/docs/concepts-messaging.md
+++ b/site2/docs/concepts-messaging.md
@@ -239,8 +239,8 @@ When a consumer subscribes to a Pulsar topic, by default it
subscribes to one sp
When subscribing to multiple topics, the Pulsar client will automatically make
a call to the Pulsar API to discover the topics that match the regex
pattern/list and then subscribe to all of them. If any of the topics don't
currently exist, the consumer will auto-subscribe to them once the topics are
created.
-> No ordering guarantees
-> When a consumer subscribes to multiple topics, all ordering guarantees
normally provided by Pulsar on single topics do not hold. If your use case for
Pulsar involves any strict ordering requirements, we would strongly recommend
against using this feature.
+> No ordering guarantees across multiple topics
+> When a producer sends messages to a single topic, all messages are
guaranteed to be read from that topic in the same order. However, these
guarantees do not hold across multiple topics. So when a producer sends message
to multiple topics, the order in which messages are read from those topics is
not guaranteed to be the same.
Here are some multi-topic subscription examples for Java:
@@ -454,4 +454,4 @@ The following is an example of delayed message delivery for
a producer in Java:
```java
// message to be delivered at the configured delay interval
producer.newMessage().deliverAfter(3L, TimeUnit.Minute).value("Hello
Pulsar!").send();
-```
\ No newline at end of file
+```
diff --git
a/site2/website/versioned_docs/version-2.1.0-incubating/concepts-messaging.md
b/site2/website/versioned_docs/version-2.1.0-incubating/concepts-messaging.md
index 9b0caf3..1df8f92 100644
---
a/site2/website/versioned_docs/version-2.1.0-incubating/concepts-messaging.md
+++
b/site2/website/versioned_docs/version-2.1.0-incubating/concepts-messaging.md
@@ -150,8 +150,8 @@ When a consumer subscribes to a Pulsar topic, by default it
subscribes to one sp
When subscribing to multiple topics, the Pulsar client will automatically make
a call to the Pulsar API to discover the topics that match the regex
pattern/list and then subscribe to all of them. If any of the topics don't
currently exist, the consumer will auto-subscribe to them once the topics are
created.
-> No ordering guarantees
-> When a consumer subscribes to multiple topics, all ordering guarantees
normally provided by Pulsar on single topics do not hold. If your use case for
Pulsar involves any strict ordering requirements, we would strongly recommend
against using this feature.
+> No ordering guarantees across multiple topics
+> When a producer sends messages to a single topic, all messages are
guaranteed to be read from that topic in the same order. However, these
guarantees
[pulsar] branch master updated: make bookie ranges stored in path/to/bkdata/ranges (#6082)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 5ae7a00 make bookie ranges stored in path/to/bkdata/ranges (#6082)
5ae7a00 is described below
commit 5ae7a002edd12c9255f37e4f0b05bf1b374de947
Author: hanbo1990
AuthorDate: Sat Jan 25 04:20:59 2020 +0100
make bookie ranges stored in path/to/bkdata/ranges (#6082)
Fixes #5326
### Motivation
After this fix, pulsar will not create data/bookkeeper/ranges in the pulsar
library folder after executing `bin/pulsar standalone --bookkeeper-dir dir1
--zookeeper-dir dir2`. All the bookie ranges will be stored in bookkeeper's
data directory.
### Modifications
Pass `storage.range.store.dirs` to storage server when starting pulsar.
Hardcode ranges's dir to ${path/to/bookie/datadir}/ranges/data, just under
bookie.
### Verifying this change
Follow the steps in [issue
5326](https://github.com/apache/pulsar/issues/5326) and start pulsar with
`bin/pulsar standalone --bookkeeper-dir dir1 --zookeeper-dir dir2`
---
.../main/java/org/apache/pulsar/zookeeper/LocalBookkeeperEnsemble.java | 3 +++
1 file changed, 3 insertions(+)
diff --git
a/pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/LocalBookkeeperEnsemble.java
b/pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/LocalBookkeeperEnsemble.java
index 0a743c8..2658035 100644
---
a/pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/LocalBookkeeperEnsemble.java
+++
b/pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/LocalBookkeeperEnsemble.java
@@ -310,6 +310,9 @@ public class LocalBookkeeperEnsemble {
// stream storage port
conf.setProperty("storageserver.grpc.port", streamStoragePort);
+// storage server settings
+conf.setProperty("storage.range.store.dirs", bkDataDirName +
"/ranges/data");
+
// initialize the stream storage metadata
ClusterInitializer initializer = new ZkClusterInitializer(zkServers);
initializer.initializeCluster(metadataServiceUri, 2);
