[incubator-pulsar] branch master updated: Shorten the timeout value of C++ ZTS client (#2572)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new f076974 Shorten the timeout value of C++ ZTS client (#2572)
f076974 is described below
commit f076974935feeece00d32094c16572ac4f45244c
Author: massakam
AuthorDate: Thu Sep 20 09:11:45 2018 +0900
Shorten the timeout value of C++ ZTS client (#2572)
* Shorten the timeout value of C++ ZTS client
* Set REQUEST_TIMEOUT to 3ms
---
pulsar-client-cpp/lib/auth/athenz/ZTSClient.cc | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/pulsar-client-cpp/lib/auth/athenz/ZTSClient.cc
b/pulsar-client-cpp/lib/auth/athenz/ZTSClient.cc
index 93010af..3097c30 100644
--- a/pulsar-client-cpp/lib/auth/athenz/ZTSClient.cc
+++ b/pulsar-client-cpp/lib/auth/athenz/ZTSClient.cc
@@ -45,7 +45,7 @@ namespace pulsar {
const static std::string DEFAULT_PRINCIPAL_HEADER = "Athenz-Principal-Auth";
const static std::string DEFAULT_ROLE_HEADER = "Athenz-Role-Auth";
-const static int REQUEST_TIMEOUT = 1;
+const static int REQUEST_TIMEOUT = 3;
const static int DEFAULT_TOKEN_EXPIRATION_TIME_SEC = 3600;
const static int MIN_TOKEN_EXPIRATION_TIME_SEC = 900;
const static int MAX_HTTP_REDIRECTS = 20;
@@ -272,7 +272,7 @@ const std::string ZTSClient::getRoleToken() const {
curl_easy_setopt(handle, CURLOPT_NOSIGNAL, 1L);
// Timer
-curl_easy_setopt(handle, CURLOPT_TIMEOUT, REQUEST_TIMEOUT);
+curl_easy_setopt(handle, CURLOPT_TIMEOUT_MS, REQUEST_TIMEOUT);
// Redirects
curl_easy_setopt(handle, CURLOPT_FOLLOWLOCATION, 1L);
[incubator-pulsar] branch master updated: Enforce boost-python was found by CMake (#2608)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new d536d66 Enforce boost-python was found by CMake (#2608)
d536d66 is described below
commit d536d6669eb79c7b4235403c43ef5d6cf3b79dcd
Author: Matteo Merli
AuthorDate: Wed Sep 19 17:03:52 2018 -0700
Enforce boost-python was found by CMake (#2608)
* Enforce boost-python was found by CMake
* Fix for linux build as well
---
pulsar-client-cpp/docker/build-wheels.sh | 1 +
pulsar-client-cpp/python/CMakeLists.txt | 9 +
2 files changed, 10 insertions(+)
diff --git a/pulsar-client-cpp/docker/build-wheels.sh
b/pulsar-client-cpp/docker/build-wheels.sh
index ebbb441..3292519 100755
--- a/pulsar-client-cpp/docker/build-wheels.sh
+++ b/pulsar-client-cpp/docker/build-wheels.sh
@@ -33,6 +33,7 @@ PYTHON_VERSIONS=(
'3.4 cp34-cp34m'
'3.5 cp35-cp35m'
'3.6 cp36-cp36m'
+ '3.7 cp37-cp37m'
)
function contains() {
diff --git a/pulsar-client-cpp/python/CMakeLists.txt
b/pulsar-client-cpp/python/CMakeLists.txt
index 9ddbc01..2c51f6d 100644
--- a/pulsar-client-cpp/python/CMakeLists.txt
+++ b/pulsar-client-cpp/python/CMakeLists.txt
@@ -48,8 +48,17 @@ endif()
set(PYTHON_WRAPPER_LIBS ${Boost_PYTHON_LIBRARY} ${Boost_PYTHON3_LIBRARY}
${Boost_PYTHON27-MT_LIBRARY}
${Boost_PYTHON37-MT_LIBRARY})
+if (APPLE)
+set(PYTHON_WRAPPER_LIBS ${PYTHON_WRAPPER_LIBS}
+ ${Boost_PYTHON27-MT_LIBRARY_RELEASE}
${Boost_PYTHON37-MT_LIBRARY_RELEASE})
+endif()
+
message(STATUS "Using Boost Python libs: ${PYTHON_WRAPPER_LIBS}")
+if (NOT PYTHON_WRAPPER_LIBS)
+MESSAGE(FATAL_ERROR "Could not find Boost Python library")
+endif ()
+
if (APPLE)
target_link_libraries(_pulsar -Wl,-all_load pulsarStatic
${PYTHON_WRAPPER_LIBS})
else ()
[incubator-pulsar] branch master updated: [Python] Consolidated duplicated subscribe_*() methods into a single one (#2580)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 99ad32b [Python] Consolidated duplicated subscribe_*() methods into a
single one (#2580)
99ad32b is described below
commit 99ad32b68565e8ee25049741336e5f58571b078a
Author: Matteo Merli
AuthorDate: Wed Sep 19 10:14:59 2018 -0700
[Python] Consolidated duplicated subscribe_*() methods into a single one
(#2580)
* [Python] Consolidated duplicated subscribe_*() methods into a single one
* Fixed missing parameter pattern_auto_discovery_period
---
pulsar-client-cpp/python/pulsar/__init__.py | 229 +++-
pulsar-client-cpp/python/pulsar_test.py | 26 ++--
2 files changed, 35 insertions(+), 220 deletions(-)
diff --git a/pulsar-client-cpp/python/pulsar/__init__.py
b/pulsar-client-cpp/python/pulsar/__init__.py
index 806c7e2..6849ecc 100644
--- a/pulsar-client-cpp/python/pulsar/__init__.py
+++ b/pulsar-client-cpp/python/pulsar/__init__.py
@@ -104,6 +104,9 @@ from pulsar.functions.function import Function
from pulsar.functions.context import Context
from pulsar.functions.serde import SerDe, IdentitySerDe, PickleSerDe
+import re
+_retype = type(re.compile('x'))
+
class MessageId:
"""
Represents a message id
@@ -412,114 +415,19 @@ class Client:
unacked_messages_timeout_ms=None,
broker_consumer_stats_cache_time_ms=3,
is_read_compacted=False,
- properties=None
+ properties=None,
+ pattern_auto_discovery_period=60
):
"""
Subscribe to the given topic and subscription combination.
**Args**
-* `topic`: The name of the topic.
-* `subscription`: The name of the subscription.
-
-**Options**
-
-* `consumer_type`:
- Select the subscription type to be used when subscribing to the
topic.
-* `message_listener`:
- Sets a message listener for the consumer. When the listener is set,
- the application will receive messages through it. Calls to
- `consumer.receive()` will not be allowed. The listener function needs
- to accept (consumer, message), for example:
-
-#!python
-def my_listener(consumer, message):
-# process message
-consumer.acknowledge(message)
-
-* `receiver_queue_size`:
- Sets the size of the consumer receive queue. The consumer receive
- queue controls how many messages can be accumulated by the consumer
- before the application calls `receive()`. Using a higher value could
- potentially increase the consumer throughput at the expense of higher
- memory utilization. Setting the consumer queue size to zero decreases
- the throughput of the consumer by disabling pre-fetching of messages.
- This approach improves the message distribution on shared
subscription
- by pushing messages only to those consumers that are ready to process
- them. Neither receive with timeout nor partitioned topics can be used
- if the consumer queue size is zero. The `receive()` function call
- should not be interrupted when the consumer queue size is zero. The
- default value is 1000 messages and should work well for most use
- cases.
-* `max_total_receiver_queue_size_across_partitions`
- Set the max total receiver queue size across partitions.
- This setting will be used to reduce the receiver queue size for
individual partitions
-* `consumer_name`:
- Sets the consumer name.
-* `unacked_messages_timeout_ms`:
- Sets the timeout in milliseconds for unacknowledged messages. The
- timeout needs to be greater than 10 seconds. An exception is thrown
if
- the given value is less than 10 seconds. If a successful
- acknowledgement is not sent within the timeout, all the
unacknowledged
- messages are redelivered.
-* `broker_consumer_stats_cache_time_ms`:
- Sets the time duration for which the broker-side consumer stats will
- be cached in the client.
-* `properties`:
- Sets the properties for the consumer. The properties associated with
a consumer
- can be used for identify a consumer at broker side.
-"""
-_check_type(str, topic, 'topic')
-_check_type(str, subscription_name, 'subscription_name')
-_check_type(ConsumerType, consumer_type, 'consumer_type')
-_check_type(int, receiver_queue_size, 'receiver_queue_size')
-_check_type(int, ma
[incubator-pulsar] branch asf-site updated: Updated site at revision ee5afa5
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/asf-site by this push: new af2c2fa Updated site at revision ee5afa5 af2c2fa is described below commit af2c2fabb89b7bf3f5cab770233090ee90819ec3 Author: jenkins AuthorDate: Wed Sep 19 08:53:40 2018 + Updated site at revision ee5afa5 --- content/api/client/allclasses-frame.html | 1 + content/api/client/allclasses-noframe.html | 1 + content/api/client/index-all.html | 30 .../apache/pulsar/client/api/ClientBuilder.html| 25 ++- .../org/apache/pulsar/client/api/PulsarClient.html | 82 ++ .../apache/pulsar/client/api/ReaderListener.html | 4 +- ...ReaderListener.html => ServiceUrlProvider.html} | 77 +--- .../client/api/SubscriptionInitialPosition.html| 4 +- .../apache/pulsar/client/api/package-frame.html| 1 + .../apache/pulsar/client/api/package-summary.html | 9 ++- .../org/apache/pulsar/client/api/package-tree.html | 1 + content/api/client/overview-tree.html | 1 + content/docs/en/next/pulsar-admin.html | 4 ++ content/docs/en/next/pulsar-admin/index.html | 4 ++ content/docs/fr/next/pulsar-admin.html | 4 ++ content/docs/fr/next/pulsar-admin/index.html | 4 ++ content/docs/ja/next/pulsar-admin.html | 4 ++ content/docs/ja/next/pulsar-admin/index.html | 4 ++ .../docs/latest/adaptors/PulsarSpark/index.html| 10 +-- .../docs/latest/adaptors/PulsarStorm/index.html| 8 +-- content/docs/latest/admin-api/overview/index.html | 18 ++--- content/docs/latest/clients/Cpp/index.html | 8 +-- content/docs/latest/clients/Java/index.html| 16 ++--- content/docs/latest/clients/Python/index.html | 10 +-- content/docs/latest/clients/WebSocket/index.html | 8 +-- content/docs/latest/clients/go/index.html | 6 +- .../docs/latest/cookbooks/Encryption/index.html| 6 +- .../latest/cookbooks/PartitionedTopics/index.html | 16 ++--- .../latest/cookbooks/RetentionExpiry/index.html| 16 ++--- .../docs/latest/cookbooks/compaction/index.html| 8 +-- .../cookbooks/message-deduplication/index.html | 12 ++-- .../docs/latest/cookbooks/message-queue/index.html | 16 ++--- .../latest/cookbooks/tiered-storage/index.html | 6 +- .../docs/latest/deployment/Kubernetes/index.html | 4 +- .../docs/latest/deployment/aws-cluster/index.html | 6 +- content/docs/latest/deployment/cluster/index.html | 6 +- content/docs/latest/deployment/instance/index.html | 6 +- .../ConceptsAndArchitecture/index.html | 4 +- .../latest/getting-started/LocalCluster/index.html | 4 +- .../latest/getting-started/Pulsar-2.0/index.html | 2 +- .../docs/latest/getting-started/docker/index.html | 4 +- .../docs/latest/project/BinaryProtocol/index.html | 4 +- content/docs/latest/project/CompileCpp/index.html | 8 +-- .../docs/latest/project/SimulationTools/index.html | 2 +- .../docs/latest/project/schema-storage/index.html | 4 +- content/docs/latest/reference/CliTools/index.html | 22 +++--- .../docs/latest/security/authorization/index.html | 14 ++-- content/docs/latest/security/encryption/index.html | 6 +- content/docs/zh-CN/next/pulsar-admin.html | 4 ++ content/docs/zh-CN/next/pulsar-admin/index.html| 4 ++ content/ja/adaptors/PulsarSpark/index.html | 8 +-- content/ja/adaptors/PulsarStorm/index.html | 6 +- content/ja/admin/AdminInterface/index.html | 12 ++-- content/ja/admin/Authz/index.html | 12 ++-- content/ja/admin/ClustersBrokers/index.html| 6 +- content/ja/admin/PropertiesNamespaces/index.html | 6 +- content/ja/advanced/PartitionedTopics/index.html | 12 ++-- content/ja/advanced/RetentionExpiry/index.html | 12 ++-- content/ja/clients/Cpp/index.html | 6 +- content/ja/clients/Java/index.html | 8 +-- content/ja/clients/Python/index.html | 8 +-- content/ja/clients/WebSocket/index.html| 8 +-- content/ja/deployment/InstanceSetup/index.html | 6 +- content/ja/deployment/Kubernetes/index.html| 4 +- .../ConceptsAndArchitecture/index.html | 2 +- content/ja/getting-started/LocalCluster/index.html | 4 +- content/ja/project/BinaryProtocol/index.html | 4 +- content/ja/project/SimulationTools/index.html | 2 +- content/ja/reference/CliTools/index.html | 18 ++--- content/swagger/swagger.json | 34 - 70 files changed, 424 insertions(+), 282 deletions(-) diff --git a/content/api/client/allclasses-frame.html b/content/api/client/allclasses-frame.html index 3244593..bd10c76 100644 --- a/content/api/cli
[incubator-pulsar] branch asf-site updated: Trigger asf-site sync
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/asf-site by this push: new 161473e Trigger asf-site sync 161473e is described below commit 161473ec2000373399fa85305f6d9e02537cda1d Author: Matteo Merli AuthorDate: Tue Sep 18 21:38:49 2018 -0700 Trigger asf-site sync
[incubator-pulsar] 01/03: Fixed linking of boost and boost-python for osx build (#2302)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch py-build-fix-2.1.1 in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git commit 6de19c8d6e4fbe322df7e2f02af5df661626 Author: Matteo Merli AuthorDate: Fri Aug 3 19:31:22 2018 +0900 Fixed linking of boost and boost-python for osx build (#2302) Build of wheels files in OSX is broken due to failures in finding boost after installing from brew. --- pulsar-client-cpp/python/pkg/osx/build.sh.template | 14 +++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/pulsar-client-cpp/python/pkg/osx/build.sh.template b/pulsar-client-cpp/python/pkg/osx/build.sh.template old mode 100644 new mode 100755 index 5720b27..71687ac --- a/pulsar-client-cpp/python/pkg/osx/build.sh.template +++ b/pulsar-client-cpp/python/pkg/osx/build.sh.template @@ -23,21 +23,29 @@ set -e GITTAG=#TAG# PYTHONVER=#PYTHONVER# +BOOST_VERSION=1.67 + /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" -brew install openssl git boost pkg-config jsoncpp cmake protobuf260 log4cxx +brew install openssl git boost@$BOOST_VERSION pkg-config jsoncpp cmake protobuf260 log4cxx + +brew link --force boost@$BOOST_VERSION if [ "$PYTHONVER" = "PYTHON2" ] then - brew install python@2 boost-python + brew install python@2 boost-python@$BOOST_VERSION + + brew link --force boost-python@$BOOST_VERSION fi if [ "$PYTHONVER" = "PYTHON3" ] then - brew install python boost-python3 + brew install python boost-python3@BOOST_VERSION + brew link --force boost-python3@$BOOST_VERSION fi brew link --force protobuf260 + rm -rf incubator-pulsar git clone --depth 1 --branch $GITTAG https://github.com/apache/incubator-pulsar.git cd incubator-pulsar/pulsar-client-cpp
[incubator-pulsar] branch py-build-fix-2.1.1 created (now fbc19fd)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch py-build-fix-2.1.1 in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git. at fbc19fd Enforce boost-python was found by CMake This branch includes the following new commits: new 6de19c8 Fixed linking of boost and boost-python for osx build (#2302) new e65143a Fixed linking of python wrapper to boost-python in MacOS build (#2366) new fbc19fd Enforce boost-python was found by CMake The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[incubator-pulsar] 03/03: Enforce boost-python was found by CMake
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch py-build-fix-2.1.1
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
commit fbc19fde3d0983daabc70b50ad6f7ec8ea9119af
Author: Matteo Merli
AuthorDate: Tue Sep 18 20:33:58 2018 -0700
Enforce boost-python was found by CMake
---
pulsar-client-cpp/python/CMakeLists.txt | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/pulsar-client-cpp/python/CMakeLists.txt
b/pulsar-client-cpp/python/CMakeLists.txt
index d506844..78fcc67 100644
--- a/pulsar-client-cpp/python/CMakeLists.txt
+++ b/pulsar-client-cpp/python/CMakeLists.txt
@@ -37,7 +37,12 @@ endif()
# Try all possible boost-python variable namings
set(PYTHON_WRAPPER_LIBS ${Boost_PYTHON_LIBRARY} ${Boost_PYTHON3_LIBRARY}
-${Boost_PYTHON27-MT_LIBRARY}
${Boost_PYTHON37-MT_LIBRARY})
+${Boost_PYTHON27-MT_LIBRARY}
${Boost_PYTHON37-MT_LIBRARY}
+${Boost_PYTHON27-MT_LIBRARY_RELEASE}
${Boost_PYTHON37-MT_LIBRARY_RELEASE})
+
+if (NOT PYTHON_WRAPPER_LIBS)
+MESSAGE(FATAL_ERROR "Could not find Boost Python library")
+endif ()
if (APPLE)
target_link_libraries(_pulsar -Wl,-all_load pulsarStatic
${PYTHON_WRAPPER_LIBS})
[incubator-pulsar] 02/03: Fixed linking of python wrapper to boost-python in MacOS build (#2366)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch py-build-fix-2.1.1
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
commit e65143a97ad5277682f84cd5f11fe7ad62c78e9c
Author: Matteo Merli
AuthorDate: Mon Aug 13 16:56:23 2018 -0700
Fixed linking of python wrapper to boost-python in MacOS build (#2366)
---
pulsar-client-cpp/python/CMakeLists.txt| 4 +++-
pulsar-client-cpp/python/pkg/osx/build.sh.template | 14 ++
2 files changed, 13 insertions(+), 5 deletions(-)
diff --git a/pulsar-client-cpp/python/CMakeLists.txt
b/pulsar-client-cpp/python/CMakeLists.txt
index 8e70f72..d506844 100644
--- a/pulsar-client-cpp/python/CMakeLists.txt
+++ b/pulsar-client-cpp/python/CMakeLists.txt
@@ -35,7 +35,9 @@ if("${CMAKE_CXX_COMPILER_ID}" STREQUAL "Clang")
set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS
"${CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS} -undefined dynamic_lookup")
endif()
-set(PYTHON_WRAPPER_LIBS ${Boost_PYTHON_LIBRARY} ${Boost_PYTHON3_LIBRARY})
+# Try all possible boost-python variable namings
+set(PYTHON_WRAPPER_LIBS ${Boost_PYTHON_LIBRARY} ${Boost_PYTHON3_LIBRARY}
+${Boost_PYTHON27-MT_LIBRARY}
${Boost_PYTHON37-MT_LIBRARY})
if (APPLE)
target_link_libraries(_pulsar -Wl,-all_load pulsarStatic
${PYTHON_WRAPPER_LIBS})
diff --git a/pulsar-client-cpp/python/pkg/osx/build.sh.template
b/pulsar-client-cpp/python/pkg/osx/build.sh.template
index 71687ac..40b7420 100755
--- a/pulsar-client-cpp/python/pkg/osx/build.sh.template
+++ b/pulsar-client-cpp/python/pkg/osx/build.sh.template
@@ -27,7 +27,7 @@ BOOST_VERSION=1.67
/usr/bin/ruby -e "$(curl -fsSL
https://raw.githubusercontent.com/Homebrew/install/master/install)"
-brew install openssl git boost@$BOOST_VERSION pkg-config jsoncpp cmake
protobuf260 log4cxx
+brew install openssl git boost@$BOOST_VERSION pkg-config jsoncpp cmake
protobuf260
brew link --force boost@$BOOST_VERSION
@@ -40,8 +40,8 @@ fi
if [ "$PYTHONVER" = "PYTHON3" ]
then
- brew install python boost-python3@BOOST_VERSION
- brew link --force boost-python3@$BOOST_VERSION
+ brew install python boost-python3
+ brew link --force boost-python3
fi
brew link --force protobuf260
@@ -52,4 +52,10 @@ cd incubator-pulsar/pulsar-client-cpp
cmake . -DBUILD_TESTS=OFF -DLINK_STATIC=ON
make _pulsar -j8
cd python
-python setup.py bdist_wheel
+
+if [ "$PYTHONVER" = "PYTHON2" ]
+then
+ python2 setup.py bdist_wheel
+else
+ python3 setup.py bdist_wheel
+fi
[incubator-pulsar] branch asf-site updated: Updated site at revision aebfbba
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/asf-site by this push: new 371973f Updated site at revision aebfbba 371973f is described below commit 371973f2f2d97154c06ff5818911056c13af11d4 Author: jenkins AuthorDate: Tue Sep 18 08:43:51 2018 + Updated site at revision aebfbba --- .../docs/latest/adaptors/PulsarSpark/index.html| 8 ++--- .../docs/latest/adaptors/PulsarStorm/index.html| 6 ++-- content/docs/latest/admin-api/overview/index.html | 10 +++--- content/docs/latest/clients/Cpp/index.html | 8 ++--- content/docs/latest/clients/Java/index.html| 14 content/docs/latest/clients/Python/index.html | 10 +++--- content/docs/latest/clients/WebSocket/index.html | 8 ++--- content/docs/latest/clients/go/index.html | 6 ++-- .../docs/latest/cookbooks/Encryption/index.html| 6 ++-- .../latest/cookbooks/PartitionedTopics/index.html | 14 .../latest/cookbooks/RetentionExpiry/index.html| 14 .../docs/latest/cookbooks/compaction/index.html| 6 ++-- .../cookbooks/message-deduplication/index.html | 10 +++--- .../docs/latest/cookbooks/message-queue/index.html | 14 .../latest/cookbooks/tiered-storage/index.html | 4 +-- .../docs/latest/deployment/Kubernetes/index.html | 4 +-- .../docs/latest/deployment/aws-cluster/index.html | 4 +-- content/docs/latest/deployment/cluster/index.html | 4 +-- content/docs/latest/deployment/instance/index.html | 4 +-- .../ConceptsAndArchitecture/index.html | 4 +-- .../latest/getting-started/LocalCluster/index.html | 4 +-- .../latest/getting-started/Pulsar-2.0/index.html | 2 +- .../docs/latest/getting-started/docker/index.html | 4 +-- .../docs/latest/project/BinaryProtocol/index.html | 4 +-- content/docs/latest/project/CompileCpp/index.html | 8 ++--- .../docs/latest/project/SimulationTools/index.html | 2 +- .../docs/latest/project/schema-storage/index.html | 4 +-- content/docs/latest/reference/CliTools/index.html | 18 +- .../docs/latest/security/authorization/index.html | 10 +++--- content/docs/latest/security/encryption/index.html | 6 ++-- content/img/docusaurus.svg | 2 +- content/img/pulsar.svg | 2 +- content/ja/adaptors/PulsarSpark/index.html | 8 ++--- content/ja/adaptors/PulsarStorm/index.html | 6 ++-- content/ja/admin/AdminInterface/index.html | 12 +++ content/ja/admin/Authz/index.html | 12 +++ content/ja/admin/ClustersBrokers/index.html| 6 ++-- content/ja/admin/PropertiesNamespaces/index.html | 6 ++-- content/ja/advanced/PartitionedTopics/index.html | 12 +++ content/ja/advanced/RetentionExpiry/index.html | 12 +++ content/ja/clients/Cpp/index.html | 6 ++-- content/ja/clients/Java/index.html | 8 ++--- content/ja/clients/Python/index.html | 8 ++--- content/ja/clients/WebSocket/index.html| 8 ++--- content/ja/deployment/InstanceSetup/index.html | 6 ++-- content/ja/deployment/Kubernetes/index.html| 4 +-- .../ConceptsAndArchitecture/index.html | 2 +- content/ja/getting-started/LocalCluster/index.html | 4 +-- content/ja/project/BinaryProtocol/index.html | 4 +-- content/ja/project/SimulationTools/index.html | 2 +- content/ja/reference/CliTools/index.html | 18 +- content/swagger/swagger.json | 40 +++--- 52 files changed, 204 insertions(+), 204 deletions(-) diff --git a/content/docs/latest/adaptors/PulsarSpark/index.html b/content/docs/latest/adaptors/PulsarSpark/index.html index a30d928..d130108 100644 --- a/content/docs/latest/adaptors/PulsarSpark/index.html +++ b/content/docs/latest/adaptors/PulsarSpark/index.html @@ -1079,9 +1079,9 @@ + Spark Streaming Pulsar receiver - Spark Streaming Pulsar receiver @@ -1325,9 +1325,9 @@ + Spark Streaming Pulsar receiver - Spark Streaming Pulsar receiver @@ -1537,8 +1537,6 @@ - - The Pulsar Java client @@ -1617,6 +1615,8 @@ + + Authentication and authorization in Pulsar diff --git a/content/docs/latest/adaptors/PulsarStorm/index.html b/content/docs/latest/adaptors/PulsarStorm/index.html index 3059cf5..67a4256 100644 --- a/content/docs/latest/adaptors/PulsarStorm
[incubator-pulsar] branch master updated: [tests] Fix the synchronization problem at BrokerClientIntegrationTest.testMaxConcurrentTopicLoading (#2595)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 259275b [tests] Fix the synchronization problem at
BrokerClientIntegrationTest.testMaxConcurrentTopicLoading (#2595)
259275b is described below
commit 259275ba8df5750e64fa15781dc2a968959ce71f
Author: Sijie Guo
AuthorDate: Mon Sep 17 13:36:50 2018 -0700
[tests] Fix the synchronization problem at
BrokerClientIntegrationTest.testMaxConcurrentTopicLoading (#2595)
*Motivation*
Following exception is observed in one of the CI jobs.
```
java.lang.NullPointerException
at
org.apache.pulsar.common.util.FutureUtil.waitForAll(FutureUtil.java:44)
at
org.apache.pulsar.client.impl.BrokerClientIntegrationTest.testMaxConcurrentTopicLoading(BrokerClientIntegrationTest.java:601)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
at
org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:54)
at
org.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:44)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
The problem seems to be coming from improper synchronization in the tests.
*Changes*
Fix the synchronization problem in
BrokerClientIntegrationTest.testMaxConcurrentTopicLoading
---
.../pulsar/client/impl/BrokerClientIntegrationTest.java | 12
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/BrokerClientIntegrationTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/BrokerClientIntegrationTest.java
index bf65fdd..0e47dfe 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/BrokerClientIntegrationTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/BrokerClientIntegrationTest.java
@@ -583,7 +583,7 @@ public class BrokerClientIntegrationTest extends
ProducerConsumerBase {
ClientCnx cnx = producer.cnx();
assertTrue(cnx.channel().isActive());
ExecutorService executor =
Executors.newFixedThreadPool(concurrentLookupRequests);
-List>> futures =
Lists.newArrayList();
+final List>> futures =
Lists.newArrayList();
final int totalProducers = 10;
CountDownLatch latch = new CountDownLatch(totalProducers);
for (int i = 0; i < totalProducers; i++) {
@@ -591,14 +591,18 @@ public class BrokerClientIntegrationTest extends
ProducerConsumerBase {
final String randomTopicName1 = topicName +
randomUUID().toString();
final String randomTopicName2 = topicName +
randomUUID().toString();
// pass producer-name to avoid exception: producer is
already connected to topic
-
futures.add(pulsarClient2.newProducer().topic(randomTopicName1).createAsync());
-
futures.add(pulsarClient.newProducer().topic(randomTopicName2).createAsync());
+synchronized (futures) {
+
futures.add(pulsarClient2.newProducer().topic(randomTopicName1).createAsync());
+
futures.add(pulsarClient.newProducer().topic(randomTopicName2).createAsync());
+}
latch.countDown();
});
}
latch.await();
-FutureUtil.waitForAll(futures).get();
+synchronized (futures) {
+FutureUtil.waitForAll(futures).get();
+}
pulsarClient.close();
pulsarClient2.close();
} finally {
[incubator-pulsar] branch asf-site updated: Updated site at revision a123380
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 4437db0 Updated site at revision a123380
4437db0 is described below
commit 4437db09cf6239e09015837aae352fbdadd2498c
Author: jenkins
AuthorDate: Mon Sep 17 08:45:06 2018 +
Updated site at revision a123380
---
content/img/docusaurus.svg | 2 +-
content/img/pulsar.svg | 2 +-
content/swagger/swagger.json | 44 ++--
3 files changed, 24 insertions(+), 24 deletions(-)
diff --git a/content/img/docusaurus.svg b/content/img/docusaurus.svg
index 1d8b516..f80b221 100644
--- a/content/img/docusaurus.svg
+++ b/content/img/docusaurus.svg
@@ -1 +1 @@
-http://www.w3.org/2000/svg;
display="block">http://www.w3.org/2000/svg; xmlns:xlink="http://www.w3.org/1999/xlink;
style="display: block;">http://www.w3.org/2000/svg; viewBox="0 0 223.44 43.08">http://www.w3.org/2000/svg; viewBox="0 0 223.44
43.08">.cls-1{fill:#188fff;}.cls-2{fill:#464e56;}Asset
2
[incubator-pulsar] branch asf-site updated: Updated site at revision 858648e
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 4b33c63 Updated site at revision 858648e
4b33c63 is described below
commit 4b33c6302c5f0b097f0eb062d0840775cb813d5f
Author: jenkins
AuthorDate: Sat Sep 15 08:42:28 2018 +
Updated site at revision 858648e
---
content/swagger/swagger.json | 20 ++--
1 file changed, 10 insertions(+), 10 deletions(-)
diff --git a/content/swagger/swagger.json b/content/swagger/swagger.json
index debb49c..2e4e78a 100644
--- a/content/swagger/swagger.json
+++ b/content/swagger/swagger.json
@@ -6367,10 +6367,10 @@
"type" : "string"
}
},
-"clientVersion" : {
+"connectedSince" : {
"type" : "string"
},
-"connectedSince" : {
+"clientVersion" : {
"type" : "string"
},
"address" : {
@@ -6884,10 +6884,10 @@
"type" : "number",
"format" : "double"
},
-"clientVersion" : {
+"connectedSince" : {
"type" : "string"
},
-"connectedSince" : {
+"clientVersion" : {
"type" : "string"
},
"producerName" : {
@@ -7588,10 +7588,10 @@
"type" : "string"
}
},
-"clientVersion" : {
+"connectedSince" : {
"type" : "string"
},
-"connectedSince" : {
+"clientVersion" : {
"type" : "string"
},
"producerName" : {
@@ -7653,15 +7653,15 @@
"ResourceDescription" : {
"type" : "object",
"properties" : {
+"usagePct" : {
+ "type" : "integer",
+ "format" : "int32"
+},
"resourceUsage" : {
"type" : "object",
"additionalProperties" : {
"$ref" : "#/definitions/ResourceUsage"
}
-},
-"usagePct" : {
- "type" : "integer",
- "format" : "int32"
}
}
},
[incubator-pulsar] branch asf-site updated: Updated site at revision 95fe84c
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/asf-site by this push:
new a72ff25 Updated site at revision 95fe84c
a72ff25 is described below
commit a72ff25292b9782961d55e1d92ab6952a8cdfdde
Author: jenkins
AuthorDate: Fri Sep 14 08:43:21 2018 +
Updated site at revision 95fe84c
---
content/api/client/allclasses-frame.html | 1 +
content/api/client/allclasses-noframe.html | 1 +
content/api/client/index-all.html | 10 ++
.../apache/pulsar/client/api/ConsumerBuilder.html | 77 +++
.../apache/pulsar/client/api/CryptoKeyReader.html | 4 +-
...ParameterSupport.html => DeadLetterPolicy.html} | 104 +++--
.../api/EncodedAuthenticationParameterSupport.html | 4 +-
.../apache/pulsar/client/api/package-frame.html| 1 +
.../apache/pulsar/client/api/package-summary.html | 8 +-
.../org/apache/pulsar/client/api/package-tree.html | 1 +
content/api/client/overview-tree.html | 1 +
.../docs/latest/adaptors/PulsarSpark/index.html| 2 +-
.../docs/latest/adaptors/PulsarStorm/index.html| 2 +-
content/docs/latest/admin-api/overview/index.html | 8 +-
content/docs/latest/clients/Java/index.html| 2 +-
.../latest/cookbooks/PartitionedTopics/index.html | 2 +-
.../latest/cookbooks/RetentionExpiry/index.html| 2 +-
.../docs/latest/cookbooks/compaction/index.html| 2 +-
.../cookbooks/message-deduplication/index.html | 2 +-
.../docs/latest/cookbooks/message-queue/index.html | 2 +-
.../latest/cookbooks/tiered-storage/index.html | 2 +-
.../docs/latest/deployment/aws-cluster/index.html | 2 +-
content/docs/latest/deployment/cluster/index.html | 2 +-
content/docs/latest/deployment/instance/index.html | 2 +-
.../docs/latest/project/BinaryProtocol/index.html | 10 ++
content/docs/latest/reference/CliTools/index.html | 4 +-
.../docs/latest/security/authorization/index.html | 4 +-
content/ja/project/BinaryProtocol/index.html | 10 ++
content/swagger/swagger.json | 68 +++---
29 files changed, 212 insertions(+), 128 deletions(-)
diff --git a/content/api/client/allclasses-frame.html
b/content/api/client/allclasses-frame.html
index dee5dcd..3244593 100644
--- a/content/api/client/allclasses-frame.html
+++ b/content/api/client/allclasses-frame.html
@@ -27,6 +27,7 @@
ConsumerStats
CryptoKeyReader
DataURLStreamHandler
+DeadLetterPolicy
EncodedAuthenticationParameterSupport
EncryptionKeyInfo
HashingScheme
diff --git a/content/api/client/allclasses-noframe.html
b/content/api/client/allclasses-noframe.html
index 4154d55..cc8d392 100644
--- a/content/api/client/allclasses-noframe.html
+++ b/content/api/client/allclasses-noframe.html
@@ -27,6 +27,7 @@
ConsumerStats
CryptoKeyReader
DataURLStreamHandler
+DeadLetterPolicy
EncodedAuthenticationParameterSupport
EncryptionKeyInfo
HashingScheme
diff --git a/content/api/client/index-all.html
b/content/api/client/index-all.html
index 2f68dc9..22b715e 100644
--- a/content/api/client/index-all.html
+++ b/content/api/client/index-all.html
@@ -490,6 +490,16 @@
DataURLStreamHandler()
- Constructor for class org.apache.pulsar.client.api.url.DataURLStreamHandler
+deadLetterPolicy(DeadLetterPolicy)
- Method in interface org.apache.pulsar.client.api.ConsumerBuilder
+
+Set dead letter policy for consumer
+
+ By default some message will redelivery so many times possible, even to the
extent that it can be never stop.
+
+DeadLetterPolicy - Class in org.apache.pulsar.client.api
+
+DeadLetterPolicy()
- Constructor for class org.apache.pulsar.client.api.DeadLetterPolicy
+
disableReplication()
- Method in interface org.apache.pulsar.client.api.MessageBuilder
Deprecated.
diff --git
a/content/api/client/org/apache/pulsar/client/api/ConsumerBuilder.html
b/content/api/client/org/apache/pulsar/client/api/ConsumerBuilder.html
index c3b8748..0ca529d 100644
--- a/content/api/client/org/apache/pulsar/client/api/ConsumerBuilder.html
+++ b/content/api/client/org/apache/pulsar/client/api/ConsumerBuilder.html
@@ -17,7 +17,7 @@
catch(err) {
}
//-->
-var methods =
{"i0":6,"i1":6,"i2":6,"i3":6,"i4":6,"i5":6,"i6":6,"i7":6,"i8":6,"i9":6,"i10":6,"i11":6,"i12":6,"i13":6,"i14":6,"i15":6,"i16":6,"i17":6,"i18":6,"i19":6,"i20":6,"i21":6,"i22":6,"i23":6,"i24":6,"i25":6};
+var methods =
{"i0":6,"i1":6,"i2":6,"i3":6,"i4":6,"i5":6,"i6":6,"i7":6,&q
[incubator-pulsar] branch master updated: ManagedLedger only closes ledger on error if current ledger (#240) (#2573)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 7c62ecf ManagedLedger only closes ledger on error if current ledger
(#240) (#2573)
7c62ecf is described below
commit 7c62ecf4d75ad396066add41246fcb4f92a7c301
Author: Ivan Kelly
AuthorDate: Fri Sep 14 00:59:55 2018 +0200
ManagedLedger only closes ledger on error if current ledger (#240) (#2573)
If we have a managed ledger, ml and we write 2 entries to it, if both
entries fail, both will end up calling ManagedLedgerImpl#ledgerClosed
with the ledger the write failed on as a parameter.
However, depending on timing, the second call to ledgerClosed could
end up adding a new ledger to the ledger list, even though the current
ledger is _not_ failing (as the failing ledger was replaced by the
first call).
This was the cause of a flake in
ManagedLedgerErrorsTest#recoverLongTimeAfterMultipleWriteErrors as
reported in (#240). However, it's not possible to get a deterministic
test for this as the timings need to be very precise. The failing
addComplete needs to run before first error handling completes, but
the runnable with ledgerClosed for the second failure needs to run
after the first error handling completes, but before the write resends
from the first error handling complete.
---
.../java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java
index 6e4a609..f332e17 100644
---
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java
+++
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java
@@ -1277,7 +1277,8 @@ public class ManagedLedgerImpl implements ManagedLedger,
CreateCallback {
synchronized void ledgerClosed(final LedgerHandle lh) {
final State state = STATE_UPDATER.get(this);
-if (state == State.ClosingLedger || state == State.LedgerOpened) {
+LedgerHandle currentLedger = this.currentLedger;
+if (currentLedger == lh && (state == State.ClosingLedger || state ==
State.LedgerOpened)) {
STATE_UPDATER.set(this, State.ClosedLedger);
} else if (state == State.Closed) {
// The managed ledger was closed during the write operation
[incubator-pulsar] branch master updated: [io] jdbc connector uses Schema.AUTO (#2538)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 7364563 [io] jdbc connector uses Schema.AUTO (#2538)
7364563 is described below
commit 73645631849ffce2f144f23d861763fe52a25af9
Author: Sijie Guo
AuthorDate: Wed Sep 12 21:32:39 2018 -0700
[io] jdbc connector uses Schema.AUTO (#2538)
* [schema] enable Schema.AUTO if functions or connectors are using
GenericRecord
* auto schema in jdbc sink
* [io] jdbc connector uses Schema.AUTO
---
pulsar-functions/java-examples/pom.xml | 1 -
.../apache/pulsar/io/jdbc/JdbcAbstractSink.java| 3 --
...AvroSchemaSink.java => JdbcAutoSchemaSink.java} | 39 +-
.../org/apache/pulsar/io/jdbc/JdbcSinkConfig.java | 3 --
.../resources/META-INF/services/pulsar-io.yaml | 2 +-
.../org/apache/pulsar/io/jdbc/JdbcSinkTest.java| 17 ++
.../integration/functions/PulsarFunctionsTest.java | 21 +---
.../tests/integration/io/JdbcSinkTester.java | 10 +++---
.../pulsar/tests/integration/io/SinkTester.java| 5 +++
9 files changed, 47 insertions(+), 54 deletions(-)
diff --git a/pulsar-functions/java-examples/pom.xml
b/pulsar-functions/java-examples/pom.xml
index b077dea..747acff 100644
--- a/pulsar-functions/java-examples/pom.xml
+++ b/pulsar-functions/java-examples/pom.xml
@@ -41,7 +41,6 @@
pulsar-functions-api
${project.version}
-
diff --git
a/pulsar-io/jdbc/src/main/java/org/apache/pulsar/io/jdbc/JdbcAbstractSink.java
b/pulsar-io/jdbc/src/main/java/org/apache/pulsar/io/jdbc/JdbcAbstractSink.java
index 425fb57..91d043d 100644
---
a/pulsar-io/jdbc/src/main/java/org/apache/pulsar/io/jdbc/JdbcAbstractSink.java
+++
b/pulsar-io/jdbc/src/main/java/org/apache/pulsar/io/jdbc/JdbcAbstractSink.java
@@ -54,8 +54,6 @@ public abstract class JdbcAbstractSink implements Sink {
private JdbcUtils.TableId tableId;
private PreparedStatement insertStatement;
-// TODO: turn to getSchema from
SinkContext.getTopicSchema.getSchema(inputTopic)
-protected String schema;
protected JdbcUtils.TableDefinition tableDefinition;
// for flush
@@ -89,7 +87,6 @@ public abstract class JdbcAbstractSink implements Sink {
connection.setAutoCommit(false);
log.info("Opened jdbc connection: {}, autoCommit: {}", jdbcUrl,
connection.getAutoCommit());
-schema = jdbcSinkConfig.getSchema();
tableName = jdbcSinkConfig.getTableName();
tableId = JdbcUtils.getTableId(connection, tableName);
tableDefinition = JdbcUtils.getTableDefinition(connection, tableId);
diff --git
a/pulsar-io/jdbc/src/main/java/org/apache/pulsar/io/jdbc/JdbcAvroSchemaSink.java
b/pulsar-io/jdbc/src/main/java/org/apache/pulsar/io/jdbc/JdbcAutoSchemaSink.java
similarity index 61%
rename from
pulsar-io/jdbc/src/main/java/org/apache/pulsar/io/jdbc/JdbcAvroSchemaSink.java
rename to
pulsar-io/jdbc/src/main/java/org/apache/pulsar/io/jdbc/JdbcAutoSchemaSink.java
index ec28220..c18dc81 100644
---
a/pulsar-io/jdbc/src/main/java/org/apache/pulsar/io/jdbc/JdbcAvroSchemaSink.java
+++
b/pulsar-io/jdbc/src/main/java/org/apache/pulsar/io/jdbc/JdbcAutoSchemaSink.java
@@ -20,50 +20,27 @@
package org.apache.pulsar.io.jdbc;
import java.sql.PreparedStatement;
-import java.util.Map;
import lombok.extern.slf4j.Slf4j;
-import org.apache.avro.Schema;
-import org.apache.avro.generic.GenericDatumReader;
-import org.apache.avro.generic.GenericRecord;
-import org.apache.avro.io.DatumReader;
-import org.apache.avro.io.DecoderFactory;
-import org.apache.avro.util.Utf8;
+import org.apache.pulsar.client.api.schema.GenericRecord;
import org.apache.pulsar.functions.api.Record;
-import org.apache.pulsar.io.core.SinkContext;
import org.apache.pulsar.io.jdbc.JdbcUtils.ColumnId;
/**
- * A Simple Jdbc sink, which assume input Record as AvroSchema format
+ * A Simple Jdbc sink, which interprets input Record in generic record.
*/
@Slf4j
-public class JdbcAvroSchemaSink extends JdbcAbstractSink {
-
-private Schema avroSchema = null;
-private DatumReader reader = null;
-
+public class JdbcAutoSchemaSink extends JdbcAbstractSink {
@Override
-public void open(Map config, SinkContext sinkContext)
throws Exception {
-super.open(config, sinkContext);
-// get reader, and read value out as GenericRecord
-if (avroSchema == null || reader == null) {
-avroSchema = Schema.parse(schema);
-reader = new GenericDatumReader<>(avroSchema);
-}
-log.info("open JdbcAvroSchemaSink with schema: {}, and
tableDefinition: {}", schema, tableDefinition.toString());
-}
-
-
public void bindValue(PreparedStatement statement,
- Rec
[incubator-pulsar] branch master updated: [build] Fix maven deploy failure (#2565)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/master by this push: new a45ea9c [build] Fix maven deploy failure (#2565) a45ea9c is described below commit a45ea9c74f45d5b3be160d62cd96ab5754e278de Author: Sijie Guo AuthorDate: Wed Sep 12 07:20:30 2018 -0700 [build] Fix maven deploy failure (#2565) *Motivation* Pulsar presto distribution doesn't inherit from apache profile. Hence when we run `mvn deploy`, it will fail to deploy the apache repository. ``` 2018-09-11T23:52:04.260 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy (default-deploy) on project pulsar-presto-distribution: Deployment failed: repository element was not specified in the POM inside distributionManagement element or in -DaltDeploymentRepository=id::layout::url parameter -> [Help 1] ``` *Changes* Make presto distribution pom file inherit from apache profile. --- pulsar-sql/presto-distribution/pom.xml | 6 ++ 1 file changed, 6 insertions(+) diff --git a/pulsar-sql/presto-distribution/pom.xml b/pulsar-sql/presto-distribution/pom.xml index 4446d83..527ba8a 100644 --- a/pulsar-sql/presto-distribution/pom.xml +++ b/pulsar-sql/presto-distribution/pom.xml @@ -22,6 +22,12 @@ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> 4.0.0 + +org.apache +apache +18 + + org.apache.pulsar pulsar-presto-distribution Pulsar SQL :: Pulsar Presto Distribution
[incubator-pulsar] branch merlimat-patch-1 deleted (was 1133239)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch merlimat-patch-1 in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git. was 1133239 Fixed website build with missing comma in replace.js The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[incubator-pulsar] branch master updated: Fixed website build with missing comma in replace.js (#2562)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new bf374fc Fixed website build with missing comma in replace.js (#2562)
bf374fc is described below
commit bf374fcb6c7501997dc99ace1331153e766ca795
Author: Matteo Merli
AuthorDate: Wed Sep 12 00:44:20 2018 -0400
Fixed website build with missing comma in replace.js (#2562)
---
site2/website/scripts/replace.js | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/site2/website/scripts/replace.js b/site2/website/scripts/replace.js
index d0cc0fc..13c9569 100644
--- a/site2/website/scripts/replace.js
+++ b/site2/website/scripts/replace.js
@@ -87,7 +87,7 @@ const options = {
`${latestVersion}`,
binaryReleaseUrl(`${latestVersion}`),
connectorReleaseUrl(`${latestVersion}`),
-prestoPulsarReleaseUrl(`${latestVersion}`)
+prestoPulsarReleaseUrl(`${latestVersion}`),
downloadPageUrl(),
rpmReleaseUrl(`${latestVersion}`, ""),
rpmReleaseUrl(`${latestVersion}`, "-debuginfo"),
@@ -129,4 +129,4 @@ for (v of versions) {
dry: true
};
doReplace(opts);
-}
\ No newline at end of file
+}
[incubator-pulsar] branch jerrypeng-patch-1 deleted (was 00b89ab)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch jerrypeng-patch-1 in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git. was 00b89ab Correcting doc id to be consistent with sidebar The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[incubator-pulsar] branch master updated: Correcting doc id to be consistent with sidebar (#2558)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/master by this push: new 29a8af5 Correcting doc id to be consistent with sidebar (#2558) 29a8af5 is described below commit 29a8af559ed0663458ca2f9f1d8c950675c723f9 Author: Boyang Jerry Peng AuthorDate: Tue Sep 11 13:43:40 2018 -0700 Correcting doc id to be consistent with sidebar (#2558) --- site2/docs/sql-deployment-configurations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site2/docs/sql-deployment-configurations.md b/site2/docs/sql-deployment-configurations.md index 45d44bb..2b22cd9 100644 --- a/site2/docs/sql-deployment-configurations.md +++ b/site2/docs/sql-deployment-configurations.md @@ -1,5 +1,5 @@ --- -id: sql-deployment-configuration +id: sql-deployment-configurations title: Pulsar SQl Deployment and Configuration sidebar_label: Deployment and Configuration ---
[incubator-pulsar] branch master updated: Fixed download links and resource page (#2557)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new e098117 Fixed download links and resource page (#2557)
e098117 is described below
commit e098117cbc6396be81c3cc6061b10c2673773912
Author: Matteo Merli
AuthorDate: Tue Sep 11 11:51:32 2018 -0400
Fixed download links and resource page (#2557)
---
site2/website/data/resources.js| 6 +++---
site2/website/pages/en/download.js | 22 +++---
2 files changed, 14 insertions(+), 14 deletions(-)
diff --git a/site2/website/data/resources.js b/site2/website/data/resources.js
index d0f04ac..4eb6a29 100644
--- a/site2/website/data/resources.js
+++ b/site2/website/data/resources.js
@@ -29,7 +29,7 @@ module.exports = {
{
forum: 'Strata San Jose',
forum_link: 'https://conferences.oreilly.com/strata/strata-ca',
- presenter: 'Matteo Merli (Pulsar co-creator)',
+ presenter: 'Matteo Merli',
date: 'March 2018',
title: 'Effectively-once semantics in Apache Pulsar',
link:
'https://www.slideshare.net/merlimat/effectivelyonce-semantics-in-apache-pulsar'
@@ -37,7 +37,7 @@ module.exports = {
{
forum: '',
forum_link: '',
- presenter: 'Matteo Merli (Pulsar co-creator)',
+ presenter: 'Matteo Merli',
date: 'November 2016',
title: 'Pulsar: a distributed pub-sub platform',
link:
'https://www.slideshare.net/merlimat/pulsar-distributed-pubsub-platform'
@@ -45,7 +45,7 @@ module.exports = {
{
forum: 'Bay Area Hadoop Meetup',
forum_link: 'https://www.meetup.com/hadoop',
- presenter: 'Matteo Merli (Pulsar co-creator)',
+ presenter: 'Matteo Merli',
date: 'October 2016',
title: 'Pulsar: a highly scalable, low-latency pub-sub messaging system',
link:
'https://www.slideshare.net/ydn/october-2016-hug-pulsar-a-highly-scalable-low-latency-pubsub-messaging-system'
diff --git a/site2/website/pages/en/download.js
b/site2/website/pages/en/download.js
index f952814..cb87cbd 100644
--- a/site2/website/pages/en/download.js
+++ b/site2/website/pages/en/download.js
@@ -18,6 +18,10 @@ function getLatestArchiveMirrorUrl(version, type) {
return
`https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download=incubator/pulsar/pulsar-${version}/apache-pulsar-${version}-${type}.tar.gz`
}
+function distUrl(version, type) {
+return
`https://www.apache.org/dist/incubator/pulsar/pulsar-${version}/apache-pulsar-${version}-${type}.tar.gz`
+}
+
function archiveUrl(version, type) {
return
`${archiveRootUrl}/pulsar-${version}/apache-pulsar-${version}-${type}.tar.gz`
}
@@ -29,8 +33,8 @@ class Download extends React.Component {
const latestVersion = `${latestRelease}-incubating`
const latestArchiveMirrorUrl = getLatestArchiveMirrorUrl(latestVersion,
'bin');
const latestSrcArchiveMirrorUrl = getLatestArchiveMirrorUrl(latestVersion,
'src');
-const latestArchiveUrl = archiveUrl(latestVersion, 'bin');
-const latestSrcArchiveUrl = archiveUrl(latestVersion, 'src')
+const latestArchiveUrl = distUrl(latestVersion, 'bin');
+const latestSrcArchiveUrl = distUrl(latestVersion, 'src')
const releaseInfo = releases.map(r => {
const version = `${r}-incubating`;
@@ -65,8 +69,7 @@ class Download extends React.Component {
pulsar-{latestVersion}-bin.tar.gz
-asc,
-sha1,
+asc,
sha512
@@ -76,8 +79,7 @@ class Download extends React.Component {
pulsar-{latestVersion}-src.tar.gz
-asc,
-sha1,
+asc,
sha512
@@ -89,7 +91,7 @@ class Download extends React.Component {
You must [verify](https://www.apache.org/info/verification.html)
the integrity of the downloaded files.
We provide OpenPGP signatures for every release file. This
signature should be matched against the
[KEYS](https://www.apache.org/dist/incubator/pulsar/KEYS) file
which contains the OpenPGP keys of
- Pulsar's Release Managers. We also provide `MD5` and `SHA-512`
checksums for every release file.
+ Pulsar's Release Managers. We also provide `SHA-512` checksums
for every release file.
After you download the file, you should calculate a checksum for
your download, and make sure it is
the same as ours.
@@ -108,7 +110,7 @@ class Download extends React.Component {
Once you've downloaded a Pulsar release, instructions on
getting up and runn
[incubator-pulsar] branch master updated: Fix status code of REST API to get list of subscriptions of partitioned topic (#2550)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 24605d3 Fix status code of REST API to get list of subscriptions of
partitioned topic (#2550)
24605d3 is described below
commit 24605d328357e89ae107b549a6f0da2ce1683857
Author: massakam
AuthorDate: Tue Sep 11 12:43:21 2018 +0900
Fix status code of REST API to get list of subscriptions of partitioned
topic (#2550)
---
.../apache/pulsar/broker/admin/impl/PersistentTopicsBase.java | 6 ++
.../test/java/org/apache/pulsar/broker/admin/AdminApiTest.java | 10 ++
2 files changed, 16 insertions(+)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
index da51453..9f5c2df 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
@@ -529,6 +529,12 @@ public class PersistentTopicsBase extends AdminResource {
// subscriptions
subscriptions.addAll(pulsar().getAdminClient().topics()
.getSubscriptions(topicName.getPartition(0).toString()));
+} catch (PulsarAdminException e) {
+if (e.getStatusCode() == Status.NOT_FOUND.getStatusCode()) {
+throw new RestException(Status.NOT_FOUND, "Internal topics
have not been generated yet");
+} else {
+throw new RestException(e);
+}
} catch (Exception e) {
throw new RestException(e);
}
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
index c7c090f..187539f 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
@@ -798,6 +798,16 @@ public class AdminApiTest extends
MockedPulsarServiceBaseTest {
fail(e.getMessage());
}
+try {
+admin.topics().getSubscriptions(partitionedTopicName);
+fail("should have failed");
+} catch (PulsarAdminException e) {
+// ok
+assertEquals(e.getStatusCode(), Status.NOT_FOUND.getStatusCode());
+} catch (Exception e) {
+fail(e.getMessage());
+}
+
// create consumer and subscription
URL pulsarUrl = new URL("http://127.0.0.1; + ":" +
BROKER_WEBSERVICE_PORT);
PulsarClient client =
PulsarClient.builder().serviceUrl(pulsarUrl.toString()).statsInterval(0,
TimeUnit.SECONDS)
[incubator-pulsar] branch master updated: [schema] enable Schema.AUTO if functions or connectors are using GenericRecord (#2537)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 9ec8a5e [schema] enable Schema.AUTO if functions or connectors are
using GenericRecord (#2537)
9ec8a5e is described below
commit 9ec8a5ee4d4a80c50986cde95093b1eb4af31ef0
Author: Sijie Guo
AuthorDate: Sat Sep 8 12:51:12 2018 -0700
[schema] enable Schema.AUTO if functions or connectors are using
GenericRecord (#2537)
* [schema] enable Schema.AUTO if functions or connectors are using
GenericRecord
* add schema-api to functions-api
---
pulsar-functions/api-java/pom.xml | 6 ++
.../pulsar/functions/source/PulsarSource.java | 2 +-
.../pulsar/functions/source/TopicSchema.java | 44 +
pulsar-functions/java-examples/pom.xml | 1 +
.../functions/api/examples/AutoSchemaFunction.java | 33 ++
tests/integration/pom.xml | 12
.../integration/functions/PulsarFunctionsTest.java | 77 +-
7 files changed, 143 insertions(+), 32 deletions(-)
diff --git a/pulsar-functions/api-java/pom.xml
b/pulsar-functions/api-java/pom.xml
index 9a84206..0f21bae 100644
--- a/pulsar-functions/api-java/pom.xml
+++ b/pulsar-functions/api-java/pom.xml
@@ -38,6 +38,12 @@
+ ${project.groupId}
+ pulsar-client-schema
+ ${project.version}
+
+
+
net.jodah
typetools
test
diff --git
a/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/PulsarSource.java
b/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/PulsarSource.java
index e1059f3..6eed8e0 100644
---
a/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/PulsarSource.java
+++
b/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/PulsarSource.java
@@ -78,7 +78,7 @@ public class PulsarSource extends PushSource implements
MessageListener
inputConsumers = configs.entrySet().stream().map(e -> {
String topic = e.getKey();
ConsumerConfig conf = e.getValue();
-log.info("Creating consumers for topic : {}", topic);
+log.info("Creating consumers for topic : {}, schema : {}", topic,
conf.getSchema());
ConsumerBuilder cb = pulsarClient.newConsumer(conf.getSchema())
// consume message even if can't decrypt and deliver it
along with encryption-ctx
.cryptoFailureAction(ConsumerCryptoFailureAction.CONSUME)
diff --git
a/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/TopicSchema.java
b/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/TopicSchema.java
index 2ac5b65..76375dc 100644
---
a/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/TopicSchema.java
+++
b/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/TopicSchema.java
@@ -26,6 +26,7 @@ import java.util.Optional;
import org.apache.commons.lang3.StringUtils;
import org.apache.pulsar.client.api.PulsarClient;
import org.apache.pulsar.client.api.Schema;
+import org.apache.pulsar.client.api.schema.GenericRecord;
import org.apache.pulsar.client.impl.PulsarClientImpl;
import org.apache.pulsar.client.impl.schema.AvroSchema;
import org.apache.pulsar.client.impl.schema.JSONSchema;
@@ -68,24 +69,31 @@ public class TopicSchema {
}
public Schema getSchema(String topic, Class clazz, SchemaType
schemaType) {
-return cachedSchemas.computeIfAbsent(topic, t -> extractSchema(clazz,
schemaType));
+return cachedSchemas.computeIfAbsent(topic, t ->
newSchemaInstance(clazz, schemaType));
}
/**
* If the topic is already created, we should be able to fetch the schema
type (avro, json, ...)
*/
private SchemaType getSchemaTypeOrDefault(String topic, Class clazz) {
-Optional schema = ((PulsarClientImpl)
client).getSchema(topic).join();
-if (schema.isPresent()) {
-return schema.get().getType();
+if (GenericRecord.class.isAssignableFrom(clazz)) {
+return SchemaType.AUTO;
} else {
-return getDefaultSchemaType(clazz);
+Optional schema = ((PulsarClientImpl)
client).getSchema(topic).join();
+if (schema.isPresent()) {
+return schema.get().getType();
+} else {
+return getDefaultSchemaType(clazz);
+}
}
}
private static SchemaType getDefaultSchemaType(Class clazz) {
if (byte[].class.equals(clazz)) {
return SchemaType.NONE;
+} else if (GenericRecord.class.isAssignableFrom(clazz)) {
+// the function is taking generic
[incubator-pulsar] branch master updated: Enable C++ AuthFactory to parse Athenz params string (#2540)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 6451efd Enable C++ AuthFactory to parse Athenz params string (#2540)
6451efd is described below
commit 6451efd6515026288ae8aa1b7118e97904eecf76
Author: massakam
AuthorDate: Sun Sep 9 04:50:35 2018 +0900
Enable C++ AuthFactory to parse Athenz params string (#2540)
---
pulsar-client-cpp/include/pulsar/Authentication.h | 2 +
pulsar-client-cpp/lib/Authentication.cc | 43 +++--
pulsar-client-cpp/lib/auth/AuthTls.cc | 5 ++
pulsar-client-cpp/tests/AuthPluginTest.cc | 58 +++
4 files changed, 94 insertions(+), 14 deletions(-)
diff --git a/pulsar-client-cpp/include/pulsar/Authentication.h
b/pulsar-client-cpp/include/pulsar/Authentication.h
index bde4134..2ea1238 100644
--- a/pulsar-client-cpp/include/pulsar/Authentication.h
+++ b/pulsar-client-cpp/include/pulsar/Authentication.h
@@ -61,6 +61,7 @@ class Authentication {
authDataContent = authData_;
return ResultOk;
}
+static ParamMap parseDefaultFormatAuthParams(const std::string&
authParamsString);
protected:
Authentication();
@@ -104,6 +105,7 @@ class AuthTls : public Authentication {
AuthTls(AuthenticationDataPtr&);
~AuthTls();
static AuthenticationPtr create(ParamMap& params);
+static AuthenticationPtr create(const std::string& authParamsString);
static AuthenticationPtr create(const std::string& certificatePath, const
std::string& privateKeyPath);
const std::string getAuthMethodName() const;
Result getAuthData(AuthenticationDataPtr& authDataTls) const;
diff --git a/pulsar-client-cpp/lib/Authentication.cc
b/pulsar-client-cpp/lib/Authentication.cc
index c1025e1..b3ebf1c 100644
--- a/pulsar-client-cpp/lib/Authentication.cc
+++ b/pulsar-client-cpp/lib/Authentication.cc
@@ -60,6 +60,22 @@ Authentication::Authentication() {}
Authentication::~Authentication() {}
+ParamMap Authentication::parseDefaultFormatAuthParams(const std::string&
authParamsString) {
+ParamMap paramMap;
+if (!authParamsString.empty()) {
+std::vector params;
+boost::algorithm::split(params, authParamsString,
boost::is_any_of(","));
+for (int i = 0; i < params.size(); i++) {
+std::vector kv;
+boost::algorithm::split(kv, params[i], boost::is_any_of(":"));
+if (kv.size() == 2) {
+paramMap[kv[0]] = kv[1];
+}
+}
+}
+return paramMap;
+}
+
class AuthDisabledData : public AuthenticationDataProvider {
public:
AuthDisabledData(ParamMap& params) {}
@@ -111,6 +127,17 @@ AuthenticationPtr tryCreateBuiltinAuth(const std::string&
pluginName, ParamMap&
}
}
+AuthenticationPtr tryCreateBuiltinAuth(const std::string& pluginName, const
std::string& authParamsString) {
+if (boost::iequals(pluginName, TLS_PLUGIN_NAME) ||
boost::iequals(pluginName, TLS_JAVA_PLUGIN_NAME)) {
+return AuthTls::create(authParamsString);
+} else if (boost::iequals(pluginName, ATHENZ_PLUGIN_NAME) ||
+ boost::iequals(pluginName, ATHENZ_JAVA_PLUGIN_NAME)) {
+return AuthAthenz::create(authParamsString);
+} else {
+return AuthenticationPtr();
+}
+}
+
AuthenticationPtr AuthFactory::create(const std::string&
pluginNameOrDynamicLibPath,
const std::string& authParamsString) {
{
@@ -121,20 +148,7 @@ AuthenticationPtr AuthFactory::create(const std::string&
pluginNameOrDynamicLibP
}
}
-ParamMap paramMap;
-if (!authParamsString.empty()) {
-std::vector params;
-boost::algorithm::split(params, authParamsString,
boost::is_any_of(","));
-for (int i = 0; i < params.size(); i++) {
-std::vector kv;
-boost::algorithm::split(kv, params[i], boost::is_any_of(":"));
-if (kv.size() == 2) {
-paramMap[kv[0]] = kv[1];
-}
-}
-}
-
-AuthenticationPtr authPtr =
tryCreateBuiltinAuth(pluginNameOrDynamicLibPath, paramMap);
+AuthenticationPtr authPtr =
tryCreateBuiltinAuth(pluginNameOrDynamicLibPath, authParamsString);
if (authPtr) {
return authPtr;
}
@@ -151,6 +165,7 @@ AuthenticationPtr AuthFactory::create(const std::string&
pluginNameOrDynamicLibP
if (createAuthentication != NULL) {
auth = createAuthentication(authParamsString);
} else {
+ParamMap paramMap =
Authentication::parseDefaultFormatAuthParams(authParamsString);
return AuthFactory::create(pluginNameOrDynamicLibPath, paramMap);
}
}
diff --
[incubator-pulsar] branch master updated: Fix status code of REST API to get partitioned topic stats (#2542)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new cd185d1 Fix status code of REST API to get partitioned topic stats
(#2542)
cd185d1 is described below
commit cd185d1d9bb3319cae3543c561454d40ac54d899
Author: massakam
AuthorDate: Sun Sep 9 04:50:52 2018 +0900
Fix status code of REST API to get partitioned topic stats (#2542)
---
.../apache/pulsar/broker/admin/impl/PersistentTopicsBase.java | 6 ++
.../java/org/apache/pulsar/broker/admin/AdminApiTest.java | 11 +++
2 files changed, 17 insertions(+)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
index f85c617..da51453 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
@@ -604,6 +604,12 @@ public class PersistentTopicsBase extends AdminResource {
stats.add(partitionStats);
stats.partitions.put(topicName.getPartition(i).toString(),
partitionStats);
}
+} catch (PulsarAdminException e) {
+if (e.getStatusCode() == Status.NOT_FOUND.getStatusCode()) {
+throw new RestException(Status.NOT_FOUND, "Internal topics
have not been generated yet");
+} else {
+throw new RestException(e);
+}
} catch (Exception e) {
throw new RestException(e);
}
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
index ae72737..c7c090f 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
@@ -53,6 +53,7 @@ import java.util.concurrent.TimeUnit;
import javax.ws.rs.client.InvocationCallback;
import javax.ws.rs.client.WebTarget;
+import javax.ws.rs.core.Response.Status;
import lombok.extern.slf4j.Slf4j;
@@ -787,6 +788,16 @@ public class AdminApiTest extends
MockedPulsarServiceBaseTest {
assertEquals(admin.topics().getPartitionedTopicMetadata("persistent://prop-xyz/ns1/ds2").partitions,
0);
+try {
+admin.topics().getPartitionedStats(partitionedTopicName, false);
+fail("should have failed");
+} catch (PulsarAdminException e) {
+// ok
+assertEquals(e.getStatusCode(), Status.NOT_FOUND.getStatusCode());
+} catch (Exception e) {
+fail(e.getMessage());
+}
+
// create consumer and subscription
URL pulsarUrl = new URL("http://127.0.0.1; + ":" +
BROKER_WEBSERVICE_PORT);
PulsarClient client =
PulsarClient.builder().serviceUrl(pulsarUrl.toString()).statsInterval(0,
TimeUnit.SECONDS)
[incubator-pulsar] branch branch-2.1 updated: Fixed PulsarZKDowngradeTest imports after cherry-pick
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new f4ef546 Fixed PulsarZKDowngradeTest imports after cherry-pick
f4ef546 is described below
commit f4ef546f19b09860cea5691328b54b632c6946ea
Author: Matteo Merli
AuthorDate: Thu Sep 6 11:30:28 2018 -0700
Fixed PulsarZKDowngradeTest imports after cherry-pick
---
tests/integration-tests-topologies/pom.xml| 5 +
.../org/apache/pulsar/tests/upgrade/PulsarZKDowngradeTest.java| 8
2 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/tests/integration-tests-topologies/pom.xml
b/tests/integration-tests-topologies/pom.xml
index 65962da..46070ea 100644
--- a/tests/integration-tests-topologies/pom.xml
+++ b/tests/integration-tests-topologies/pom.xml
@@ -51,6 +51,11 @@
integration-tests-utils
${project.version}
+
+ org.apache.pulsar
+ pulsar-client-original
+ ${project.version}
+
diff --git
a/tests/integration-tests-topologies/src/main/java/org/apache/pulsar/tests/upgrade/PulsarZKDowngradeTest.java
b/tests/integration-tests-topologies/src/main/java/org/apache/pulsar/tests/upgrade/PulsarZKDowngradeTest.java
index 6005aa7..ec213fe 100644
---
a/tests/integration-tests-topologies/src/main/java/org/apache/pulsar/tests/upgrade/PulsarZKDowngradeTest.java
+++
b/tests/integration-tests-topologies/src/main/java/org/apache/pulsar/tests/upgrade/PulsarZKDowngradeTest.java
@@ -16,7 +16,7 @@
* specific language governing permissions and limitations
* under the License.
*/
-package org.apache.pulsar.tests.integration.upgrade;
+package org.apache.pulsar.tests.upgrade;
import static java.util.stream.Collectors.joining;
import static org.testng.Assert.assertEquals;
@@ -33,9 +33,9 @@ import org.apache.pulsar.client.api.Message;
import org.apache.pulsar.client.api.Producer;
import org.apache.pulsar.client.api.PulsarClient;
import org.apache.pulsar.client.api.Schema;
-import org.apache.pulsar.tests.integration.topologies.PulsarCluster;
-import org.apache.pulsar.tests.integration.topologies.PulsarClusterSpec;
-import org.apache.pulsar.tests.integration.topologies.PulsarClusterTestBase;
+import org.apache.pulsar.tests.topologies.PulsarCluster;
+import org.apache.pulsar.tests.topologies.PulsarClusterSpec;
+import org.apache.pulsar.tests.topologies.PulsarClusterTestBase;
import org.testng.annotations.AfterSuite;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.Test;
[incubator-pulsar] 01/04: Add multi-topic and regex consumer in Go client (#2448)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
commit e1dddcf21608b76f57e3d4830d45fbf87f8d2e99
Author: Matteo Merli
AuthorDate: Mon Aug 27 16:34:00 2018 -0700
Add multi-topic and regex consumer in Go client (#2448)
---
pulsar-client-cpp/include/pulsar/c/client.h | 10 +++
pulsar-client-cpp/lib/c/c_Client.cc | 21 +
pulsar-client-go/pulsar/c_consumer.go | 38 +++--
pulsar-client-go/pulsar/c_go_pulsar.h | 13 +++
pulsar-client-go/pulsar/consumer.go | 10 ++-
pulsar-client-go/pulsar/consumer_test.go| 124 +++-
6 files changed, 207 insertions(+), 9 deletions(-)
diff --git a/pulsar-client-cpp/include/pulsar/c/client.h
b/pulsar-client-cpp/include/pulsar/c/client.h
index 2da7c6d..4b603bb 100644
--- a/pulsar-client-cpp/include/pulsar/c/client.h
+++ b/pulsar-client-cpp/include/pulsar/c/client.h
@@ -86,6 +86,16 @@ void pulsar_client_subscribe_async(pulsar_client_t *client,
const char *topic, c
const pulsar_consumer_configuration_t *conf,
pulsar_subscribe_callback callback, void
*ctx);
+void pulsar_client_subscribe_multi_topics_async(pulsar_client_t *client, const
char **topics, int topicsCount,
+const char *subscriptionName,
+const
pulsar_consumer_configuration_t *conf,
+pulsar_subscribe_callback
callback, void *ctx);
+
+void pulsar_client_subscribe_pattern_async(pulsar_client_t *client, const char
*topicPattern,
+ const char *subscriptionName,
+ const
pulsar_consumer_configuration_t *conf,
+ pulsar_subscribe_callback callback,
void *ctx);
+
/**
* Create a topic reader with given {@code ReaderConfiguration} for reading
messages from the specified
* topic.
diff --git a/pulsar-client-cpp/lib/c/c_Client.cc
b/pulsar-client-cpp/lib/c/c_Client.cc
index 905e410..ecefe20 100644
--- a/pulsar-client-cpp/lib/c/c_Client.cc
+++ b/pulsar-client-cpp/lib/c/c_Client.cc
@@ -98,6 +98,27 @@ void pulsar_client_subscribe_async(pulsar_client_t *client,
const char *topic, c
boost::bind(_subscribe_callback, _1,
_2, callback, ctx));
}
+void pulsar_client_subscribe_multi_topics_async(pulsar_client_t *client, const
char **topics, int topicsCount,
+const char *subscriptionName,
+const
pulsar_consumer_configuration_t *conf,
+pulsar_subscribe_callback
callback, void *ctx) {
+std::vector topicsList;
+for (int i = 0; i < topicsCount; i++) {
+topicsList.push_back(topics[i]);
+}
+
+client->client->subscribeAsync(topicsList, subscriptionName,
conf->consumerConfiguration,
+ boost::bind(_subscribe_callback, _1,
_2, callback, ctx));
+}
+
+void pulsar_client_subscribe_pattern_async(pulsar_client_t *client, const char
*topicPattern,
+ const char *subscriptionName,
+ const
pulsar_consumer_configuration_t *conf,
+ pulsar_subscribe_callback callback,
void *ctx) {
+client->client->subscribeWithRegexAsync(topicPattern, subscriptionName,
conf->consumerConfiguration,
+
boost::bind(_subscribe_callback, _1, _2, callback, ctx));
+}
+
pulsar_result pulsar_client_create_reader(pulsar_client_t *client, const char
*topic,
const pulsar_message_id_t
*startMessageId,
pulsar_reader_configuration_t *conf,
pulsar_reader_t **c_reader) {
diff --git a/pulsar-client-go/pulsar/c_consumer.go
b/pulsar-client-go/pulsar/c_consumer.go
index aebabc8..093dd9d 100644
--- a/pulsar-client-go/pulsar/c_consumer.go
+++ b/pulsar-client-go/pulsar/c_consumer.go
@@ -25,10 +25,10 @@ package pulsar
import "C"
import (
+ "context"
"runtime"
"time"
"unsafe"
- "context"
)
type consumer struct {
@@ -64,7 +64,7 @@ type subscribeContext struct {
}
func subscribeAsync(client *client, options ConsumerOptions, callback
func(Consumer, error)) {
- if options.Topic == "" {
+ if options.Topic == "" && options.Topics == nil &&
options.TopicsPattern == "" {
go callback(nil, newError(C.pulsar_result_InvalidConfiguration,
"topic is requi
[incubator-pulsar] 02/04: Support compaction options in Go client (#2449)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
commit 74862956221c7353df551dbe17defe39860d2c61
Author: Matteo Merli
AuthorDate: Tue Aug 28 06:42:42 2018 -0700
Support compaction options in Go client (#2449)
Added the option for ReadCompacted in Go based consumer/reader
configuration.
---
pulsar-client-go/pulsar/c_consumer.go| 2 +
pulsar-client-go/pulsar/c_reader.go | 2 +
pulsar-client-go/pulsar/consumer.go | 12 ++-
pulsar-client-go/pulsar/consumer_test.go | 141 ++-
pulsar-client-go/pulsar/producer_test.go | 1 -
pulsar-client-go/pulsar/reader.go| 9 ++
pulsar-client-go/pulsar/reader_test.go | 91
7 files changed, 255 insertions(+), 3 deletions(-)
diff --git a/pulsar-client-go/pulsar/c_consumer.go
b/pulsar-client-go/pulsar/c_consumer.go
index 093dd9d..7f613b2 100644
--- a/pulsar-client-go/pulsar/c_consumer.go
+++ b/pulsar-client-go/pulsar/c_consumer.go
@@ -120,6 +120,8 @@ func subscribeAsync(client *client, options
ConsumerOptions, callback func(Consu
C.pulsar_consumer_set_consumer_name(conf, name)
}
+ C.pulsar_consumer_set_read_compacted(conf, cBool(options.ReadCompacted))
+
subName := C.CString(options.SubscriptionName)
defer C.free(unsafe.Pointer(subName))
diff --git a/pulsar-client-go/pulsar/c_reader.go
b/pulsar-client-go/pulsar/c_reader.go
index 12c1103..04bb5cf 100644
--- a/pulsar-client-go/pulsar/c_reader.go
+++ b/pulsar-client-go/pulsar/c_reader.go
@@ -99,6 +99,8 @@ func createReaderAsync(client *client, options ReaderOptions,
callback func(Read
C.pulsar_reader_configuration_set_subscription_role_prefix(conf, prefix)
}
+ C.pulsar_reader_configuration_set_read_compacted(conf,
cBool(options.ReadCompacted))
+
if options.Name != "" {
name := C.CString(options.Name)
defer C.free(unsafe.Pointer(name))
diff --git a/pulsar-client-go/pulsar/consumer.go
b/pulsar-client-go/pulsar/consumer.go
index ed56d9e..b9f2616 100644
--- a/pulsar-client-go/pulsar/consumer.go
+++ b/pulsar-client-go/pulsar/consumer.go
@@ -20,8 +20,8 @@
package pulsar
import (
- "time"
"context"
+ "time"
)
// Pair of a Consumer and Message
@@ -92,6 +92,16 @@ type ConsumerOptions struct {
// Set the consumer name.
Name string
+
+ // If enabled, the consumer will read messages from the compacted topic
rather than reading the full message backlog
+ // of the topic. This means that, if the topic has been compacted, the
consumer will only see the latest value for
+ // each key in the topic, up until the point in the topic message
backlog that has been compacted. Beyond that
+ // point, the messages will be sent as normal.
+ //
+ // ReadCompacted can only be enabled subscriptions to persistent
topics, which have a single active consumer (i.e.
+ // failure or exclusive subscriptions). Attempting to enable it on
subscriptions to a non-persistent topics or on a
+ // shared subscription, will lead to the subscription call throwing a
PulsarClientException.
+ ReadCompacted bool
}
// An interface that abstracts behavior of Pulsar's consumer
diff --git a/pulsar-client-go/pulsar/consumer_test.go
b/pulsar-client-go/pulsar/consumer_test.go
index 75a454b..f81ce56 100644
--- a/pulsar-client-go/pulsar/consumer_test.go
+++ b/pulsar-client-go/pulsar/consumer_test.go
@@ -22,6 +22,9 @@ package pulsar
import (
"context"
"fmt"
+ "io/ioutil"
+ "net/http"
+ "strings"
"testing"
"time"
)
@@ -99,6 +102,111 @@ func TestConsumer(t *testing.T) {
consumer.Unsubscribe()
}
+func TestConsumerCompaction(t *testing.T) {
+ client, err := NewClient(ClientOptions{
+ URL: "pulsar://localhost:6650",
+ })
+
+ assertNil(t, err)
+ defer client.Close()
+
+ topic := fmt.Sprintf("my-compaction-topic-%d", time.Now().Unix())
+
+ producer, err := client.CreateProducer(ProducerOptions{
+ Topic: topic,
+ })
+
+ assertNil(t, err)
+ defer producer.Close()
+
+ // Pre-create both subscriptions to retain published messages
+ consumer1, err := client.Subscribe(ConsumerOptions{
+ Topic:topic,
+ SubscriptionName: "my-sub-1",
+ })
+
+ assertNil(t, err)
+ consumer1.Close()
+
+ consumer2, err := client.Subscribe(ConsumerOptions{
+ Topic:topic,
+ SubscriptionName: "my-sub-2",
+ ReadCompacted:true,
+ })
[incubator-pulsar] 03/04: Added producer/consumer properties in Go client (#2447)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
commit af1ff3a789bec223b102bc7e27e33cf7bdc17104
Author: Matteo Merli
AuthorDate: Fri Aug 31 15:39:11 2018 -0700
Added producer/consumer properties in Go client (#2447)
---
pulsar-client-cpp/include/pulsar/ProducerConfiguration.h| 5 +++--
pulsar-client-cpp/include/pulsar/c/consumer_configuration.h | 3 +++
pulsar-client-cpp/include/pulsar/c/producer_configuration.h | 3 +++
pulsar-client-cpp/lib/c/c_ConsumerConfiguration.cc | 5 +
pulsar-client-cpp/lib/c/c_ProducerConfiguration.cc | 5 +
pulsar-client-go/pulsar/c_consumer.go | 12
pulsar-client-go/pulsar/c_producer.go | 12
pulsar-client-go/pulsar/consumer.go | 4
pulsar-client-go/pulsar/producer.go | 4
pulsar-client-go/pulsar/producer_test.go| 8 ++--
10 files changed, 57 insertions(+), 4 deletions(-)
diff --git a/pulsar-client-cpp/include/pulsar/ProducerConfiguration.h
b/pulsar-client-cpp/include/pulsar/ProducerConfiguration.h
index 45154c5..565a6ab 100644
--- a/pulsar-client-cpp/include/pulsar/ProducerConfiguration.h
+++ b/pulsar-client-cpp/include/pulsar/ProducerConfiguration.h
@@ -128,7 +128,7 @@ class ProducerConfiguration {
ProducerConfiguration& addEncryptionKey(std::string key);
/**
- * Check whether the message has a specific property attached.
+ * Check whether the producer has a specific property attached.
*
* @param name the name of the property to check
* @return true if the message has the specified property
@@ -150,7 +150,8 @@ class ProducerConfiguration {
std::map& getProperties() const;
/**
- * Sets a new property on a message.
+ * Sets a new property on the producer
+ * .
* @param name the name of the property
* @param value the associated value
*/
diff --git a/pulsar-client-cpp/include/pulsar/c/consumer_configuration.h
b/pulsar-client-cpp/include/pulsar/c/consumer_configuration.h
index 7299867..fca47eb 100644
--- a/pulsar-client-cpp/include/pulsar/c/consumer_configuration.h
+++ b/pulsar-client-cpp/include/pulsar/c/consumer_configuration.h
@@ -152,6 +152,9 @@ int
pulsar_consumer_is_read_compacted(pulsar_consumer_configuration_t *consumer_
void pulsar_consumer_set_read_compacted(pulsar_consumer_configuration_t
*consumer_configuration,
int compacted);
+void
pulsar_consumer_configuration_set_property(pulsar_consumer_configuration_t
*conf, const char *name,
+const char *value);
+
// const CryptoKeyReaderPtr getCryptoKeyReader()
//
// const;
diff --git a/pulsar-client-cpp/include/pulsar/c/producer_configuration.h
b/pulsar-client-cpp/include/pulsar/c/producer_configuration.h
index ae88198..670bf50 100644
--- a/pulsar-client-cpp/include/pulsar/c/producer_configuration.h
+++ b/pulsar-client-cpp/include/pulsar/c/producer_configuration.h
@@ -134,6 +134,9 @@ void
pulsar_producer_configuration_set_batching_max_publish_delay_ms(pulsar_prod
unsigned long pulsar_producer_configuration_get_batching_max_publish_delay_ms(
pulsar_producer_configuration_t *conf);
+void
pulsar_producer_configuration_set_property(pulsar_producer_configuration_t
*conf, const char *name,
+const char *value);
+
// const CryptoKeyReaderPtr getCryptoKeyReader() const;
// ProducerConfiguration (CryptoKeyReaderPtr
cryptoKeyReader);
//
diff --git a/pulsar-client-cpp/lib/c/c_ConsumerConfiguration.cc
b/pulsar-client-cpp/lib/c/c_ConsumerConfiguration.cc
index c8d5453..75cdc47 100644
--- a/pulsar-client-cpp/lib/c/c_ConsumerConfiguration.cc
+++ b/pulsar-client-cpp/lib/c/c_ConsumerConfiguration.cc
@@ -113,3 +113,8 @@ void
pulsar_consumer_set_read_compacted(pulsar_consumer_configuration_t *consume
int compacted) {
consumer_configuration->consumerConfiguration.setReadCompacted(compacted);
}
+
+void
pulsar_consumer_configuration_set_property(pulsar_consumer_configuration_t
*conf, const char *name,
+const char *value) {
+conf->consumerConfiguration.setProperty(name, value);
+}
diff --git a/pulsar-client-cpp/lib/c/c_ProducerConfiguration.cc
b/pulsar-client-cpp/lib/c/c_ProducerConfiguration.cc
index 914fc0a..a8eb5be 100644
--- a/pulsar-client-cpp/lib/c/c_ProducerConfiguration.cc
+++ b/pulsar-client-cpp/lib/c/c_ProducerConfiguration.cc
@@ -174,3 +174,8 @@ unsigned long
pulsar_producer_configuration_get_batching_max_publish_delay_ms(
pulsar_producer_configuration_t *conf) {
return conf->conf.getBatchingMaxPublishDela
[incubator-pulsar] branch branch-2.1 updated (6a02c94 -> 050d47d)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git. from 6a02c94 Downgrading ZK to stable version 3.4.13 (#2473) new e1dddcf Add multi-topic and regex consumer in Go client (#2448) new 7486295 Support compaction options in Go client (#2449) new af1ff3a Added producer/consumer properties in Go client (#2447) new 050d47d [go] Ensure producer/consumer/reader keep a ref of client instance so it won't be finalized (#2527) The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../include/pulsar/ProducerConfiguration.h | 5 +- pulsar-client-cpp/include/pulsar/c/client.h| 10 + .../include/pulsar/c/consumer_configuration.h | 3 + .../include/pulsar/c/producer_configuration.h | 3 + pulsar-client-cpp/lib/c/c_Client.cc| 21 ++ pulsar-client-cpp/lib/c/c_ConsumerConfiguration.cc | 5 + pulsar-client-cpp/lib/c/c_ProducerConfiguration.cc | 5 + pulsar-client-go/pulsar/c_consumer.go | 55 - pulsar-client-go/pulsar/c_go_pulsar.h | 13 + pulsar-client-go/pulsar/c_producer.go | 24 +- pulsar-client-go/pulsar/c_reader.go| 5 +- pulsar-client-go/pulsar/consumer.go| 26 +- pulsar-client-go/pulsar/consumer_test.go | 265 - pulsar-client-go/pulsar/producer.go| 4 + pulsar-client-go/pulsar/producer_test.go | 9 +- pulsar-client-go/pulsar/reader.go | 9 + pulsar-client-go/pulsar/reader_test.go | 91 +++ 17 files changed, 530 insertions(+), 23 deletions(-)
[incubator-pulsar] 04/04: [go] Ensure producer/consumer/reader keep a ref of client instance so it won't be finalized (#2527)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
commit 050d47dcb1e3138d093941f9a00641c7caf7c3fd
Author: Matteo Merli
AuthorDate: Thu Sep 6 07:40:24 2018 -0700
[go] Ensure producer/consumer/reader keep a ref of client instance so it
won't be finalized (#2527)
---
pulsar-client-go/pulsar/c_consumer.go | 3 ++-
pulsar-client-go/pulsar/c_producer.go | 12 +++-
pulsar-client-go/pulsar/c_reader.go | 3 ++-
3 files changed, 11 insertions(+), 7 deletions(-)
diff --git a/pulsar-client-go/pulsar/c_consumer.go
b/pulsar-client-go/pulsar/c_consumer.go
index 1b41a71..c78a58e 100644
--- a/pulsar-client-go/pulsar/c_consumer.go
+++ b/pulsar-client-go/pulsar/c_consumer.go
@@ -32,6 +32,7 @@ import (
)
type consumer struct {
+ client *client
ptr*C.pulsar_consumer_t
defaultChannel chan ConsumerMessage
}
@@ -76,7 +77,7 @@ func subscribeAsync(client *client, options ConsumerOptions,
callback func(Consu
conf := C.pulsar_consumer_configuration_create()
- consumer := {}
+ consumer := {client: client}
if options.MessageChannel == nil {
// If there is no message listener, set a default channel so
that we can have receive to
diff --git a/pulsar-client-go/pulsar/c_producer.go
b/pulsar-client-go/pulsar/c_producer.go
index 284315d..620b64d 100644
--- a/pulsar-client-go/pulsar/c_producer.go
+++ b/pulsar-client-go/pulsar/c_producer.go
@@ -24,13 +24,14 @@ package pulsar
*/
import "C"
import (
+ "context"
"runtime"
- "unsafe"
"time"
- "context"
+ "unsafe"
)
type createProducerCtx struct {
+ client *client
callback func(producer Producer, err error)
conf *C.pulsar_producer_configuration_t
}
@@ -44,7 +45,7 @@ func pulsarCreateProducerCallbackProxy(res C.pulsar_result,
ptr *C.pulsar_produc
if res != C.pulsar_result_Ok {
producerCtx.callback(nil, newError(res, "Failed to create
Producer"))
} else {
- p := {ptr: ptr}
+ p := {client: producerCtx.client, ptr: ptr}
runtime.SetFinalizer(p, producerFinalizer)
producerCtx.callback(p, nil)
}
@@ -140,7 +141,7 @@ func createProducerAsync(client *client, options
ProducerOptions, callback func(
defer C.free(unsafe.Pointer(topicName))
C._pulsar_client_create_producer_async(client.ptr, topicName, conf,
- savePointer(createProducerCtx{callback, conf}))
+ savePointer(createProducerCtx{client,callback, conf}))
}
type topicMetadata struct {
@@ -161,7 +162,8 @@ func pulsarRouterCallbackProxy(msg *C.pulsar_message_t,
metadata *C.pulsar_topic
/// Producer
type producer struct {
- ptr *C.pulsar_producer_t
+ client *client
+ ptr*C.pulsar_producer_t
}
func producerFinalizer(p *producer) {
diff --git a/pulsar-client-go/pulsar/c_reader.go
b/pulsar-client-go/pulsar/c_reader.go
index 04bb5cf..7336c1a 100644
--- a/pulsar-client-go/pulsar/c_reader.go
+++ b/pulsar-client-go/pulsar/c_reader.go
@@ -31,6 +31,7 @@ import (
)
type reader struct {
+ client *client
ptr*C.pulsar_reader_t
defaultChannel chan ReaderMessage
}
@@ -73,7 +74,7 @@ func createReaderAsync(client *client, options ReaderOptions,
callback func(Read
return
}
- reader := {}
+ reader := {client: client}
if options.MessageChannel == nil {
// If there is no message listener, set a default channel so
that we can have receive to
[incubator-pulsar] branch branch-2.1 updated: Downgrading ZK to stable version 3.4.13 (#2473)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/branch-2.1 by this push: new 6a02c94 Downgrading ZK to stable version 3.4.13 (#2473) 6a02c94 is described below commit 6a02c9434fb3dbb821d0ce6766f0ba49f00565e2 Author: Matteo Merli AuthorDate: Thu Sep 6 10:58:17 2018 -0700 Downgrading ZK to stable version 3.4.13 (#2473) * Downgrading ZK to stable version 3.4.13 * Added integration test for ZK server downgrade * Fixed ZookeeperClientFactoryImplTest.testZKCreationFailure * Fixed dependencies version in license file * Addressed comments * Fixed jline version in license file * There are 2 jline jars to count --- distribution/server/licenses/LICENSE-JLine.txt | 29 + distribution/server/src/assemble/LICENSE.bin.txt | 43 +++ pom.xml| 8 +- pulsar-zookeeper-utils/pom.xml | 57 + .../pulsar/zookeeper/FileTxnSnapLogWrapper.java| 60 -- .../pulsar/zookeeper/LocalBookkeeperEnsemble.java | 2 +- .../pulsar/zookeeper/SerializeUtilsAspect.java | 130 + .../zookeeper/ZookeeperBkClientFactoryImpl.java| 7 +- .../zookeeper/ZookeeperClientFactoryImplTest.java | 12 +- .../pulsar/zookeeper/ZooKeeperServerAspect.java| 16 --- .../pulsar/tests/topologies/PulsarCluster.java | 13 +++ .../pulsar/tests/topologies/PulsarClusterSpec.java | 9 +- .../tests/upgrade/PulsarZKDowngradeTest.java | 115 ++ .../src/main/resources/zk-3.5-test-data/log.1 | Bin 0 -> 20480 bytes .../src/main/resources/zk-3.5-test-data/log.85 | Bin 0 -> 1024 bytes .../src/main/resources/zk-3.5-test-data/snapshot.0 | Bin 0 -> 424 bytes .../main/resources/zk-3.5-test-data/snapshot.84| Bin 0 -> 10747 bytes 17 files changed, 394 insertions(+), 107 deletions(-) diff --git a/distribution/server/licenses/LICENSE-JLine.txt b/distribution/server/licenses/LICENSE-JLine.txt new file mode 100644 index 000..9a34d43 --- /dev/null +++ b/distribution/server/licenses/LICENSE-JLine.txt @@ -0,0 +1,29 @@ +Redistribution and use in source and binary forms, with or +without modification, are permitted provided that the following +conditions are met: + +Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + +Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with +the distribution. + +Neither the name of JLine nor the names of its contributors +may be used to endorse or promote products derived from this +software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, +BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY +AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO +EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, +OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED +AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING +IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED +OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/distribution/server/src/assemble/LICENSE.bin.txt b/distribution/server/src/assemble/LICENSE.bin.txt index d813041..bef4498 100644 --- a/distribution/server/src/assemble/LICENSE.bin.txt +++ b/distribution/server/src/assemble/LICENSE.bin.txt @@ -370,27 +370,27 @@ The Apache Software License, Version 2.0 - org.apache.logging.log4j-log4j-web-2.10.0.jar * Java Native Access JNA -- net.java.dev.jna-jna-4.2.0.jar * BookKeeper -- org.apache.bookkeeper-bookkeeper-common-4.7.1.jar -- org.apache.bookkeeper-bookkeeper-proto-4.7.1.jar -- org.apache.bookkeeper-bookkeeper-server-4.7.1.jar -- org.apache.bookkeeper-circe-checksum-4.7.1.jar -- org.apache.bookkeeper-statelib-4.7.1.jar -- org.apache.bookkeeper-stream-storage-api-4.7.1.jar -- org.apache.bookkeeper-stream-storage-common-4.7.1.jar -- org.apache.bookkeeper-stream-storage-java-client-4.7.1.jar -- org.apache.bookkeeper-stream-storage-java-client-base-4.7.1.jar -- org.apache.bookkeeper-stream-storage-proto-4.7.1.jar -- org.apache.bookkeeper-stream-storage-server-4.7.1.jar -- org.apache.bookkeeper-stream-storage-service-api-4.7.1.jar -- org.apache.bookkeeper-stream-storage-
[incubator-pulsar] branch master updated: Downgrading ZK to stable version 3.4.13 (#2473)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/master by this push: new d734738 Downgrading ZK to stable version 3.4.13 (#2473) d734738 is described below commit d734738d439963a5ed0348ab778385f8c4e8d35b Author: Matteo Merli AuthorDate: Thu Sep 6 10:58:17 2018 -0700 Downgrading ZK to stable version 3.4.13 (#2473) * Downgrading ZK to stable version 3.4.13 * Added integration test for ZK server downgrade * Fixed ZookeeperClientFactoryImplTest.testZKCreationFailure * Fixed dependencies version in license file * Addressed comments * Fixed jline version in license file * There are 2 jline jars to count --- distribution/server/licenses/LICENSE-JLine.txt | 29 + distribution/server/src/assemble/LICENSE.bin.txt | 43 +++ pom.xml| 8 +- pulsar-sql/presto-distribution/LICENSE | 4 +- pulsar-zookeeper-utils/pom.xml | 57 + .../pulsar/zookeeper/FileTxnSnapLogWrapper.java| 60 -- .../pulsar/zookeeper/LocalBookkeeperEnsemble.java | 2 +- .../pulsar/zookeeper/SerializeUtilsAspect.java | 130 + .../zookeeper/ZookeeperBkClientFactoryImpl.java| 7 +- .../zookeeper/ZookeeperClientFactoryImplTest.java | 12 +- .../pulsar/zookeeper/ZooKeeperServerAspect.java| 16 --- .../integration/topologies/PulsarCluster.java | 10 ++ .../integration/topologies/PulsarClusterSpec.java | 8 ++ .../integration/upgrade/PulsarZKDowngradeTest.java | 115 ++ .../src/test/resources/zk-3.5-test-data/log.1 | Bin 0 -> 20480 bytes .../src/test/resources/zk-3.5-test-data/log.85 | Bin 0 -> 1024 bytes .../src/test/resources/zk-3.5-test-data/snapshot.0 | Bin 0 -> 424 bytes .../test/resources/zk-3.5-test-data/snapshot.84| Bin 0 -> 10747 bytes 18 files changed, 394 insertions(+), 107 deletions(-) diff --git a/distribution/server/licenses/LICENSE-JLine.txt b/distribution/server/licenses/LICENSE-JLine.txt new file mode 100644 index 000..9a34d43 --- /dev/null +++ b/distribution/server/licenses/LICENSE-JLine.txt @@ -0,0 +1,29 @@ +Redistribution and use in source and binary forms, with or +without modification, are permitted provided that the following +conditions are met: + +Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + +Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with +the distribution. + +Neither the name of JLine nor the names of its contributors +may be used to endorse or promote products derived from this +software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, +BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY +AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO +EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, +OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED +AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING +IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED +OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/distribution/server/src/assemble/LICENSE.bin.txt b/distribution/server/src/assemble/LICENSE.bin.txt index e0b1eb7..5f0f07b 100644 --- a/distribution/server/src/assemble/LICENSE.bin.txt +++ b/distribution/server/src/assemble/LICENSE.bin.txt @@ -366,27 +366,27 @@ The Apache Software License, Version 2.0 - org.apache.logging.log4j-log4j-web-2.10.0.jar * Java Native Access JNA -- net.java.dev.jna-jna-4.2.0.jar * BookKeeper -- org.apache.bookkeeper-bookkeeper-common-4.7.1.jar -- org.apache.bookkeeper-bookkeeper-proto-4.7.1.jar -- org.apache.bookkeeper-bookkeeper-server-4.7.1.jar -- org.apache.bookkeeper-circe-checksum-4.7.1.jar -- org.apache.bookkeeper-statelib-4.7.1.jar -- org.apache.bookkeeper-stream-storage-api-4.7.1.jar -- org.apache.bookkeeper-stream-storage-common-4.7.1.jar -- org.apache.bookkeeper-stream-storage-java-client-4.7.1.jar -- org.apache.bookkeeper-stream-storage-java-client-base-4.7.1.jar -- org.apache.bookkeeper-stream-storage-proto-4.7.1.jar -- org.apache.bookkeeper-stream-storage-server-4.7.1.jar -- org.apache.bookkeeper-stream-storage-service-api-4.7.1.jar --
[incubator-pulsar] branch master updated: [go] Ensure producer/consumer/reader keep a ref of client instance so it won't be finalized (#2527)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 3ffcf61 [go] Ensure producer/consumer/reader keep a ref of client
instance so it won't be finalized (#2527)
3ffcf61 is described below
commit 3ffcf61c45be5d64c1f6ec538cdd38cfe0d07c2b
Author: Matteo Merli
AuthorDate: Thu Sep 6 07:40:24 2018 -0700
[go] Ensure producer/consumer/reader keep a ref of client instance so it
won't be finalized (#2527)
---
pulsar-client-go/pulsar/c_consumer.go | 3 ++-
pulsar-client-go/pulsar/c_producer.go | 12 +++-
pulsar-client-go/pulsar/c_reader.go | 3 ++-
3 files changed, 11 insertions(+), 7 deletions(-)
diff --git a/pulsar-client-go/pulsar/c_consumer.go
b/pulsar-client-go/pulsar/c_consumer.go
index 1b41a71..c78a58e 100644
--- a/pulsar-client-go/pulsar/c_consumer.go
+++ b/pulsar-client-go/pulsar/c_consumer.go
@@ -32,6 +32,7 @@ import (
)
type consumer struct {
+ client *client
ptr*C.pulsar_consumer_t
defaultChannel chan ConsumerMessage
}
@@ -76,7 +77,7 @@ func subscribeAsync(client *client, options ConsumerOptions,
callback func(Consu
conf := C.pulsar_consumer_configuration_create()
- consumer := {}
+ consumer := {client: client}
if options.MessageChannel == nil {
// If there is no message listener, set a default channel so
that we can have receive to
diff --git a/pulsar-client-go/pulsar/c_producer.go
b/pulsar-client-go/pulsar/c_producer.go
index 284315d..620b64d 100644
--- a/pulsar-client-go/pulsar/c_producer.go
+++ b/pulsar-client-go/pulsar/c_producer.go
@@ -24,13 +24,14 @@ package pulsar
*/
import "C"
import (
+ "context"
"runtime"
- "unsafe"
"time"
- "context"
+ "unsafe"
)
type createProducerCtx struct {
+ client *client
callback func(producer Producer, err error)
conf *C.pulsar_producer_configuration_t
}
@@ -44,7 +45,7 @@ func pulsarCreateProducerCallbackProxy(res C.pulsar_result,
ptr *C.pulsar_produc
if res != C.pulsar_result_Ok {
producerCtx.callback(nil, newError(res, "Failed to create
Producer"))
} else {
- p := {ptr: ptr}
+ p := {client: producerCtx.client, ptr: ptr}
runtime.SetFinalizer(p, producerFinalizer)
producerCtx.callback(p, nil)
}
@@ -140,7 +141,7 @@ func createProducerAsync(client *client, options
ProducerOptions, callback func(
defer C.free(unsafe.Pointer(topicName))
C._pulsar_client_create_producer_async(client.ptr, topicName, conf,
- savePointer(createProducerCtx{callback, conf}))
+ savePointer(createProducerCtx{client,callback, conf}))
}
type topicMetadata struct {
@@ -161,7 +162,8 @@ func pulsarRouterCallbackProxy(msg *C.pulsar_message_t,
metadata *C.pulsar_topic
/// Producer
type producer struct {
- ptr *C.pulsar_producer_t
+ client *client
+ ptr*C.pulsar_producer_t
}
func producerFinalizer(p *producer) {
diff --git a/pulsar-client-go/pulsar/c_reader.go
b/pulsar-client-go/pulsar/c_reader.go
index 04bb5cf..7336c1a 100644
--- a/pulsar-client-go/pulsar/c_reader.go
+++ b/pulsar-client-go/pulsar/c_reader.go
@@ -31,6 +31,7 @@ import (
)
type reader struct {
+ client *client
ptr*C.pulsar_reader_t
defaultChannel chan ReaderMessage
}
@@ -73,7 +74,7 @@ func createReaderAsync(client *client, options ReaderOptions,
callback func(Read
return
}
- reader := {}
+ reader := {client: client}
if options.MessageChannel == nil {
// If there is no message listener, set a default channel so
that we can have receive to
[incubator-pulsar] branch master updated: Reintroduce Aerospike connector (#2524)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 2ecd03a Reintroduce Aerospike connector (#2524)
2ecd03a is described below
commit 2ecd03abbbf6128d59d27a9d151ed47cc7ead4c7
Author: Ali Ahmed
AuthorDate: Thu Sep 6 07:39:37 2018 -0700
Reintroduce Aerospike connector (#2524)
---
distribution/io/src/assemble/io.xml| 8 +
pom.xml| 1 +
pulsar-io/{ => aerospike}/pom.xml | 56 +--
.../pulsar/io/aerospike/AerospikeAbstractSink.java | 169 +
.../pulsar/io/aerospike/AerospikeSinkConfig.java | 64
.../pulsar/io/aerospike/AerospikeStringSink.java | 35 +
.../resources/META-INF/services/pulsar-io.yaml | 22 +++
pulsar-io/pom.xml | 1 +
site2/docs/deploy-bare-metal.md| 1 +
site2/docs/getting-started-standalone.md | 1 +
site2/docs/io-aerospike.md | 21 +++
site2/docs/io-connectors.md| 1 +
site2/docs/io-overview.md | 3 +-
site2/docs/io-quickstart.md| 3 +-
14 files changed, 368 insertions(+), 18 deletions(-)
diff --git a/distribution/io/src/assemble/io.xml
b/distribution/io/src/assemble/io.xml
index bb75e84..08ff859 100644
--- a/distribution/io/src/assemble/io.xml
+++ b/distribution/io/src/assemble/io.xml
@@ -74,15 +74,23 @@
connectors
644
+
${basedir}/../../pulsar-io/jdbc/target/pulsar-io-jdbc-${project.version}.nar
connectors
644
+
${basedir}/../../pulsar-io/data-genenator/target/pulsar-io-data-generator-${project.version}.nar
connectors
644
+
+
+
${basedir}/../../pulsar-io/aerospike/target/pulsar-io-aerospike-${project.version}.nar
+ connectors
+ 644
+
diff --git a/pom.xml b/pom.xml
index 681f18e..a35ca81 100644
--- a/pom.xml
+++ b/pom.xml
@@ -162,6 +162,7 @@ flexible messaging model and an intuitive client
API.
0.8.3
2.2.0
3.4.0
+4.1.11
0.10.2.1
5.1.1
1.11.297
diff --git a/pulsar-io/pom.xml b/pulsar-io/aerospike/pom.xml
similarity index 50%
copy from pulsar-io/pom.xml
copy to pulsar-io/aerospike/pom.xml
index e89cc02..34b2d75 100644
--- a/pulsar-io/pom.xml
+++ b/pulsar-io/aerospike/pom.xml
@@ -19,27 +19,51 @@
-->
http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
4.0.0
- pom
org.apache.pulsar
-pulsar
+pulsar-io
2.2.0-incubating-SNAPSHOT
- pulsar-io
- Pulsar IO :: Parent
-
-
-core
-twitter
-cassandra
-kafka
-rabbitmq
-kinesis
-jdbc
-data-genenator
-
+ pulsar-io-aerospike
+ Pulsar IO :: Aerospike
+
+
+
+
+ ${project.groupId}
+ pulsar-io-core
+ ${project.version}
+
+
+
+ com.fasterxml.jackson.core
+ jackson-databind
+
+
+
+ com.fasterxml.jackson.dataformat
+ jackson-dataformat-yaml
+
+
+
+ com.aerospike
+ aerospike-client-bc
+ ${aerospike-client.version}
+
+
+
+
+
+
+
+org.apache.nifi
+nifi-nar-maven-plugin
+
+
+
+
diff --git
a/pulsar-io/aerospike/src/main/java/org/apache/pulsar/io/aerospike/AerospikeAbstractSink.java
b/pulsar-io/aerospike/src/main/java/org/apache/pulsar/io/aerospike/AerospikeAbstractSink.java
new file mode 100644
index 000..fe3787a
--- /dev/null
+++
b/pulsar-io/aerospike/src/main/java/org/apache/pulsar/io/aerospike/AerospikeAbstractSink.java
@@ -0,0 +1,169 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.pulsar.io.aerospike;
+
+impor
[incubator-pulsar] branch branch-2.1 updated: Copy ByteBufPair buffers when using with SSL (#2401) (#2464)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new e0097df Copy ByteBufPair buffers when using with SSL (#2401) (#2464)
e0097df is described below
commit e0097dfc651dbf20eb067889113711c908692b5f
Author: Ivan Kelly
AuthorDate: Sat Sep 1 00:39:31 2018 +0200
Copy ByteBufPair buffers when using with SSL (#2401) (#2464)
The netty SSL handler uses a coalescing buffer queue, which modifies
the buffers used to queue the writes so that SSL_write can be given
larger chunks, thereby increasing the 'goodput'.
If we pass in a retained duplicate as we have been doing until now,
then later clients will be passed junk, as SSL will have modified cached
entry buffers.
This patch introduces a copying ByteBufPair encoder, which is only
used with SSL connections.
---
.../broker/service/PulsarChannelInitializer.java | 4 +++-
.../apache/pulsar/client/impl/ConnectionPool.java | 5 +++--
.../org/apache/pulsar/common/api/ByteBufPair.java | 25 +-
3 files changed, 30 insertions(+), 4 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarChannelInitializer.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarChannelInitializer.java
index 8c16a55..bcb6473 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarChannelInitializer.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarChannelInitializer.java
@@ -56,9 +56,11 @@ public class PulsarChannelInitializer extends
ChannelInitializer
serviceConfig.getTlsCiphers(),
serviceConfig.getTlsProtocols(),
serviceConfig.getTlsRequireTrustedClientCertOnConnect());
ch.pipeline().addLast(TLS_HANDLER, sslCtx.newHandler(ch.alloc()));
+ch.pipeline().addLast("ByteBufPairEncoder",
ByteBufPair.COPYING_ENCODER);
+} else {
+ch.pipeline().addLast("ByteBufPairEncoder", ByteBufPair.ENCODER);
}
-ch.pipeline().addLast("ByteBufPairEncoder", ByteBufPair.ENCODER);
ch.pipeline().addLast("frameDecoder", new
LengthFieldBasedFrameDecoder(PulsarDecoder.MaxFrameSize, 0, 4, 0, 4));
ch.pipeline().addLast("handler", new ServerCnx(brokerService));
}
diff --git
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConnectionPool.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConnectionPool.java
index afba7d2..3188b35 100644
---
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConnectionPool.java
+++
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConnectionPool.java
@@ -100,9 +100,10 @@ public class ConnectionPool implements Closeable {
conf.getTlsTrustCertsFilePath());
}
ch.pipeline().addLast(TLS_HANDLER,
sslCtx.newHandler(ch.alloc()));
+ch.pipeline().addLast("ByteBufPairEncoder",
ByteBufPair.COPYING_ENCODER);
+} else {
+ch.pipeline().addLast("ByteBufPairEncoder",
ByteBufPair.ENCODER);
}
-
-ch.pipeline().addLast("ByteBufPairEncoder",
ByteBufPair.ENCODER);
ch.pipeline().addLast("frameDecoder", new
LengthFieldBasedFrameDecoder(MaxMessageSize, 0, 4, 0, 4));
ch.pipeline().addLast("handler", clientCnxSupplier.get());
}
diff --git
a/pulsar-common/src/main/java/org/apache/pulsar/common/api/ByteBufPair.java
b/pulsar-common/src/main/java/org/apache/pulsar/common/api/ByteBufPair.java
index 94e1fb7..b99270b 100644
--- a/pulsar-common/src/main/java/org/apache/pulsar/common/api/ByteBufPair.java
+++ b/pulsar-common/src/main/java/org/apache/pulsar/common/api/ByteBufPair.java
@@ -109,6 +109,7 @@ public final class ByteBufPair extends
AbstractReferenceCounted {
}
public static final Encoder ENCODER = new Encoder();
+public static final CopyingEncoder COPYING_ENCODER = new CopyingEncoder();
@Sharable
public static class Encoder extends ChannelOutboundHandlerAdapter {
@@ -132,4 +133,26 @@ public final class ByteBufPair extends
AbstractReferenceCounted {
}
}
-}
\ No newline at end of file
+@Sharable
+public static class CopyingEncoder extends ChannelOutboundHandlerAdapter {
+@Override
+public void write(ChannelHandlerContext ctx, Object msg,
ChannelPromise promise) throws Exception {
+if (msg instanceof ByteBufPair) {
+ByteBufPair b = (ByteBufPair) msg;
+
+// Some handlers in the pipeline will modify the bytebufs
[incubator-pulsar] branch master updated: [documentation] Add instructions for contributing c++/python code changes (#2513)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 6fa95fa [documentation] Add instructions for contributing c++/python
code changes (#2513)
6fa95fa is described below
commit 6fa95fa291483d28296760503c1150cf1c393d34
Author: Sijie Guo
AuthorDate: Wed Sep 5 14:41:37 2018 -0700
[documentation] Add instructions for contributing c++/python code changes
(#2513)
---
pulsar-client-cpp/docker-format.sh | 44 ++
site2/website/contributing.md | 24 +
2 files changed, 68 insertions(+)
diff --git a/pulsar-client-cpp/docker-format.sh
b/pulsar-client-cpp/docker-format.sh
new file mode 100755
index 000..f0ce191
--- /dev/null
+++ b/pulsar-client-cpp/docker-format.sh
@@ -0,0 +1,44 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+# Build Pulsar C++ client within a Docker container
+
+# Fail script in case of errors
+set -e
+
+ROOT_DIR=$(git rev-parse --show-toplevel)
+cd $ROOT_DIR/pulsar-client-cpp
+
+BUILD_IMAGE_NAME="${BUILD_IMAGE_NAME:-apachepulsar/pulsar-build}"
+BUILD_IMAGE_VERSION="${BUILD_IMAGE_VERSION:-ubuntu-16.04}"
+
+IMAGE="$BUILD_IMAGE_NAME:$BUILD_IMAGE_VERSION"
+
+echo " Build Pulsar C++ client using image $IMAGE"
+
+docker pull $IMAGE
+
+DOCKER_CMD="docker run -i -v $ROOT_DIR:/pulsar $IMAGE"
+
+# Remove any cached CMake relate file from previous builds
+find . -name CMakeCache.txt | xargs rm -f
+find . -name CMakeFiles | xargs rm -rf
+
+$DOCKER_CMD bash -c "cd /pulsar/pulsar-client-cpp && cmake . $CMAKE_ARGS &&
make format"
diff --git a/site2/website/contributing.md b/site2/website/contributing.md
index a82a287..9552244 100644
--- a/site2/website/contributing.md
+++ b/site2/website/contributing.md
@@ -181,6 +181,30 @@ Then you can push your local, committed changes to your
(forked) repository on G
All code should have appropriate unit testing coverage. New code should have
new tests in the same contribution. Bug fixes should include a regression test
to prevent the issue from reoccurring.
+### Develop C++/Python Client
+
+Currently C++ and Python client are developed under `pulsar-client-cpp`
directory. You can use following commands to build and test your c++ or python
code changes.
+
+ Build C++/Python Client
+
+```
+$ ./pulsar-client-cpp/docker-build.sh
+```
+
+### Run C++/Python Client Tests
+
+```
+$ ./pulsar-client-cpp/docker-test.sh
+```
+
+### Checkstyle
+
+Before you push the C++/Python changes for review, you should format the files
to apply the checkstyle rule Pulsar is using for C++/Python code.
+
+```
+$ ./pulsar-client-cpp/docker-format.sh
+```
+
## Review
Once the initial code is complete and the tests pass, it’s time to start the
code review process. We review and discuss all code, no matter who authors it.
It’s a great way to build community, since you can learn from other developers,
and they become familiar with your contribution. It also builds a strong
project by encouraging a high quality bar and keeping code consistent
throughout the project.
[incubator-pulsar] branch master updated: Handle /managed-ledgers znode existance on cluster init (#2379) (#2510)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 889b1bf Handle /managed-ledgers znode existance on cluster init
(#2379) (#2510)
889b1bf is described below
commit 889b1bfe8c28075d64ee02bd63510de0d01772b9
Author: Ivan Kelly
AuthorDate: Tue Sep 4 23:41:47 2018 +0200
Handle /managed-ledgers znode existance on cluster init (#2379) (#2510)
ManagedLedger clients may create the /managed-ledger znode on boot, so
if a broker starts before the metadata is initialized, it could
potentially block initialization.
This patch changes this by making the existance of /managed-ledger a
non-error condition.
---
.../main/java/org/apache/pulsar/PulsarClusterMetadataSetup.java| 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/PulsarClusterMetadataSetup.java
b/pulsar-broker/src/main/java/org/apache/pulsar/PulsarClusterMetadataSetup.java
index e35daef..9dd065a 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/PulsarClusterMetadataSetup.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/PulsarClusterMetadataSetup.java
@@ -147,7 +147,12 @@ public class PulsarClusterMetadataSetup {
}
}
-localZk.create("/managed-ledgers", new byte[0],
ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
+try {
+localZk.create("/managed-ledgers", new byte[0],
ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
+} catch (NodeExistsException e) {
+// Ignore
+}
+
localZk.create("/namespace", new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE,
CreateMode.PERSISTENT);
try {
[incubator-pulsar] branch master updated: Include pulsar python client dependencies in pulsar image (#2491)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 16f3517 Include pulsar python client dependencies in pulsar image
(#2491)
16f3517 is described below
commit 16f3517b9d02fee072fa07f987bb150eb3efd92f
Author: Sijie Guo
AuthorDate: Sun Sep 2 12:19:36 2018 -0700
Include pulsar python client dependencies in pulsar image (#2491)
---
docker/pulsar/Dockerfile | 6 ++-
docker/pulsar/pom.xml | 44 ++
.../pulsar}/scripts/install-pulsar-client.sh | 0
.../docker-images/latest-version-image/Dockerfile | 8 +---
tests/docker-images/latest-version-image/pom.xml | 44 --
5 files changed, 51 insertions(+), 51 deletions(-)
diff --git a/docker/pulsar/Dockerfile b/docker/pulsar/Dockerfile
index 508b4f3..f4f93c1 100644
--- a/docker/pulsar/Dockerfile
+++ b/docker/pulsar/Dockerfile
@@ -20,7 +20,7 @@
FROM openjdk:8-jdk
# Install some utilities
-RUN apt-get update && apt-get install -y netcat dnsutils python-kazoo
python-yaml
+RUN apt-get update && apt-get install -y netcat dnsutils python-kazoo
python-yaml python-pip
ARG PULSAR_TARBALL
@@ -32,6 +32,10 @@ COPY scripts/gen-yml-from-env.py /pulsar/bin
COPY scripts/generate-zookeeper-config.sh /pulsar/bin
COPY scripts/pulsar-zookeeper-ruok.sh /pulsar/bin
COPY scripts/watch-znode.py /pulsar/bin
+COPY scripts/install-pulsar-client.sh /pulsar/bin
+
+ADD target/python-client/ /pulsar/pulsar-client
+RUN /pulsar/bin/install-pulsar-client.sh
WORKDIR /pulsar
diff --git a/docker/pulsar/pom.xml b/docker/pulsar/pom.xml
index f798c9a..d7a94c6 100644
--- a/docker/pulsar/pom.xml
+++ b/docker/pulsar/pom.xml
@@ -47,6 +47,50 @@
docker
+
+
+org.codehaus.mojo
+exec-maven-plugin
+${exec-maven-plugin.version}
+
+
+build-pulsar-clients
+compile
+
+ exec
+
+
+ ${skipBuildPythonClient}
+
${project.basedir}/target
+
${project.basedir}/../../pulsar-client-cpp/docker/build-wheels.sh
+
+
+2.7 cp27-cp27mu
+
+
+
+
+
+
+org.apache.maven.plugins
+maven-antrun-plugin
+
+
+compile
+
+ run
+
+
+ ${skipBuildPythonClient}
+
+copy python wheel file
+
+
+
+
+
+
+
com.spotify
dockerfile-maven-plugin
diff --git
a/tests/docker-images/latest-version-image/scripts/install-pulsar-client.sh
b/docker/pulsar/scripts/install-pulsar-client.sh
similarity index 100%
rename from
tests/docker-images/latest-version-image/scripts/install-pulsar-client.sh
rename to docker/pulsar/scripts/install-pulsar-client.sh
diff --git a/tests/docker-images/latest-version-image/Dockerfile
b/tests/docker-images/latest-version-image/Dockerfile
index 9c93de9..862b53c 100644
--- a/tests/docker-images/latest-version-image/Dockerfile
+++ b/tests/docker-images/latest-version-image/Dockerfile
@@ -19,7 +19,7 @@
FROM apachepulsar/pulsar-all:latest
-RUN apt-get update && apt-get install -y supervisor python-pip
+RUN apt-get update && apt-get install -y supervisor
RUN mkdir -p /var/log/pulsar && mkdir -p /var/run/supervisor/ && mkdir -p
/pulsar/ssl
@@ -32,8 +32,4 @@ COPY ssl/ca.cert.pem ssl/broker.key-pk8.pem
ssl/broker.cert.pem \
COPY scripts/init-cluster.sh scripts/run-global-zk.sh scripts/run-local-zk.sh \
scripts/run-bookie.sh scripts/run-broker.sh
scripts/run-functions-worker.sh scripts/run-proxy.sh \
- scripts/install-pulsar-client.sh /pulsar/bin/
-
-ADD target/python-client/ /pulsar/pulsar-client
-RUN /pulsar/bin/install-pulsar-client.sh
-
+ /pulsar/bin/
diff --git a/tests/docker-images/latest-version-image/pom.xml
b/tests/docker-images/latest-version-image/pom.xml
index 8e47055..78a12e2 100644
--- a/tests/docker-images/latest-version-image/pom.xml
+++ b/tests/docker-images/latest-version-image/pom.xml
@@ -48,52 +48,8 @@
-
-org.codehaus.mojo
-exec-maven-plugin
-${exec-maven-plugin.version}
-
-
-build-pulsar-clients
-c
[incubator-pulsar] branch master updated: Fix typos (#2507)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 23afb69 Fix typos (#2507)
23afb69 is described below
commit 23afb6963623498d7ed315976f0280acac245b4c
Author: Masakazu Kitajo
AuthorDate: Sun Sep 2 02:00:08 2018 +0900
Fix typos (#2507)
* Fix typo: mental -> metal
* Fix typo: adnin -> admin
---
site/docs/latest/admin-api/tenants.md | 2 +-
site2/docs/admin-api-schemas.md | 2 +-
site2/docs/admin-api-tenants.md | 2 +-
site2/docs/getting-started-standalone.md| 2 +-
.../versioned_docs/version-2.1.0-incubating/admin-api-schemas.md| 2 +-
.../versioned_docs/version-2.1.0-incubating/admin-api-tenants.md| 2 +-
.../version-2.1.0-incubating/getting-started-standalone.md | 2 +-
7 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/site/docs/latest/admin-api/tenants.md
b/site/docs/latest/admin-api/tenants.md
index a0d90d7..7b81ac3 100644
--- a/site/docs/latest/admin-api/tenants.md
+++ b/site/docs/latest/admin-api/tenants.md
@@ -89,7 +89,7 @@ $ pulsar-admin tenants get my-tenant
### Delete
- pulsar-adnin
+ pulsar-admin
You can delete a tenant using the
[`delete`](../../reference/CliTools#pulsar-admin-tenants-delete) subcommand and
specifying the tenant name:
diff --git a/site2/docs/admin-api-schemas.md b/site2/docs/admin-api-schemas.md
index dae258a..f8cde5b 100644
--- a/site2/docs/admin-api-schemas.md
+++ b/site2/docs/admin-api-schemas.md
@@ -85,7 +85,7 @@ Retrieve the schema of a given version:
### Delete Schema
- pulsar-adnin
+ pulsar-admin
You can delete a schema using the
[`delete`](reference-pulsar-admin.md#delete-8) subcommand.
diff --git a/site2/docs/admin-api-tenants.md b/site2/docs/admin-api-tenants.md
index fdd63bf..866bfb1 100644
--- a/site2/docs/admin-api-tenants.md
+++ b/site2/docs/admin-api-tenants.md
@@ -70,7 +70,7 @@ $ pulsar-admin tenants get my-tenant
### Delete
- pulsar-adnin
+ pulsar-admin
You can delete a tenant using the
[`delete`](reference-pulsar-admin.md#tenants-delete) subcommand and specifying
the tenant name:
diff --git a/site2/docs/getting-started-standalone.md
b/site2/docs/getting-started-standalone.md
index 6207cff..944eee0 100644
--- a/site2/docs/getting-started-standalone.md
+++ b/site2/docs/getting-started-standalone.md
@@ -97,7 +97,7 @@ pulsar-io-twitter-{{pulsar:version}}.nar
> NOTES
>
-> If you are running Pulsar in a bare mental cluster, you need to make sure
`connectors` tarball is unzipped in every broker's pulsar directory
+> If you are running Pulsar in a bare metal cluster, you need to make sure
`connectors` tarball is unzipped in every broker's pulsar directory
> (or in every function-worker's pulsar directory if you are running a
> separate worker cluster for Pulsar functions).
>
> If you are [running Pulsar in Docker](getting-started-docker.md) or
> deploying Pulsar using a docker image (e.g. [K8S](deploy-kubernetes.md) or
> [DCOS](deploy-dcos.md)),
diff --git
a/site2/website/versioned_docs/version-2.1.0-incubating/admin-api-schemas.md
b/site2/website/versioned_docs/version-2.1.0-incubating/admin-api-schemas.md
index 8b2e528..76e04e1 100644
--- a/site2/website/versioned_docs/version-2.1.0-incubating/admin-api-schemas.md
+++ b/site2/website/versioned_docs/version-2.1.0-incubating/admin-api-schemas.md
@@ -86,7 +86,7 @@ Retrieve the schema of a given version:
### Delete Schema
- pulsar-adnin
+ pulsar-admin
You can delete a schema using the
[`delete`](reference-pulsar-admin.md#delete-8) subcommand.
diff --git
a/site2/website/versioned_docs/version-2.1.0-incubating/admin-api-tenants.md
b/site2/website/versioned_docs/version-2.1.0-incubating/admin-api-tenants.md
index 38dbe3b..7303a25 100644
--- a/site2/website/versioned_docs/version-2.1.0-incubating/admin-api-tenants.md
+++ b/site2/website/versioned_docs/version-2.1.0-incubating/admin-api-tenants.md
@@ -71,7 +71,7 @@ $ pulsar-admin tenants get my-tenant
### Delete
- pulsar-adnin
+ pulsar-admin
You can delete a tenant using the
[`delete`](reference-pulsar-admin.md#tenants-delete) subcommand and specifying
the tenant name:
diff --git
a/site2/website/versioned_docs/version-2.1.0-incubating/getting-started-standalone.md
b/site2/website/versioned_docs/version-2.1.0-incubating/getting-started-standalone.md
index 597b450..29cf000 100644
---
a/site2/website/versioned_docs/version-2.1.0-incubating/getting-started-standalone.md
+++
b/site2/website/versioned_docs/version-2.1.0-incubating/getting-started-standalone.md
@@ -99,7 +99,7 @@ pulsar-io-twitter-{{
[incubator-pulsar] branch master updated: Fix sql cli (#2505)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 95f2bf8 Fix sql cli (#2505)
95f2bf8 is described below
commit 95f2bf8817cdc3b0a32bff153a6f7252f7fc3b3d
Author: Boyang Jerry Peng
AuthorDate: Sat Sep 1 09:59:31 2018 -0700
Fix sql cli (#2505)
* minor fix of sql cli args
* remove debug
---
bin/pulsar | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/bin/pulsar b/bin/pulsar
index c2def39..9cacfe7 100755
--- a/bin/pulsar
+++ b/bin/pulsar
@@ -319,9 +319,9 @@ elif [ $COMMAND == "zookeeper-shell" ]; then
elif [ $COMMAND == "compact-topic" ]; then
exec $JAVA $OPTS org.apache.pulsar.compaction.CompactorTool --broker-conf
$PULSAR_BROKER_CONF $@
elif [ $COMMAND == "sql" ]; then
-exec $JAVA -cp "${PRESTO_HOME}/lib/*" com.facebook.presto.cli.Presto
--server localhost:8081 $@
+exec $JAVA -cp "${PRESTO_HOME}/lib/*" com.facebook.presto.cli.Presto
--server localhost:8081 "${@}"
elif [ $COMMAND == "sql-worker" ]; then
-exec ${PRESTO_HOME}/bin/launcher --etc-dir ${PULSAR_PRESTO_CONF} $@
+exec ${PRESTO_HOME}/bin/launcher --etc-dir ${PULSAR_PRESTO_CONF} "${@}"
elif [ $COMMAND == "help" ]; then
pulsar_help;
else
[incubator-pulsar] branch master updated: Copy ByteBufPair buffers when using with SSL (#2401) (#2464)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 34e9fd6 Copy ByteBufPair buffers when using with SSL (#2401) (#2464)
34e9fd6 is described below
commit 34e9fd6a441ccb5a895562c6872a7ce992cb39b9
Author: Ivan Kelly
AuthorDate: Sat Sep 1 00:39:31 2018 +0200
Copy ByteBufPair buffers when using with SSL (#2401) (#2464)
The netty SSL handler uses a coalescing buffer queue, which modifies
the buffers used to queue the writes so that SSL_write can be given
larger chunks, thereby increasing the 'goodput'.
If we pass in a retained duplicate as we have been doing until now,
then later clients will be passed junk, as SSL will have modified cached
entry buffers.
This patch introduces a copying ByteBufPair encoder, which is only
used with SSL connections.
---
.../broker/service/PulsarChannelInitializer.java | 4 +++-
.../apache/pulsar/client/impl/ConnectionPool.java | 5 +++--
.../org/apache/pulsar/common/api/ByteBufPair.java | 25 +-
3 files changed, 30 insertions(+), 4 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarChannelInitializer.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarChannelInitializer.java
index 7858a11..a3fb772 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarChannelInitializer.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarChannelInitializer.java
@@ -57,9 +57,11 @@ public class PulsarChannelInitializer extends
ChannelInitializer
serviceConfig.getTlsCiphers(),
serviceConfig.getTlsProtocols(),
serviceConfig.getTlsRequireTrustedClientCertOnConnect());
ch.pipeline().addLast(TLS_HANDLER, sslCtx.newHandler(ch.alloc()));
+ch.pipeline().addLast("ByteBufPairEncoder",
ByteBufPair.COPYING_ENCODER);
+} else {
+ch.pipeline().addLast("ByteBufPairEncoder", ByteBufPair.ENCODER);
}
-ch.pipeline().addLast("ByteBufPairEncoder", ByteBufPair.ENCODER);
ch.pipeline().addLast("frameDecoder", new
LengthFieldBasedFrameDecoder(PulsarDecoder.MaxFrameSize, 0, 4, 0, 4));
ch.pipeline().addLast("handler", new ServerCnx(pulsar));
}
diff --git
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConnectionPool.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConnectionPool.java
index afba7d2..3188b35 100644
---
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConnectionPool.java
+++
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConnectionPool.java
@@ -100,9 +100,10 @@ public class ConnectionPool implements Closeable {
conf.getTlsTrustCertsFilePath());
}
ch.pipeline().addLast(TLS_HANDLER,
sslCtx.newHandler(ch.alloc()));
+ch.pipeline().addLast("ByteBufPairEncoder",
ByteBufPair.COPYING_ENCODER);
+} else {
+ch.pipeline().addLast("ByteBufPairEncoder",
ByteBufPair.ENCODER);
}
-
-ch.pipeline().addLast("ByteBufPairEncoder",
ByteBufPair.ENCODER);
ch.pipeline().addLast("frameDecoder", new
LengthFieldBasedFrameDecoder(MaxMessageSize, 0, 4, 0, 4));
ch.pipeline().addLast("handler", clientCnxSupplier.get());
}
diff --git
a/pulsar-common/src/main/java/org/apache/pulsar/common/api/ByteBufPair.java
b/pulsar-common/src/main/java/org/apache/pulsar/common/api/ByteBufPair.java
index 94e1fb7..b99270b 100644
--- a/pulsar-common/src/main/java/org/apache/pulsar/common/api/ByteBufPair.java
+++ b/pulsar-common/src/main/java/org/apache/pulsar/common/api/ByteBufPair.java
@@ -109,6 +109,7 @@ public final class ByteBufPair extends
AbstractReferenceCounted {
}
public static final Encoder ENCODER = new Encoder();
+public static final CopyingEncoder COPYING_ENCODER = new CopyingEncoder();
@Sharable
public static class Encoder extends ChannelOutboundHandlerAdapter {
@@ -132,4 +133,26 @@ public final class ByteBufPair extends
AbstractReferenceCounted {
}
}
-}
\ No newline at end of file
+@Sharable
+public static class CopyingEncoder extends ChannelOutboundHandlerAdapter {
+@Override
+public void write(ChannelHandlerContext ctx, Object msg,
ChannelPromise promise) throws Exception {
+if (msg instanceof ByteBufPair) {
+ByteBufPair b = (ByteBufPair) msg;
+
+// Some handlers in the pipeline will modify the bytebufs
passed in to t
[incubator-pulsar] branch master updated: Added producer/consumer properties in Go client (#2447)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 79b88cf Added producer/consumer properties in Go client (#2447)
79b88cf is described below
commit 79b88cf66eb2a4d337e21ade7f699c7d641b5c9a
Author: Matteo Merli
AuthorDate: Fri Aug 31 15:39:11 2018 -0700
Added producer/consumer properties in Go client (#2447)
---
pulsar-client-cpp/include/pulsar/ProducerConfiguration.h| 5 +++--
pulsar-client-cpp/include/pulsar/c/consumer_configuration.h | 3 +++
pulsar-client-cpp/include/pulsar/c/producer_configuration.h | 3 +++
pulsar-client-cpp/lib/c/c_ConsumerConfiguration.cc | 5 +
pulsar-client-cpp/lib/c/c_ProducerConfiguration.cc | 5 +
pulsar-client-go/pulsar/c_consumer.go | 12
pulsar-client-go/pulsar/c_producer.go | 12
pulsar-client-go/pulsar/consumer.go | 4
pulsar-client-go/pulsar/producer.go | 4
pulsar-client-go/pulsar/producer_test.go| 8 ++--
10 files changed, 57 insertions(+), 4 deletions(-)
diff --git a/pulsar-client-cpp/include/pulsar/ProducerConfiguration.h
b/pulsar-client-cpp/include/pulsar/ProducerConfiguration.h
index 45154c5..565a6ab 100644
--- a/pulsar-client-cpp/include/pulsar/ProducerConfiguration.h
+++ b/pulsar-client-cpp/include/pulsar/ProducerConfiguration.h
@@ -128,7 +128,7 @@ class ProducerConfiguration {
ProducerConfiguration& addEncryptionKey(std::string key);
/**
- * Check whether the message has a specific property attached.
+ * Check whether the producer has a specific property attached.
*
* @param name the name of the property to check
* @return true if the message has the specified property
@@ -150,7 +150,8 @@ class ProducerConfiguration {
std::map& getProperties() const;
/**
- * Sets a new property on a message.
+ * Sets a new property on the producer
+ * .
* @param name the name of the property
* @param value the associated value
*/
diff --git a/pulsar-client-cpp/include/pulsar/c/consumer_configuration.h
b/pulsar-client-cpp/include/pulsar/c/consumer_configuration.h
index 7299867..fca47eb 100644
--- a/pulsar-client-cpp/include/pulsar/c/consumer_configuration.h
+++ b/pulsar-client-cpp/include/pulsar/c/consumer_configuration.h
@@ -152,6 +152,9 @@ int
pulsar_consumer_is_read_compacted(pulsar_consumer_configuration_t *consumer_
void pulsar_consumer_set_read_compacted(pulsar_consumer_configuration_t
*consumer_configuration,
int compacted);
+void
pulsar_consumer_configuration_set_property(pulsar_consumer_configuration_t
*conf, const char *name,
+const char *value);
+
// const CryptoKeyReaderPtr getCryptoKeyReader()
//
// const;
diff --git a/pulsar-client-cpp/include/pulsar/c/producer_configuration.h
b/pulsar-client-cpp/include/pulsar/c/producer_configuration.h
index ae88198..670bf50 100644
--- a/pulsar-client-cpp/include/pulsar/c/producer_configuration.h
+++ b/pulsar-client-cpp/include/pulsar/c/producer_configuration.h
@@ -134,6 +134,9 @@ void
pulsar_producer_configuration_set_batching_max_publish_delay_ms(pulsar_prod
unsigned long pulsar_producer_configuration_get_batching_max_publish_delay_ms(
pulsar_producer_configuration_t *conf);
+void
pulsar_producer_configuration_set_property(pulsar_producer_configuration_t
*conf, const char *name,
+const char *value);
+
// const CryptoKeyReaderPtr getCryptoKeyReader() const;
// ProducerConfiguration (CryptoKeyReaderPtr
cryptoKeyReader);
//
diff --git a/pulsar-client-cpp/lib/c/c_ConsumerConfiguration.cc
b/pulsar-client-cpp/lib/c/c_ConsumerConfiguration.cc
index c8d5453..75cdc47 100644
--- a/pulsar-client-cpp/lib/c/c_ConsumerConfiguration.cc
+++ b/pulsar-client-cpp/lib/c/c_ConsumerConfiguration.cc
@@ -113,3 +113,8 @@ void
pulsar_consumer_set_read_compacted(pulsar_consumer_configuration_t *consume
int compacted) {
consumer_configuration->consumerConfiguration.setReadCompacted(compacted);
}
+
+void
pulsar_consumer_configuration_set_property(pulsar_consumer_configuration_t
*conf, const char *name,
+const char *value) {
+conf->consumerConfiguration.setProperty(name, value);
+}
diff --git a/pulsar-client-cpp/lib/c/c_ProducerConfiguration.cc
b/pulsar-client-cpp/lib/c/c_ProducerConfiguration.cc
index 914fc0a..a8eb5be 100644
--- a/pulsar-client-cpp/lib/c/c_ProducerConfiguration.cc
+++ b/pulsar-client-cpp/lib/c/c_ProducerConfiguration.cc
@@ -174,3 +174,8 @@
[incubator-pulsar] branch asf-site updated: Trigger git-site publish
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 625eb54 Trigger git-site publish
625eb54 is described below
commit 625eb549684d6bc637c84ccd10bfe65072e132b2
Author: Matteo Merli
AuthorDate: Fri Aug 31 09:10:57 2018 -0700
Trigger git-site publish
---
content/index.html | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/content/index.html b/content/index.html
index be6dfda..d2d1a26 100644
--- a/content/index.html
+++ b/content/index.html
@@ -92,4 +92,5 @@
inputSelector: '#search_input_react',
algoliaOptions:
{"facetFilters":["language:en","version:2.1.0-incubating"]}
});
-
\ No newline at end of file
+
+
[incubator-pulsar] branch master updated: Clarified project roles in team page (#2470)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new e71540e Clarified project roles in team page (#2470)
e71540e is described below
commit e71540e27e1c577fff5ccddb4bf7b3fe9a90d248
Author: Matteo Merli
AuthorDate: Thu Aug 30 23:28:55 2018 -0700
Clarified project roles in team page (#2470)
---
site2/website/pages/en/team.js | 17 +
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/site2/website/pages/en/team.js b/site2/website/pages/en/team.js
index a85104b..dce284e 100644
--- a/site2/website/pages/en/team.js
+++ b/site2/website/pages/en/team.js
@@ -29,18 +29,19 @@ class Team extends React.Component {
- A successful project requires many people to play many roles.
- Some members write code or documentation, while others are
valuable as testers,
- submitting patches and suggestions.
+ A successful project requires many people to play many roles.
Some write
+ code or documentation, while others are valuable as testers,
submitting
+ patches and suggestions.
- The team is comprised of Members and Contributors.
- Members have direct access to the source of a project and
actively evolve the codebase.
- Contributors improve the project through submission of patches
and
- suggestions to the Members. The number of Contributors to the
project is unbounded.
- Get involved today. All contributions to the project are greatly
appreciated.
+ The team is comprised of PMC members, Committers and
Contributors.
+ Committers have direct access to the source of a project and
actively
+ evolve the codebase. Contributors improve the project through
submission of
+ patches and suggestions to be reviewed by the Committers. The
number of
+ Committers and Contributors to the project is unbounded. Get
involved
+ today. All contributions to the project are greatly appreciated.
[incubator-pulsar] branch master updated: Added Contributing page on website (#2459)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/master by this push: new c12e9ae Added Contributing page on website (#2459) c12e9ae is described below commit c12e9ae0f4cf548186dc8ad64b921c6aa931f837 Author: Matteo Merli AuthorDate: Thu Aug 30 23:28:20 2018 -0700 Added Contributing page on website (#2459) * Added Contributing page on website * Fixed mentions to BookKeeper * Addressed comments * Added section "Becoming a committer" * Also added section for PMC member * Reworded the "become committer" section --- site2/website/contributing.md | 315 + site2/website/core/Footer.js | 2 + site2/website/pages/en/contributing.js | 35 3 files changed, 352 insertions(+) diff --git a/site2/website/contributing.md b/site2/website/contributing.md new file mode 100644 index 000..a82a287 --- /dev/null +++ b/site2/website/contributing.md @@ -0,0 +1,315 @@ + +The Apache Pulsar community welcomes contributions from anyone with a passion for distributed systems! Pulsar has many different opportunities for contributions -- +write new examples/tutorials, add new user-facing libraries, write new Pulsar IO connectors, or participate on the documentation effort. + +We use a review-then-commit workflow in Pulsar for all contributions. + +**For larger contributions or those that affect multiple components:** + +1. **Engage**: We encourage you to work with the Pulsar community on the + [Github Issues](https://github.com/apache/incubator-pulsar/issues) and + [developer’s mailing list](/contact) to identify + good areas for contribution. +1. **Design:** More complicated contributions will likely benefit from some early discussion in + order to scope and design them well. + +**For all contributions:** + +1. **Code:** The best part ;-) +1. **Review:** Submit a pull request with your contribution to our + [GitHub Repo](https://github.com/apache/incubator-pulsar). Work with a committer to review and + iterate on the code, if needed. +1. **Commit:** Once at least 2 Pulsar committers have approved the pull request, a Pulsar committer +will merge it into the master branch (and potentially backport to stable branches in case of +bug fixes). + +We look forward to working with you! + +## Engage + +### Mailing list(s) + +We discuss design and implementation issues on the [d...@pulsar.incubator.apache.org](mailto:d...@pulsar.incubator.apache.org) +mailing list, which is archived [here](https://lists.apache.org/list.html?d...@pulsar.apache.org). +Join by emailing [`dev-subscr...@pulsar.incubator.apache.org`](mailto:dev-subscr...@pulsar.incubator.apache.org). + +If interested, you can also join the other [mailing lists](/contact). + +### Github Issues + +We are using [Github Issues](https://github.com/apache/incubator-pulsar/issues) as the issue tracking +and project management tool, as well as a way to communicate among a very diverse and distributed set +of contributors. To be able to gather feedback, avoid frustration, and avoid duplicated efforts all +Pulsar related work are being tracked there. + +If you do not already have an Github account, sign up [here](https://github.com/join). + +If a quick [search](https://github.com/apache/incubator-pulsar/issues) doesn’t turn up an existing +Github issue for the work you want to contribute, create it. Please discuss your idea with a +committer in Github or, alternatively, on the developer mailing list. + +If there’s an existing Github issue for your intended contribution, please comment about your intended +work. Once the work is understood, a committer will assign the issue to you. If an issue is currently +assigned, please check with the current assignee before reassigning. + +For moderate or large contributions, you should not start coding or writing a design document unless +there is a corresponding Github issue assigned to you for that work. Simple changes, like fixing typos, +do not require an associated issue. + +### Online discussions + +We are using [Apache Pulsar Slack channel](https://apache-pulsar.slack.com/) for online discussions. +You can self-invite yourself by accessing [this link](https://apache-pulsar.herokuapp.com/). + +Slack channels are great for quick questions or discussions on specialized topics. Remember that we +strongly encourage communication via the mailing lists, and we prefer to discuss more complex subjects +by email. Developers should be careful to move or duplicate all the official or useful discussions to +the issue tracking system and/or the dev mailing list. + +## Design + +To avoid potential frustration during the code review cycle, we encourage you to clearly scope and +design non-t
[incubator-pulsar] branch master updated: Revert "Fix: Function assignment can support large number of topics (#2438)" (#2474)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new ce74b11 Revert "Fix: Function assignment can support large number of
topics (#2438)" (#2474)
ce74b11 is described below
commit ce74b11b563462d18ae0068cf2d9d9ef0271165a
Author: Boyang Jerry Peng
AuthorDate: Thu Aug 30 22:44:21 2018 -0700
Revert "Fix: Function assignment can support large number of topics
(#2438)" (#2474)
This reverts commit b283ebd3e51aa86f6b9f0a6607f430996f0032f7.
---
conf/functions_worker.yml | 1 -
.../org/apache/pulsar/io/PulsarSinkE2ETest.java| 2 -
.../functions/worker/FunctionAssignmentTailer.java | 70 ++
.../functions/worker/FunctionRuntimeManager.java | 67 +
.../pulsar/functions/worker/MembershipManager.java | 2 -
.../pulsar/functions/worker/SchedulerManager.java | 46 --
.../org/apache/pulsar/functions/worker/Worker.java | 59 --
.../pulsar/functions/worker/WorkerConfig.java | 3 +-
.../worker/FunctionRuntimeManagerTest.java | 33 +-
.../functions/worker/MembershipManagerTest.java| 3 +-
.../functions/worker/SchedulerManagerTest.java | 8 ---
11 files changed, 67 insertions(+), 227 deletions(-)
diff --git a/conf/functions_worker.yml b/conf/functions_worker.yml
index 497c414..444b7fb 100644
--- a/conf/functions_worker.yml
+++ b/conf/functions_worker.yml
@@ -27,7 +27,6 @@ connectorsDirectory: ./connectors
functionMetadataTopicName: metadata
clusterCoordinationTopicName: coordinate
pulsarFunctionsNamespace: public/functions
-pulsarAssignmentNamespace: public/assignment
pulsarFunctionsCluster: standalone
pulsarServiceUrl: pulsar://localhost:6650
pulsarWebServiceUrl: http://localhost:8080
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarSinkE2ETest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarSinkE2ETest.java
index 54c14c0..dd39222 100644
--- a/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarSinkE2ETest.java
+++ b/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarSinkE2ETest.java
@@ -105,7 +105,6 @@ public class PulsarSinkE2ETest {
WorkerService functionsWorkerService;
final String tenant = "external-repl-prop";
String pulsarFunctionsNamespace = tenant + "/use/pulsar-function-admin";
-String pulsarAssignmentNamespace = tenant + "/use/pulsar-assignment";
String primaryHost;
String workerId;
@@ -213,7 +212,6 @@ public class PulsarSinkE2ETest {
private WorkerService createPulsarFunctionWorker(ServiceConfiguration
config) {
workerConfig = new WorkerConfig();
workerConfig.setPulsarFunctionsNamespace(pulsarFunctionsNamespace);
-workerConfig.setPulsarAssignmentNamespace(pulsarAssignmentNamespace);
workerConfig.setSchedulerClassName(
org.apache.pulsar.functions.worker.scheduler.RoundRobinScheduler.class.getName());
workerConfig.setThreadContainerFactory(new
WorkerConfig.ThreadContainerFactory().setThreadGroupName("use"));
diff --git
a/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionAssignmentTailer.java
b/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionAssignmentTailer.java
index 060120b..366eaba 100644
---
a/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionAssignmentTailer.java
+++
b/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionAssignmentTailer.java
@@ -20,15 +20,11 @@ package org.apache.pulsar.functions.worker;
import lombok.extern.slf4j.Slf4j;
import org.apache.pulsar.client.api.Message;
-import org.apache.pulsar.client.api.MessageId;
import org.apache.pulsar.client.api.PulsarClientException;
import org.apache.pulsar.client.api.Reader;
import org.apache.pulsar.functions.proto.Request;
-import com.google.common.collect.Lists;
-
import java.io.IOException;
-import java.util.List;
import java.util.function.Function;
@Slf4j
@@ -37,22 +33,13 @@ public class FunctionAssignmentTailer
private final FunctionRuntimeManager functionRuntimeManager;
private final Reader reader;
-
-private long currentVersion = 0;
-private final List
currentVersionAssignments;
-private volatile MessageId previousOldAssignmentMsgId = null;
public FunctionAssignmentTailer(FunctionRuntimeManager
functionRuntimeManager,
Reader reader)
throws PulsarClientException {
-this.functionRuntimeManager = functionRuntimeManager;
-this.reader = reader;
-this.currentVersionAssignments = Lists.newArrayList();
-// complete init if reade
[incubator-pulsar] branch master updated: Added the --custom-schema-inputs cli arg to allow custom schema input to functions (#2486)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 03742b7 Added the --custom-schema-inputs cli arg to allow custom
schema input to functions (#2486)
03742b7 is described below
commit 03742b73b3ec0f391840f849041875621b47140f
Author: Sanjeev Kulkarni
AuthorDate: Thu Aug 30 22:39:59 2018 -0700
Added the --custom-schema-inputs cli arg to allow custom schema input to
functions (#2486)
* Added the --custom-schema-inputs cli arg to allow custom schema input to
functions
* Update documentation regarding the same
---
.../src/main/java/org/apache/pulsar/admin/cli/CmdFunctions.java| 7 +++
site2/docs/reference-pulsar-admin.md | 3 +++
2 files changed, 10 insertions(+)
diff --git
a/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdFunctions.java
b/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdFunctions.java
index 18d483c..1ba7751 100644
---
a/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdFunctions.java
+++
b/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdFunctions.java
@@ -256,6 +256,8 @@ public class CmdFunctions extends CmdBase {
protected String DEPRECATED_customSerdeInputString;
@Parameter(names = "--custom-serde-inputs", description = "The map of
input topics to SerDe class names (as a JSON string)")
protected String customSerdeInputString;
+@Parameter(names = "--custom-schema-inputs", description = "The map of
input topics to Schema class names (as a JSON string)")
+protected String customSchemaInputString;
// for backwards compatibility purposes
@Parameter(names = "--outputSerdeClassName", description = "The SerDe
class to be used for messages output by the function", hidden = true)
protected String DEPRECATED_outputSerdeClassName;
@@ -377,6 +379,11 @@ public class CmdFunctions extends CmdBase {
Map customSerdeInputMap = new
Gson().fromJson(customSerdeInputString, type);
functionConfig.setCustomSerdeInputs(customSerdeInputMap);
}
+if (null != customSchemaInputString) {
+Type type = new TypeToken>(){}.getType();
+Map customschemaInputMap = new
Gson().fromJson(customSchemaInputString, type);
+functionConfig.setCustomSchemaInputs(customschemaInputMap);
+}
if (null != topicsPattern) {
functionConfig.setTopicsPattern(topicsPattern);
}
diff --git a/site2/docs/reference-pulsar-admin.md
b/site2/docs/reference-pulsar-admin.md
index 459afce..53d33b1 100644
--- a/site2/docs/reference-pulsar-admin.md
+++ b/site2/docs/reference-pulsar-admin.md
@@ -310,6 +310,7 @@ Options
|`--broker-service-url `|The URL of the Pulsar broker||
|`--classname`|The name of the function’s class||
|`--custom-serde-inputs`|A map of the input topic to SerDe name||
+|`--custom-schema-inputs`|A map of the input topic to Schema class name||
|`--client-auth-params`|Client Authentication Params||
|`--function-config-file`|The path of the YAML config file used to configure
the function||
|`--hostname-verification-enabled`|Enable Hostname verification||
@@ -354,6 +355,7 @@ Options
|`--auto-ack`|Let the functions framework manage acking||
|`--classname`|The name of the function’s class||
|`--custom-serde-inputs`|A map of the input topic to SerDe name||
+|`--custom-schema-inputs`|A map of the input topic to Schema class name||
|`--function-config-file`|The path of the YAML config file used to configure
the function||
|`--inputs`|The input topics for the function (as a comma-separated list if
more than one topic is desired)||
|`--log-topic`|The topic to which logs from this function are published||
@@ -411,6 +413,7 @@ Options
|`--auto-ack`|Let the functions framework manage acking||
|`--classname`|The name of the function’s class||
|`--custom-serde-inputs`|A map of the input topic to SerDe name||
+|`--custom-schema-inputs`|A map of the input topic to Schema class name||
|`--function-config-file`|The path of the YAML config file used to configure
the function||
|`--inputs`|The input topics for the function (as a comma-separated list if
more than one topic is desired)||
|`--log-topic`|The topic to which logs from this function are published||
[incubator-pulsar] branch master updated: Fix pulsar sql cli bug (#2469)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 6aaf613 Fix pulsar sql cli bug (#2469)
6aaf613 is described below
commit 6aaf613e9b94e97041ea0c731b0d9d70d875f755
Author: Boyang Jerry Peng
AuthorDate: Tue Aug 28 16:39:39 2018 -0700
Fix pulsar sql cli bug (#2469)
---
bin/pulsar | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/bin/pulsar b/bin/pulsar
index 5d731ed..80f2d3d 100755
--- a/bin/pulsar
+++ b/bin/pulsar
@@ -318,7 +318,7 @@ elif [ $COMMAND == "zookeeper-shell" ]; then
exec $JAVA $OPTS org.apache.zookeeper.ZooKeeperMain $@
elif [ $COMMAND == "compact-topic" ]; then
exec $JAVA $OPTS org.apache.pulsar.compaction.CompactorTool --broker-conf
$PULSAR_BROKER_CONF $@
-elif [ $COMMAND == "sql-cli" ]; then
+elif [ $COMMAND == "sql" ]; then
exec $JAVA -cp "${PRESTO_HOME}/lib/*" com.facebook.presto.cli.Presto
--server localhost:8081 $@
elif [ $COMMAND == "sql-worker" ]; then
exec ${PRESTO_HOME}/bin/launcher --etc-dir ${PULSAR_PRESTO_CONF} run $@
[incubator-pulsar.wiki] branch master updated: Updated Apache Maturity Model Assessment for Pulsar (markdown)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new 8956428 Updated Apache Maturity Model Assessment for Pulsar (markdown) 8956428 is described below commit 8956428dad51af7dff29db74257261399d886e5f Author: Matteo Merli AuthorDate: Tue Aug 28 15:31:51 2018 -0700 Updated Apache Maturity Model Assessment for Pulsar (markdown) --- Apache-Maturity-Model-Assessment-for-Pulsar.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/Apache-Maturity-Model-Assessment-for-Pulsar.md b/Apache-Maturity-Model-Assessment-for-Pulsar.md index 7657037..89d84e9 100644 --- a/Apache-Maturity-Model-Assessment-for-Pulsar.md +++ b/Apache-Maturity-Model-Assessment-for-Pulsar.md @@ -55,3 +55,6 @@ Mentors and community members are encouraged to contribute to this page and comm | CS30 | Documented voting rules are used to build consensus when discussion is not sufficient. | **YES** The project uses the standard ASF voting rules. Voting rules are clearly stated before the voting starts for each individual vote. | | CS40 | In Apache projects, vetoes are only valid for code commits and are justified by a technical explanation, as per the Apache voting rules defined in CS30. | **YES** The project hasn’t used a veto at any point and relies on mandatory code reviews. | | CS50 | All "important" discussions happen asynchronously in written form on the project's main communications channel. Offline, face-to-face or private discussions that affect the project are also documented on that channel. | **YES** The project has been making all decisions on the project mailing lists. Technical discussions may happen during code reviews, or when commenting on issues. These conversations are also in written form and asynchronous, and are copied back to the proje [...] +| **Independence** | | | +| IN10 | The project is independent from any corporate or organizational influence. | **YES** Most of the active committers are employed by 3 different companies. There are also several other active committers and contributors that participate on a volunteer basis. | +| IN20 | Contributors act as themselves as opposed to representatives of a corporation or organization. | **YES** All contributors are aware that contributions to the project are made on an individual base and not on behalf of a corporation or organization. Issues in project public communications have been discussed throughout the incubation process to make sure every party (committers, contributors, organizations) is aware of, and following, ASF PR directives. |
[incubator-pulsar] branch master updated: Add backlog quota retention policy to server config. (#2460)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 6c5f2f6 Add backlog quota retention policy to server config. (#2460)
6c5f2f6 is described below
commit 6c5f2f63defe077a59003f37e06430d243b769dc
Author: penghui
AuthorDate: Wed Aug 29 01:39:54 2018 +0800
Add backlog quota retention policy to server config. (#2460)
* Add backlog quota retention policy to server config.
* Rename backlogQuotaRetentionPolicy to backlogQuotaDefaultRetentionPolicy
* Change backlogQuotaDefaultRetentionPolicy String to enum.
---
conf/broker.conf | 6 ++
.../org/apache/pulsar/broker/ServiceConfiguration.java | 14 ++
.../apache/pulsar/broker/service/BacklogQuotaManager.java | 2 +-
3 files changed, 21 insertions(+), 1 deletion(-)
diff --git a/conf/broker.conf b/conf/broker.conf
index 5465c9a..a8dd44b 100644
--- a/conf/broker.conf
+++ b/conf/broker.conf
@@ -67,6 +67,12 @@ backlogQuotaCheckIntervalInSeconds=60
# Default per-topic backlog quota limit
backlogQuotaDefaultLimitGB=10
+# Default backlog quota retention policy. Default is producer_request_hold
+# 'producer_request_hold' Policy which holds producer's send request until the
resource becomes available (or holding times out)
+# 'producer_exception' Policy which throws
javax.jms.ResourceAllocationException to the producer
+# 'consumer_backlog_eviction' Policy which evicts the oldest message from the
slowest consumer's backlog
+backlogQuotaDefaultRetentionPolicy=producer_request_hold
+
# Enable the deletion of inactive topics
brokerDeleteInactiveTopicsEnabled=true
diff --git
a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
index 8ad0fc8..8057540 100644
---
a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
+++
b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
@@ -31,6 +31,7 @@ import org.apache.pulsar.common.configuration.FieldContext;
import org.apache.pulsar.common.configuration.PulsarConfiguration;
import com.google.common.collect.Sets;
+import org.apache.pulsar.common.policies.data.BacklogQuota;
/**
* Pulsar service configuration object.
@@ -89,6 +90,11 @@ public class ServiceConfiguration implements
PulsarConfiguration {
private int backlogQuotaCheckIntervalInSeconds = 60;
// Default per-topic backlog quota limit
private long backlogQuotaDefaultLimitGB = 50;
+//Default backlog quota retention policy. Default is producer_request_hold
+//'producer_request_hold' Policy which holds producer's send request until
the resource becomes available (or holding times out)
+//'producer_exception' Policy which throws
javax.jms.ResourceAllocationException to the producer
+//'consumer_backlog_eviction' Policy which evicts the oldest message from
the slowest consumer's backlog
+private BacklogQuota.RetentionPolicy backlogQuotaDefaultRetentionPolicy =
BacklogQuota.RetentionPolicy.producer_request_hold;
// Enable the deletion of inactive topics
private boolean brokerDeleteInactiveTopicsEnabled = true;
// How often to check for inactive topics
@@ -1729,4 +1735,12 @@ public class ServiceConfiguration implements
PulsarConfiguration {
public int getBrokerServiceCompactionMonitorIntervalInSeconds() {
return this.brokerServiceCompactionMonitorIntervalInSeconds;
}
+
+public BacklogQuota.RetentionPolicy
getBacklogQuotaDefaultRetentionPolicy() {
+return backlogQuotaDefaultRetentionPolicy;
+}
+
+public void
setBacklogQuotaDefaultRetentionPolicy(BacklogQuota.RetentionPolicy
backlogQuotaDefaultRetentionPolicy) {
+this.backlogQuotaDefaultRetentionPolicy =
backlogQuotaDefaultRetentionPolicy;
+}
}
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BacklogQuotaManager.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BacklogQuotaManager.java
index e7e38dd..7d607cd 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BacklogQuotaManager.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BacklogQuotaManager.java
@@ -50,7 +50,7 @@ public class BacklogQuotaManager {
public BacklogQuotaManager(PulsarService pulsar) {
this.defaultQuota = new BacklogQuota(
pulsar.getConfiguration().getBacklogQuotaDefaultLimitGB() *
1024 * 1024 * 1024,
-RetentionPolicy.producer_request_hold);
+
pulsar.getConfiguration().getBacklogQuotaDefaultRetentionPolicy());
this.zkCache = pulsar.getConfigurationCache().policiesCache();
}
[incubator-pulsar] branch master updated: Remove Aerospike Connector (#2456)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 83a3377 Remove Aerospike Connector (#2456)
83a3377 is described below
commit 83a3377e39cf17c2ebdf4c92f68eac26696356ee
Author: Ali Ahmed
AuthorDate: Tue Aug 28 00:10:52 2018 -0700
Remove Aerospike Connector (#2456)
---
distribution/io/src/assemble/io.xml| 6 -
pom.xml| 1 -
pulsar-io/aerospike/pom.xml| 75 -
.../pulsar/io/aerospike/AerospikeAbstractSink.java | 169 -
.../pulsar/io/aerospike/AerospikeSinkConfig.java | 64
.../pulsar/io/aerospike/AerospikeStringSink.java | 35 -
.../resources/META-INF/services/pulsar-io.yaml | 22 ---
pulsar-io/pom.xml | 1 -
site2/docs/deploy-bare-metal.md| 1 -
site2/docs/getting-started-standalone.md | 1 -
site2/docs/io-aerospike.md | 21 ---
site2/docs/io-connectors.md| 1 -
site2/docs/io-overview.md | 3 +-
site2/docs/io-quickstart.md| 3 +-
14 files changed, 2 insertions(+), 401 deletions(-)
diff --git a/distribution/io/src/assemble/io.xml
b/distribution/io/src/assemble/io.xml
index f69d5ce..28f9ad6 100644
--- a/distribution/io/src/assemble/io.xml
+++ b/distribution/io/src/assemble/io.xml
@@ -74,11 +74,5 @@
connectors
644
-
-
-
${basedir}/../../pulsar-io/aerospike/target/pulsar-io-aerospike-${project.version}.nar
- connectors
- 644
-
diff --git a/pom.xml b/pom.xml
index 66d371e..f7a93f1 100644
--- a/pom.xml
+++ b/pom.xml
@@ -161,7 +161,6 @@ flexible messaging model and an intuitive client
API.
0.8.3
2.2.0
3.4.0
-4.1.5
0.10.2.1
5.1.1
1.11.297
diff --git a/pulsar-io/aerospike/pom.xml b/pulsar-io/aerospike/pom.xml
deleted file mode 100644
index 5f678be..000
--- a/pulsar-io/aerospike/pom.xml
+++ /dev/null
@@ -1,75 +0,0 @@
-
-http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
- 4.0.0
-
-org.apache.pulsar
-pulsar-io
-2.2.0-incubating-SNAPSHOT
-
-
- pulsar-io-aerospike
- Pulsar IO :: Aerospike
-
-
-
-
- ${project.groupId}
- pulsar-io-core
- ${project.version}
-
-
-
- com.fasterxml.jackson.core
- jackson-databind
-
-
-
- com.fasterxml.jackson.dataformat
- jackson-dataformat-yaml
-
-
-
- com.aerospike
- aerospike-client
- ${aerospike-client.version}
-
-
- org.gnu
- gnu-crypto
-
-
-
-
-
-
-
-
-
-org.apache.nifi
-nifi-nar-maven-plugin
-
-
-
-
-
-
diff --git
a/pulsar-io/aerospike/src/main/java/org/apache/pulsar/io/aerospike/AerospikeAbstractSink.java
b/pulsar-io/aerospike/src/main/java/org/apache/pulsar/io/aerospike/AerospikeAbstractSink.java
deleted file mode 100644
index fe3787a..000
---
a/pulsar-io/aerospike/src/main/java/org/apache/pulsar/io/aerospike/AerospikeAbstractSink.java
+++ /dev/null
@@ -1,169 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.pulsar.io.aerospike;
-
-import com.aerospike.client.AerospikeClient;
-import com.aerospike.client.AerospikeException;
-import com.aerospike.client.Bin;
-import com.aerospike.client.Host;
-import com.aerospike.client.Key;
-import com.aerospike.client.Value;
-import com.aerospike.client.async.EventLoop;
-import com.aerospike.client.async.EventPolicy;
-import com.aerospike.client.async.NioEventLoops;
-import com.aerospike.client.listener.WriteListener;
-import com.aerospike.client.policy.ClientPolicy;
-import com.aerospike.client.policy.WritePolicy;
-
-import java.util.Map;
-import java.util.
[incubator-pulsar] branch master updated: Fix: Function assignment can support large number of topics (#2438)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new b283ebd Fix: Function assignment can support large number of topics
(#2438)
b283ebd is described below
commit b283ebd3e51aa86f6b9f0a6607f430996f0032f7
Author: Rajan Dhabalia
AuthorDate: Mon Aug 27 23:56:51 2018 -0700
Fix: Function assignment can support large number of topics (#2438)
---
conf/functions_worker.yml | 1 +
.../org/apache/pulsar/io/PulsarSinkE2ETest.java| 2 +
.../functions/worker/FunctionAssignmentTailer.java | 70 --
.../functions/worker/FunctionRuntimeManager.java | 67 -
.../pulsar/functions/worker/MembershipManager.java | 2 +
.../pulsar/functions/worker/SchedulerManager.java | 46 ++
.../org/apache/pulsar/functions/worker/Worker.java | 59 ++
.../pulsar/functions/worker/WorkerConfig.java | 3 +-
.../worker/FunctionRuntimeManagerTest.java | 33 +-
.../functions/worker/MembershipManagerTest.java| 3 +-
.../functions/worker/SchedulerManagerTest.java | 8 +++
11 files changed, 227 insertions(+), 67 deletions(-)
diff --git a/conf/functions_worker.yml b/conf/functions_worker.yml
index 444b7fb..497c414 100644
--- a/conf/functions_worker.yml
+++ b/conf/functions_worker.yml
@@ -27,6 +27,7 @@ connectorsDirectory: ./connectors
functionMetadataTopicName: metadata
clusterCoordinationTopicName: coordinate
pulsarFunctionsNamespace: public/functions
+pulsarAssignmentNamespace: public/assignment
pulsarFunctionsCluster: standalone
pulsarServiceUrl: pulsar://localhost:6650
pulsarWebServiceUrl: http://localhost:8080
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarSinkE2ETest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarSinkE2ETest.java
index dd39222..54c14c0 100644
--- a/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarSinkE2ETest.java
+++ b/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarSinkE2ETest.java
@@ -105,6 +105,7 @@ public class PulsarSinkE2ETest {
WorkerService functionsWorkerService;
final String tenant = "external-repl-prop";
String pulsarFunctionsNamespace = tenant + "/use/pulsar-function-admin";
+String pulsarAssignmentNamespace = tenant + "/use/pulsar-assignment";
String primaryHost;
String workerId;
@@ -212,6 +213,7 @@ public class PulsarSinkE2ETest {
private WorkerService createPulsarFunctionWorker(ServiceConfiguration
config) {
workerConfig = new WorkerConfig();
workerConfig.setPulsarFunctionsNamespace(pulsarFunctionsNamespace);
+workerConfig.setPulsarAssignmentNamespace(pulsarAssignmentNamespace);
workerConfig.setSchedulerClassName(
org.apache.pulsar.functions.worker.scheduler.RoundRobinScheduler.class.getName());
workerConfig.setThreadContainerFactory(new
WorkerConfig.ThreadContainerFactory().setThreadGroupName("use"));
diff --git
a/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionAssignmentTailer.java
b/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionAssignmentTailer.java
index 366eaba..060120b 100644
---
a/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionAssignmentTailer.java
+++
b/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionAssignmentTailer.java
@@ -20,11 +20,15 @@ package org.apache.pulsar.functions.worker;
import lombok.extern.slf4j.Slf4j;
import org.apache.pulsar.client.api.Message;
+import org.apache.pulsar.client.api.MessageId;
import org.apache.pulsar.client.api.PulsarClientException;
import org.apache.pulsar.client.api.Reader;
import org.apache.pulsar.functions.proto.Request;
+import com.google.common.collect.Lists;
+
import java.io.IOException;
+import java.util.List;
import java.util.function.Function;
@Slf4j
@@ -33,13 +37,22 @@ public class FunctionAssignmentTailer
private final FunctionRuntimeManager functionRuntimeManager;
private final Reader reader;
+
+private long currentVersion = 0;
+private final List
currentVersionAssignments;
+private volatile MessageId previousOldAssignmentMsgId = null;
public FunctionAssignmentTailer(FunctionRuntimeManager
functionRuntimeManager,
Reader reader)
throws PulsarClientException {
-this.functionRuntimeManager = functionRuntimeManager;
-this.reader = reader;
+this.functionRuntimeManager = functionRuntimeManager;
+this.reader = reader;
+this.currentVersionAssignments = Lists.newArrayList();
+// complete init if reader has no message to read so,
sc
[incubator-pulsar] branch branch-2.1 updated: Fix compilation issue on CLITest (#2454)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new 73cc537 Fix compilation issue on CLITest (#2454)
73cc537 is described below
commit 73cc53757205f81eff9a43535bf26eefb93a490b
Author: Sijie Guo
AuthorDate: Mon Aug 27 23:56:13 2018 -0700
Fix compilation issue on CLITest (#2454)
### Motivation
#2182 is cherry-picked into branch 2.1. But some of integration test
changes are not cherry-picked back.
So the compilation is broken
### Changes
Fix the compilation issue on CLITest
---
.../test/java/org/apache/pulsar/tests/integration/cli/CLITest.java| 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git
a/tests/integration/semantics/src/test/java/org/apache/pulsar/tests/integration/cli/CLITest.java
b/tests/integration/semantics/src/test/java/org/apache/pulsar/tests/integration/cli/CLITest.java
index 0f2163a..d2cf6cf 100644
---
a/tests/integration/semantics/src/test/java/org/apache/pulsar/tests/integration/cli/CLITest.java
+++
b/tests/integration/semantics/src/test/java/org/apache/pulsar/tests/integration/cli/CLITest.java
@@ -167,7 +167,7 @@ public class CLITest extends PulsarClusterTestBase {
@Test
public void testSetInfiniteRetention() throws Exception {
-ContainerExecResult result;
+ExecResult result;
String namespace = "get-and-set-retention" + randomName(8);
pulsarCluster.createNamespace(namespace);
@@ -179,7 +179,6 @@ public class CLITest extends PulsarClusterTestBase {
};
result = pulsarCluster.runAdminCommandOnAnyBroker(setCommand);
-assertEquals(0, result.getExitCode());
assertTrue(
result.getStdout().isEmpty(),
result.getStdout()
@@ -194,7 +193,6 @@ public class CLITest extends PulsarClusterTestBase {
};
result = pulsarCluster.runAdminCommandOnAnyBroker(getCommand);
-assertEquals(0, result.getExitCode());
assertTrue(
result.getStdout().contains("\"retentionTimeInMinutes\" : -1"),
result.getStdout());
[incubator-pulsar.wiki] branch master updated: Updated Apache Maturity Model Assessment for Pulsar (markdown)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new 6493707 Updated Apache Maturity Model Assessment for Pulsar (markdown) 6493707 is described below commit 649370722c74b70e6a3705e988eee4490cd1a071 Author: Matteo Merli AuthorDate: Mon Aug 27 23:34:56 2018 -0700 Updated Apache Maturity Model Assessment for Pulsar (markdown) --- Apache-Maturity-Model-Assessment-for-Pulsar.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/Apache-Maturity-Model-Assessment-for-Pulsar.md b/Apache-Maturity-Model-Assessment-for-Pulsar.md index 053257e..7657037 100644 --- a/Apache-Maturity-Model-Assessment-for-Pulsar.md +++ b/Apache-Maturity-Model-Assessment-for-Pulsar.md @@ -1,3 +1,4 @@ + ## Overview This is an assessment of the Pulsar podling’s maturity, meant to help inform the decision (of the mentors, community, Incubator PMC and ASF Board of Directors) to graduate it as a top-level Apache project. @@ -19,9 +20,9 @@ Mentors and community members are encouraged to contribute to this page and comm | ***Code*** | | | | CD10 | The project produces Open Source software, for distribution to the public at no charge. | **YES** The project source code is licensed under the Apache License, version 2.0. | | CD20 | The project's code is easily discoverable and publicly accessible. | **YES** Linked from the website, available via GitBox https://gitbox.apache.org/repos/asf?p=incubator-pulsar.git and https://github.com/apache/incubator-pulsar. | -| CD30 | The code can be built in a reproducible way using widely available standard tools. | **YES** The build uses Apache Maven for Java code and CMake for C++ code. Continuous integration is used | +| CD30 | The code can be built in a reproducible way using widely available standard tools. | **YES** The build uses Apache Maven for Java code and CMake for C++ code. Continuous integration is used to automate the testing and validation of new commits. | | CD40 | The full history of the project's code is available via a source code control system, in a way that allows any released version to be recreated. | **YES** All the history of the project is available through Git. All releases are properly tagged. | -| CD50 | The provenance of each line of code is established via the source code control system, in a reliable way based on strong authentication of the committer. When third-party contributions are committed, commit messages provide reliable information about the code provenance. | **YES** The git repository is managed by Apache Infra. Only Pulsar committers have write access. All code is checked in after at least 2 committers have approved a pull-request. For 3rd party contribution [...] +| CD50 | The provenance of each line of code is established via the source code control system, in a reliable way based on strong authentication of the committer. When third-party contributions are committed, commit messages provide reliable information about the code provenance. | **YES** The git repository is managed by Apache Infra. Only Pulsar committers have write access. All code is checked in after at least 2 committers have reviewed and approved a pull-request. For 3rd party [...] | ***Licenses and Copyright*** | | | | LC10 | The code is released under the Apache License, version 2.0. | **YES** Source distributions clearly state license. Convenience binaries clearly state license. | | LC20 | Libraries that are mandatory dependencies of the project's code do not create more restrictions than the Apache License does. | **YES** The list of mandatory dependencies have been reviewed to contain approved licenses only.|
[incubator-pulsar.wiki] branch master updated: Updated Apache Maturity Model Assessment for Pulsar (markdown)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new 9d0829b Updated Apache Maturity Model Assessment for Pulsar (markdown) 9d0829b is described below commit 9d0829b764a5b19c7162c8d51c8257c38310ec39 Author: Matteo Merli AuthorDate: Mon Aug 27 23:29:57 2018 -0700 Updated Apache Maturity Model Assessment for Pulsar (markdown) --- Apache-Maturity-Model-Assessment-for-Pulsar.md | 16 1 file changed, 16 insertions(+) diff --git a/Apache-Maturity-Model-Assessment-for-Pulsar.md b/Apache-Maturity-Model-Assessment-for-Pulsar.md index 2074b03..053257e 100644 --- a/Apache-Maturity-Model-Assessment-for-Pulsar.md +++ b/Apache-Maturity-Model-Assessment-for-Pulsar.md @@ -1,3 +1,19 @@ +## Overview + +This is an assessment of the Pulsar podling’s maturity, meant to help inform the decision (of the mentors, community, Incubator PMC and ASF Board of Directors) to graduate it as a top-level Apache project. + +It is based on the ASF project maturity model at https://community.apache.org/apache-way/apache-project-maturity-model.html + + +## Status of this document + +All open items are updated with the latest status. + +## Maturity model assessment + +Mentors and community members are encouraged to contribute to this page and comment on it, the following table summarizes project’s self-assessment against the Apache Maturity Model. + + | ID | Description | Status | |:---|:|:---| | ***Code*** | | |
[incubator-pulsar] branch master updated: Fixed cpp multi-topic consumer when topics are not partitioned (#2453)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new ecc34a9 Fixed cpp multi-topic consumer when topics are not
partitioned (#2453)
ecc34a9 is described below
commit ecc34a943bb005712ad2492cda6a1d3d8674255f
Author: Matteo Merli
AuthorDate: Mon Aug 27 23:28:55 2018 -0700
Fixed cpp multi-topic consumer when topics are not partitioned (#2453)
* Fixed cpp multi-topic consumer when topics are not partitioned
* Fixed formatting
---
pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc | 27 ++--
pulsar-client-cpp/tests/BasicEndToEndTest.cc | 25 +-
2 files changed, 40 insertions(+), 12 deletions(-)
diff --git a/pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc
b/pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc
index 6750273..69c3cc0 100644
--- a/pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc
+++ b/pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc
@@ -175,17 +175,30 @@ void
MultiTopicsConsumerImpl::subscribeTopicPartitions(const Result result,
boost::shared_ptr> partitionsNeedCreate =
boost::make_shared>(numPartitions);
-for (int i = 0; i < numPartitions; i++) {
-std::string topicPartitionName = topicName->getTopicPartitionName(i);
-consumer = boost::make_shared(client_,
topicPartitionName, subscriptionName_, config,
-internalListenerExecutor,
Partitioned);
+if (numPartitions == 1) {
+// We don't have to add partition-n suffix
+consumer = boost::make_shared(client_,
topicName->toString(), subscriptionName_, config,
+internalListenerExecutor,
NonPartitioned);
consumer->getConsumerCreatedFuture().addListener(
boost::bind(::handleSingleConsumerCreated,
shared_from_this(), _1, _2,
partitionsNeedCreate, topicSubResultPromise));
-consumer->setPartitionIndex(i);
-consumers_.insert(std::make_pair(topicPartitionName, consumer));
-LOG_DEBUG("Create Consumer for - " << topicPartitionName << " - " <<
consumerStr_);
+consumers_.insert(std::make_pair(topicName->toString(), consumer));
+LOG_DEBUG("Creating Consumer for - " << topicName << " - " <<
consumerStr_);
consumer->start();
+
+} else {
+for (int i = 0; i < numPartitions; i++) {
+std::string topicPartitionName =
topicName->getTopicPartitionName(i);
+consumer = boost::make_shared(client_,
topicPartitionName, subscriptionName_,
+config,
internalListenerExecutor, Partitioned);
+consumer->getConsumerCreatedFuture().addListener(
+
boost::bind(::handleSingleConsumerCreated,
shared_from_this(), _1, _2,
+partitionsNeedCreate, topicSubResultPromise));
+consumer->setPartitionIndex(i);
+consumers_.insert(std::make_pair(topicPartitionName, consumer));
+LOG_DEBUG("Creating Consumer for - " << topicPartitionName << " -
" << consumerStr_);
+consumer->start();
+}
}
}
diff --git a/pulsar-client-cpp/tests/BasicEndToEndTest.cc
b/pulsar-client-cpp/tests/BasicEndToEndTest.cc
index e87850c..73430fc 100644
--- a/pulsar-client-cpp/tests/BasicEndToEndTest.cc
+++ b/pulsar-client-cpp/tests/BasicEndToEndTest.cc
@@ -1604,10 +1604,12 @@ TEST(BasicEndToEndTest, testMultiTopicsConsumerPubSub) {
std::string topicName1 =
"persistent://prop/unit/ns/testMultiTopicsConsumer1";
std::string topicName2 =
"persistent://prop/unit/ns/testMultiTopicsConsumer2";
std::string topicName3 =
"persistent://prop/unit/ns/testMultiTopicsConsumer3";
+std::string topicName4 =
"persistent://prop/unit/ns/testMultiTopicsConsumer4";
topicNames.push_back(topicName1);
topicNames.push_back(topicName2);
topicNames.push_back(topicName3);
+topicNames.push_back(topicName4);
// call admin api to make topics partitioned
std::string url1 = adminUrl +
"admin/persistent/prop/unit/ns/testMultiTopicsConsumer1/partitions";
@@ -1631,7 +1633,11 @@ TEST(BasicEndToEndTest, testMultiTopicsConsumerPubSub) {
result = client.createProducer(topicName3, producer3);
ASSERT_EQ(ResultOk, result);
-LOG_INFO("created 3 producers");
+Producer producer4;
+result = client.createProducer(topicName4, producer4);
+ASSERT_EQ(ResultOk, result);
+
+LOG_INFO("created 4 producers");
int messa
[incubator-pulsar.wiki] branch master updated: Created Apache Maturity Model Assessment for Pulsar (markdown)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.wiki.git The following commit(s) were added to refs/heads/master by this push: new b4cacd5 Created Apache Maturity Model Assessment for Pulsar (markdown) b4cacd5 is described below commit b4cacd5edd84de42d9b6c89bf6155f27d258b56f Author: Matteo Merli AuthorDate: Mon Aug 27 18:11:05 2018 -0700 Created Apache Maturity Model Assessment for Pulsar (markdown) --- Apache-Maturity-Model-Assessment-for-Pulsar.md | 40 ++ 1 file changed, 40 insertions(+) diff --git a/Apache-Maturity-Model-Assessment-for-Pulsar.md b/Apache-Maturity-Model-Assessment-for-Pulsar.md new file mode 100644 index 000..2074b03 --- /dev/null +++ b/Apache-Maturity-Model-Assessment-for-Pulsar.md @@ -0,0 +1,40 @@ +| ID | Description | Status | +|:---|:|:---| +| ***Code*** | | | +| CD10 | The project produces Open Source software, for distribution to the public at no charge. | **YES** The project source code is licensed under the Apache License, version 2.0. | +| CD20 | The project's code is easily discoverable and publicly accessible. | **YES** Linked from the website, available via GitBox https://gitbox.apache.org/repos/asf?p=incubator-pulsar.git and https://github.com/apache/incubator-pulsar. | +| CD30 | The code can be built in a reproducible way using widely available standard tools. | **YES** The build uses Apache Maven for Java code and CMake for C++ code. Continuous integration is used | +| CD40 | The full history of the project's code is available via a source code control system, in a way that allows any released version to be recreated. | **YES** All the history of the project is available through Git. All releases are properly tagged. | +| CD50 | The provenance of each line of code is established via the source code control system, in a reliable way based on strong authentication of the committer. When third-party contributions are committed, commit messages provide reliable information about the code provenance. | **YES** The git repository is managed by Apache Infra. Only Pulsar committers have write access. All code is checked in after at least 2 committers have approved a pull-request. For 3rd party contribution [...] +| ***Licenses and Copyright*** | | | +| LC10 | The code is released under the Apache License, version 2.0. | **YES** Source distributions clearly state license. Convenience binaries clearly state license. | +| LC20 | Libraries that are mandatory dependencies of the project's code do not create more restrictions than the Apache License does. | **YES** The list of mandatory dependencies have been reviewed to contain approved licenses only.| +| LC30 | The libraries mentioned in LC20 are available as Open Source software. | **YES** All dependencies are available as open source software. | +| LC40 | Committers are bound by an Individual Contributor Agreement (the "Apache iCLA") that defines which code they are allowed to commit and how they need to identify code that is not their own. | **YES** The project uses a repository managed by Apache Infra write access requires an Apache account, which requires an ICLA on file. | +| LC50 | The copyright ownership of everything that the project produces is clearly defined and documented. | **YES** Software Grant Agreement for the initial donation from Yahoo was filed. All files in the source repository have appropriate headers. Automated process is in place to ensure every file has expected headers. | +| ***Releases*** | | | +| RE10 | Releases consist of source code, distributed using standard and open archive formats that are expected to stay readable in the long term. | **YES** Source releases are distributed via https://dist.apache.org/repos/dist/release/incubator/pulsar/ and linked from the website at https://pulsar.incubator.apache.org/download/. +| RE20 | Releases are approved by the project's PMC (see CS10), in order to make them an act of the Foundation.| **YES** All incubating releases have been approved by the Pulsar community with at least 3 PPMC votes and from the Incubator with 3 IPMC votes. | +| RE30 | Releases are signed and/or distributed along with digests that can be reliably used to validate the downloaded archives. | **YES** All releases are signed, and the KEYS file is provided on dist.apache.org. | +| RE40 | Convenience binaries can be distributed alongside source code but they are not Apache Releases -- they are just a convenience provided with no guarantee. | **YES** Convenience binaries are distributed via via dist.apache.org. Java binary artifacts are also distributed through Maven Central Repository. | +| RE50 | The release process is documented and repeatable to the extent that s
[incubator-pulsar] branch asf-site updated: Updated site at revision a16b64a
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 26c98e7 Updated site at revision a16b64a
26c98e7 is described below
commit 26c98e72aa096ead5d53233ef2ea9e9d5032998f
Author: jenkins
AuthorDate: Sun Aug 26 08:38:26 2018 +
Updated site at revision a16b64a
---
content/api/pulsar-functions/index-all.html| 4 ++
.../org/apache/pulsar/functions/api/Record.html| 32 ---
content/docs/zh-CN/client-libraries-cpp.html | 2 +-
content/docs/zh-CN/client-libraries-cpp/index.html | 2 +-
content/docs/zh-CN/client-libraries-go.html| 8 ++--
content/docs/zh-CN/client-libraries-go/index.html | 8 ++--
content/docs/zh-CN/client-libraries-java.html | 38 +-
.../docs/zh-CN/client-libraries-java/index.html| 38 +-
content/docs/zh-CN/next/client-libraries-cpp.html | 2 +-
.../zh-CN/next/client-libraries-cpp/index.html | 2 +-
content/docs/zh-CN/next/client-libraries-go.html | 8 ++--
.../docs/zh-CN/next/client-libraries-go/index.html | 8 ++--
content/docs/zh-CN/next/client-libraries-java.html | 38 +-
.../zh-CN/next/client-libraries-java/index.html| 38 +-
content/docs/zh-CN/next/security-extending.html| 2 +-
.../docs/zh-CN/next/security-extending/index.html | 2 +-
content/docs/zh-CN/security-extending.html | 2 +-
content/docs/zh-CN/security-extending/index.html | 2 +-
.../docs/zh-CN/security-tls-authentication.html| 4 +-
.../zh-CN/security-tls-authentication/index.html | 4 +-
content/docs/zh-CN/security-tls-transport.html | 4 +-
.../docs/zh-CN/security-tls-transport/index.html | 4 +-
content/swagger/swagger.json | 46 +++---
23 files changed, 161 insertions(+), 137 deletions(-)
diff --git a/content/api/pulsar-functions/index-all.html
b/content/api/pulsar-functions/index-all.html
index 9377c11..d40daca 100644
--- a/content/api/pulsar-functions/index-all.html
+++ b/content/api/pulsar-functions/index-all.html
@@ -126,6 +126,10 @@
Access the record associated with the current input
value
+getEventTime()
- Method in interface org.apache.pulsar.functions.api.Record
+
+Retrieves the event time of the record from the
source.
+
getFunctionId()
- Method in interface org.apache.pulsar.functions.api.Context
The id of the function that we are executing
diff --git
a/content/api/pulsar-functions/org/apache/pulsar/functions/api/Record.html
b/content/api/pulsar-functions/org/apache/pulsar/functions/api/Record.html
index 93bc5f3..638db51 100644
--- a/content/api/pulsar-functions/org/apache/pulsar/functions/api/Record.html
+++ b/content/api/pulsar-functions/org/apache/pulsar/functions/api/Record.html
@@ -17,7 +17,7 @@
catch(err) {
}
//-->
-var methods = {"i0":18,"i1":18,"i2":6,"i3":18,"i4":18,"i5":18,"i6":18,"i7":6};
+var methods =
{"i0":18,"i1":18,"i2":18,"i3":6,"i4":18,"i5":18,"i6":18,"i7":18,"i8":6};
var tabs = {65535:["t0","All Methods"],2:["t2","Instance
Methods"],4:["t3","Abstract Methods"],16:["t5","Default Methods"]};
var altColor = "altColor";
var rowColor = "rowColor";
@@ -132,36 +132,42 @@ var activeTableTab = "activeTableTab";
+default OptionalLong
+getEventTime()
+Retrieves the event time of the record from the
source.
+
+
+
OptionalString
getKey()
Return a key if the key has one associated
-
+
default OptionalString
getPartitionId()
Retrieves the partition information if any of the
record.
-
+
default MapString,String
getProperties()
Retrieves user-defined properties attached to record.
-
+
default OptionalLong
getRecordSequence()
Retrieves the sequence of the record from a source
partition.
-
+
default OptionalString
getTopicName()
If the record originated from a topic, report the topic
name
-
+
T
getValue()
Retrieves the actual data of the record
@@ -216,6 +222,20 @@ var activeTableTab = "activeTableTab";
+
+
+
+
+
+getEventTime
+defaultOptionalLonggetEventTime()
+Retrieves the event time of the record from the
source.
+
+Returns:
+millis since epoch
+
+
+
diff --git a/content/docs/zh-CN/client-libraries-cpp.html
b/content/docs/zh-CN/client-libraries-cpp.html
index a7b5de7..6f2709e 100644
--- a/content/docs/zh-CN/client-libraries-cpp.html
+++ b/content/docs/zh-CN/client-libraries-cpp.html
@@ -126,7 +126,7 @@
Pulsar protocol URLs are assigned to specific clusters, use the pulsar
scheme and have a default port of 66
[incubator-pulsar] branch asf-site updated: Updated site at revision f44d367
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 9e38436 Updated site at revision f44d367
9e38436 is described below
commit 9e38436de54abf4f2d943db371088440eddafe9e
Author: jenkins
AuthorDate: Sat Aug 25 08:36:43 2018 +
Updated site at revision f44d367
---
content/api/admin/allclasses-frame.html| 2 +-
content/api/admin/allclasses-noframe.html | 2 +-
content/api/admin/index-all.html | 20 +-
.../org/apache/pulsar/client/admin/Functions.html | 52 +---
.../apache/pulsar/client/admin/PulsarAdmin.html| 10 +-
.../org/apache/pulsar/client/admin/Topics.html | 4 +-
.../org/apache/pulsar/client/admin/Worker.html | 316 +
.../apache/pulsar/client/admin/package-frame.html | 2 +-
.../pulsar/client/admin/package-summary.html | 2 +-
.../apache/pulsar/client/admin/package-tree.html | 2 +-
content/api/admin/overview-tree.html | 2 +-
content/swagger/swagger.json | 12 +-
12 files changed, 364 insertions(+), 62 deletions(-)
diff --git a/content/api/admin/allclasses-frame.html
b/content/api/admin/allclasses-frame.html
index 35ece94..14f1a30 100644
--- a/content/api/admin/allclasses-frame.html
+++ b/content/api/admin/allclasses-frame.html
@@ -87,7 +87,7 @@
Tenants
Topics
TopicStats
-WorkerStats
+Worker
diff --git a/content/api/admin/allclasses-noframe.html
b/content/api/admin/allclasses-noframe.html
index 85ff8bf..516727f 100644
--- a/content/api/admin/allclasses-noframe.html
+++ b/content/api/admin/allclasses-noframe.html
@@ -87,7 +87,7 @@
Tenants
Topics
TopicStats
-WorkerStats
+Worker
diff --git a/content/api/admin/index-all.html b/content/api/admin/index-all.html
index bd4623d..9c80959 100644
--- a/content/api/admin/index-all.html
+++ b/content/api/admin/index-all.html
@@ -707,6 +707,10 @@
Get all namespaces that grouped with given anti-affinity
group
+getAssignments()
- Method in interface org.apache.pulsar.client.admin.Worker
+
+Get the function assignment among the cluster
+
getAvailablePrimaryBrokers(SortedSetBrokerStatus)
- Method in class org.apache.pulsar.common.policies.impl.NamespaceIsolationPolicyImpl
getAvailablePrimaryBrokers(SortedSetBrokerStatus)
- Method in interface org.apache.pulsar.common.policies.NamespaceIsolationPolicy
@@ -773,9 +777,13 @@
Get the configuration data for the specified cluster.
-getCluster()
- Method in interface org.apache.pulsar.client.admin.Functions
+getCluster()
- Method in interface org.apache.pulsar.client.admin.Worker
+
+Get List of all workers belonging to this cluster
+
+getClusterLeader()
- Method in interface org.apache.pulsar.client.admin.Worker
-Get list of workers present under a cluster
+Get the worker who is the leader of the cluster
getClusters()
- Method in interface org.apache.pulsar.client.admin.Clusters
@@ -825,7 +833,7 @@
Get the list of functions.
-getFunctionsStats()
- Method in interface org.apache.pulsar.client.admin.WorkerStats
+getFunctionsStats()
- Method in interface org.apache.pulsar.client.admin.Worker
Get all functions stats on a worker
@@ -931,7 +939,7 @@
Returns Monitoring metrics
-getMetrics()
- Method in interface org.apache.pulsar.client.admin.WorkerStats
+getMetrics()
- Method in interface org.apache.pulsar.client.admin.Worker
Get worker metrics.
@@ -2394,9 +2402,9 @@ the order they are declared.
waitingReadOp
- Variable in class org.apache.pulsar.common.policies.data.PersistentTopicInternalStats.CursorStats
-workerStats()
- Method in class org.apache.pulsar.client.admin.PulsarAdmin
+worker()
- Method in class org.apache.pulsar.client.admin.PulsarAdmin
-WorkerStats - Interface in org.apache.pulsar.client.admin
+Worker -
Interface in org.apache.pulsar.client.admin
Admin interface for worker stats management.
diff --git a/content/api/admin/org/apache/pulsar/client/admin/Functions.html
b/content/api/admin/org/apache/pulsar/client/admin/Functions.html
index 040c109..8d9f973 100644
--- a/content/api/admin/org/apache/pulsar/client/admin/Functions.html
+++ b/content/api/admin/org/apache/pulsar/client/admin/Functions.html
@@ -17,7 +17,7 @@
catch(err) {
}
//-->
-var methods =
{"i0":6,"i1":6,"i2":6,"i3":6,"i4":6,"i5":6,"i6":6,"i7":6,"i8":6,"i9":6,"i10":6,"i11":6,"i12":6,"i13":6,"i14":6,"i15":6,"i16":6,"i17":6,"i18":6};
+var methods =
{"i0":6,"i1":6,"i2":6,"i3":6,"i4":6,"i5":6,"i6":6,"i7":6,"i8":6,&qu
[incubator-pulsar] branch master updated: fix issue with protocol version not being correct (#2444)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new d4b214c fix issue with protocol version not being correct (#2444)
d4b214c is described below
commit d4b214cc0f5266b0fd38c016af1e611d98bad3ed
Author: Boyang Jerry Peng
AuthorDate: Fri Aug 24 22:40:58 2018 -0700
fix issue with protocol version not being correct (#2444)
---
.../src/main/java/org/apache/pulsar/client/impl/ClientCnx.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ClientCnx.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ClientCnx.java
index 02443cc..7287194 100644
--- a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ClientCnx.java
+++ b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ClientCnx.java
@@ -243,8 +243,9 @@ public class ClientCnx extends PulsarHandler {
if (log.isDebugEnabled()) {
log.debug("{} Connection is ready", ctx.channel());
}
-connectionFuture.complete(null);
+// set remote protocol version to the correct version before we
complete the connection future
remoteEndpointProtocolVersion = connected.getProtocolVersion();
+connectionFuture.complete(null);
state = State.Ready;
}
[incubator-pulsar] branch asf-site updated: Updated site at revision 31ebf61
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/asf-site by this push: new 776cb1c Updated site at revision 31ebf61 776cb1c is described below commit 776cb1cbbafa115a3070ceca588616a08a29521b Author: jenkins AuthorDate: Fri Aug 24 23:19:13 2018 + Updated site at revision 31ebf61 --- .../docs/latest/adaptors/PulsarSpark/index.html| 8 ++-- .../docs/latest/adaptors/PulsarStorm/index.html| 6 +-- content/docs/latest/admin-api/overview/index.html | 10 ++--- content/docs/latest/clients/Cpp/index.html | 8 ++-- content/docs/latest/clients/Java/index.html| 14 +++ content/docs/latest/clients/Python/index.html | 10 ++--- content/docs/latest/clients/WebSocket/index.html | 8 ++-- content/docs/latest/clients/go/index.html | 6 +-- .../docs/latest/cookbooks/Encryption/index.html| 6 +-- .../latest/cookbooks/PartitionedTopics/index.html | 14 +++ .../latest/cookbooks/RetentionExpiry/index.html| 14 +++ .../docs/latest/cookbooks/compaction/index.html| 6 +-- .../cookbooks/message-deduplication/index.html | 10 ++--- .../docs/latest/cookbooks/message-queue/index.html | 14 +++ .../latest/cookbooks/tiered-storage/index.html | 4 +- .../docs/latest/deployment/Kubernetes/index.html | 4 +- .../docs/latest/deployment/aws-cluster/index.html | 4 +- content/docs/latest/deployment/cluster/index.html | 4 +- content/docs/latest/deployment/instance/index.html | 4 +- .../ConceptsAndArchitecture/index.html | 4 +- .../latest/getting-started/LocalCluster/index.html | 4 +- .../latest/getting-started/Pulsar-2.0/index.html | 2 +- .../docs/latest/getting-started/docker/index.html | 4 +- .../docs/latest/project/BinaryProtocol/index.html | 4 +- content/docs/latest/project/CompileCpp/index.html | 8 ++-- .../docs/latest/project/SimulationTools/index.html | 2 +- .../docs/latest/project/schema-storage/index.html | 4 +- content/docs/latest/reference/CliTools/index.html | 18 .../docs/latest/security/authorization/index.html | 10 ++--- content/docs/latest/security/encryption/index.html | 6 +-- content/en/team.html | 2 +- content/en/team/index.html | 2 +- content/fr/team.html | 2 +- content/fr/team/index.html | 2 +- content/ja/adaptors/PulsarSpark/index.html | 8 ++-- content/ja/adaptors/PulsarStorm/index.html | 6 +-- content/ja/admin/AdminInterface/index.html | 12 +++--- content/ja/admin/Authz/index.html | 12 +++--- content/ja/admin/ClustersBrokers/index.html| 6 +-- content/ja/admin/PropertiesNamespaces/index.html | 6 +-- content/ja/advanced/PartitionedTopics/index.html | 12 +++--- content/ja/advanced/RetentionExpiry/index.html | 12 +++--- content/ja/clients/Cpp/index.html | 6 +-- content/ja/clients/Java/index.html | 8 ++-- content/ja/clients/Python/index.html | 8 ++-- content/ja/clients/WebSocket/index.html| 8 ++-- content/ja/deployment/InstanceSetup/index.html | 6 +-- content/ja/deployment/Kubernetes/index.html| 4 +- .../ConceptsAndArchitecture/index.html | 2 +- content/ja/getting-started/LocalCluster/index.html | 4 +- content/ja/project/BinaryProtocol/index.html | 4 +- content/ja/project/SimulationTools/index.html | 2 +- content/ja/reference/CliTools/index.html | 18 content/ja/team.html | 2 +- content/ja/team/index.html | 2 +- content/swagger/swagger.json | 48 +++--- content/team.html | 2 +- content/team/index.html| 2 +- content/zh-CN/team.html| 2 +- content/zh-CN/team/index.html | 2 +- 60 files changed, 216 insertions(+), 216 deletions(-) diff --git a/content/docs/latest/adaptors/PulsarSpark/index.html b/content/docs/latest/adaptors/PulsarSpark/index.html index d130108..a30d928 100644 --- a/content/docs/latest/adaptors/PulsarSpark/index.html +++ b/content/docs/latest/adaptors/PulsarSpark/index.html @@ -1079,9 +1079,9 @@ - Spark Streaming Pulsar receiver + Spark Streaming Pulsar receiver @@ -1325,9 +1325,9 @@ - Spark Streaming Pulsar receiver + Spark Streaming Pulsar receiver @@ -1537,6 +1537,8
[incubator-pulsar] branch master updated: Added missing committers from "team" page (#2443)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 31ebf61 Added missing committers from "team" page (#2443)
31ebf61 is described below
commit 31ebf61d593d798da25024032f55af6969fc272b
Author: Matteo Merli
AuthorDate: Fri Aug 24 15:37:53 2018 -0700
Added missing committers from "team" page (#2443)
---
site2/website/data/team.js | 32 +++-
1 file changed, 31 insertions(+), 1 deletion(-)
diff --git a/site2/website/data/team.js b/site2/website/data/team.js
index 6c0cc81..1877166 100644
--- a/site2/website/data/team.js
+++ b/site2/website/data/team.js
@@ -1,6 +1,12 @@
module.exports = {
committers: [
{
+ name: 'Boyang Jerry Peng',
+ apacheId: 'jerrypeng',
+ org: 'Streamlio',
+ roles: 'Committer, PPMC'
+},
+{
name: 'Brad McMillen',
apacheId: 'bradtm',
org: 'Yahoo',
@@ -13,12 +19,24 @@ module.exports = {
roles: 'Committer, PPMC'
},
{
+ name: 'Ivan Brendan Kelly',
+ apacheId: 'ivank',
+ org: 'Streamlio',
+ roles: 'Committer, PPMC'
+},
+{
name: 'Jai Asher',
apacheId: 'jai1',
org: 'Yahoo',
roles: 'Committer, PPMC'
},
{
+ name: 'Jia Zhai',
+ apacheId: 'zhaijia',
+ org: 'Streamlio',
+ roles: 'Committer, PPMC'
+},
+{
name: 'Joe Francis',
apacheId: 'joef',
org: 'Yahoo',
@@ -27,7 +45,7 @@ module.exports = {
{
name: 'Ludwig Pummer',
apacheId: 'ludwigp',
- org: 'Yahoo',
+ org: 'Yahoo',
roles: 'Committer, PPMC'
},
{
@@ -67,6 +85,12 @@ module.exports = {
roles: 'Committer, PPMC'
},
{
+ name: 'Sanjeev Kulkarni',
+ apacheId: 'sanjeevrk',
+ org: 'Streamlio',
+ roles: 'Committer, PPMC'
+},
+{
name: 'Sebastián Schepens',
apacheId: 'sschepens',
org: 'MercadoLibre',
@@ -79,6 +103,12 @@ module.exports = {
roles: 'Committer, PPMC'
},
{
+ name: 'Sijie Guo',
+ apacheId: 'sijie',
+ org: 'Streamlio',
+ roles: 'Committer, PPMC'
+},
+{
name: 'Yuki Shiga',
apacheId: 'yushiga',
org: 'Yahoo Japan Corporation',
[incubator-pulsar] branch master updated: Standalone bookie should always advertise 'localhost' when running in docker container (#2425)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 30fee6b Standalone bookie should always advertise 'localhost' when
running in docker container (#2425)
30fee6b is described below
commit 30fee6bb2f83e344dd2e86b675671762a9242f26
Author: Matteo Merli
AuthorDate: Fri Aug 24 12:55:12 2018 -0700
Standalone bookie should always advertise 'localhost' when running in
docker container (#2425)
* Standalone bookie should always advertise 'localhost' when running in
docker container
* Fixed advertised address coming from config file
---
pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandalone.java | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandalone.java
b/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandalone.java
index 3e44f76..cf07d8c 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandalone.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandalone.java
@@ -253,7 +253,8 @@ public class PulsarStandalone implements AutoCloseable {
if (!this.isOnlyBroker()) {
// Start LocalBookKeeper
bkEnsemble = new LocalBookkeeperEnsemble(
-this.getNumOfBk(), this.getZkPort(), this.getBkPort(),
this.getStreamStoragePort(), this.getZkDir(), this.getBkDir(),
this.isWipeData(), config.getAdvertisedAddress());
+this.getNumOfBk(), this.getZkPort(), this.getBkPort(),
this.getStreamStoragePort(), this.getZkDir(),
+this.getBkDir(), this.isWipeData(), "127.0.0.1");
bkEnsemble.startStandalone(!this.isNoStreamStorage());
}
@@ -352,7 +353,7 @@ public class PulsarStandalone implements AutoCloseable {
log.info(e.getMessage());
}
}
-
+
/** this methods gets a buidler to use to build and an embedded pulsar
instance `
* i.e.
*
[incubator-pulsar] branch asf-site updated: Updated site at revision 024d2e0
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 1340c1b Updated site at revision 024d2e0
1340c1b is described below
commit 1340c1b5dcb05621982c0c5ff92965d10e50a9de
Author: jenkins
AuthorDate: Fri Aug 24 08:34:49 2018 +
Updated site at revision 024d2e0
---
content/swagger/swagger.json | 60 ++--
1 file changed, 30 insertions(+), 30 deletions(-)
diff --git a/content/swagger/swagger.json b/content/swagger/swagger.json
index 0d4090a..a45dea8 100644
--- a/content/swagger/swagger.json
+++ b/content/swagger/swagger.json
@@ -6367,10 +6367,10 @@
"type" : "string"
}
},
-"clientVersion" : {
+"connectedSince" : {
"type" : "string"
},
-"connectedSince" : {
+"clientVersion" : {
"type" : "string"
},
"address" : {
@@ -6690,16 +6690,10 @@
"type" : "number",
"format" : "double"
},
-"underLoaded" : {
- "type" : "boolean"
-},
-"overLoaded" : {
- "type" : "boolean"
-},
-"loadReportType" : {
- "type" : "string"
+"bandwidthIn" : {
+ "$ref" : "#/definitions/ResourceUsage"
},
-"cpu" : {
+"bandwidthOut" : {
"$ref" : "#/definitions/ResourceUsage"
},
"memory" : {
@@ -6708,21 +6702,27 @@
"directMemory" : {
"$ref" : "#/definitions/ResourceUsage"
},
-"bandwidthIn" : {
- "$ref" : "#/definitions/ResourceUsage"
+"msgThroughputOut" : {
+ "type" : "number",
+ "format" : "double"
},
-"bandwidthOut" : {
- "$ref" : "#/definitions/ResourceUsage"
+"underLoaded" : {
+ "type" : "boolean"
+},
+"loadReportType" : {
+ "type" : "string"
},
"lastUpdate" : {
"type" : "integer",
"format" : "int64"
},
-"msgThroughputIn" : {
- "type" : "number",
- "format" : "double"
+"cpu" : {
+ "$ref" : "#/definitions/ResourceUsage"
},
-"msgThroughputOut" : {
+"overLoaded" : {
+ "type" : "boolean"
+},
+"msgThroughputIn" : {
"type" : "number",
"format" : "double"
}
@@ -6887,10 +6887,10 @@
"producerName" : {
"type" : "string"
},
-"clientVersion" : {
+"connectedSince" : {
"type" : "string"
},
-"connectedSince" : {
+"clientVersion" : {
"type" : "string"
},
"address" : {
@@ -7591,10 +7591,10 @@
"producerName" : {
"type" : "string"
},
-"clientVersion" : {
+"connectedSince" : {
"type" : "string"
},
-"connectedSince" : {
+"clientVersion" : {
"type" : "string"
},
"address" : {
@@ -7653,15 +7653,15 @@
"ResourceDescription" : {
"type" : "object",
"properties" : {
-"usagePct" : {
- "type" : "integer",
- "format" : "int32"
-},
"resourceUsage" : {
"type" : "object",
"additionalProperties" : {
"$ref" : "#/definitions/ResourceUsage"
}
+},
+"usagePct" : {
+ "type" : "integer",
+ "format" : "int32"
}
}
},
@@ -7696,11 +7696,11 @@
"ResourceUnit" : {
"type" : "object",
"properties" : {
-"resourceId" : {
- "type" : "string"
-},
"availableResource" : {
"$ref" : "#/definitions/ResourceDescription"
+},
+"resourceId" : {
+ "type" : "string"
}
}
},
[incubator-pulsar] branch master updated: Allow to configure number of IO and managed ledger threads (#2412)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new ef37890 Allow to configure number of IO and managed ledger threads
(#2412)
ef37890 is described below
commit ef37890331197ee4f40da9fa787ed5241a55cb6a
Author: Matteo Merli
AuthorDate: Wed Aug 22 10:33:49 2018 -0700
Allow to configure number of IO and managed ledger threads (#2412)
---
conf/broker.conf | 9 +++
conf/standalone.conf | 9 +++
.../mledger/ManagedLedgerFactoryConfig.java| 34 +++
.../mledger/impl/ManagedLedgerFactoryImpl.java | 66 ++
.../apache/pulsar/broker/ServiceConfiguration.java | 33 +++
.../pulsar/broker/ManagedLedgerClientFactory.java | 2 +
.../pulsar/broker/service/BrokerService.java | 2 +-
.../org/apache/pulsar/common/util/FieldParser.java | 7 ++-
8 files changed, 100 insertions(+), 62 deletions(-)
diff --git a/conf/broker.conf b/conf/broker.conf
index 427d848..5465c9a 100644
--- a/conf/broker.conf
+++ b/conf/broker.conf
@@ -43,6 +43,9 @@ bindAddress=0.0.0.0
# Hostname or IP address the service advertises to the outside world. If not
set, the value of InetAddress.getLocalHost().getHostName() is used.
advertisedAddress=
+# Number of threads to use for Netty IO. Default is set to 2 *
Runtime.getRuntime().availableProcessors()
+numIOThreads=
+
# Name of the cluster to which this broker belongs to
clusterName=
@@ -311,6 +314,12 @@ managedLedgerDefaultAckQuorum=2
# Other possible options are "CRC32", "MAC" or "DUMMY" (no checksum).
managedLedgerDigestType=CRC32C
+# Number of threads to be used for managed ledger tasks dispatching
+managedLedgerNumWorkerThreads=8
+
+# Number of threads to be used for managed ledger scheduled tasks
+managedLedgerNumSchedulerThreads=8
+
# Amount of memory to use for caching data payload in managed ledger. This
memory
# is allocated from JVM direct memory and it's shared across all the topics
# running in the same broker
diff --git a/conf/standalone.conf b/conf/standalone.conf
index cc03b95..13bef8a 100644
--- a/conf/standalone.conf
+++ b/conf/standalone.conf
@@ -36,6 +36,9 @@ bindAddress=0.0.0.0
# Hostname or IP address the service advertises to the outside world. If not
set, the value of InetAddress.getLocalHost().getHostName() is used.
advertisedAddress=
+# Number of threads to use for Netty IO. Default is set to 2 *
Runtime.getRuntime().availableProcessors()
+numIOThreads=
+
# Name of the cluster to which this broker belongs to
clusterName=standalone
@@ -261,6 +264,12 @@ managedLedgerDefaultAckQuorum=1
# Other possible options are "CRC32C" (which is faster), "MAC" or "DUMMY" (no
checksum).
managedLedgerDigestType=CRC32
+# Number of threads to be used for managed ledger tasks dispatching
+managedLedgerNumWorkerThreads=4
+
+# Number of threads to be used for managed ledger scheduled tasks
+managedLedgerNumSchedulerThreads=4
+
# Amount of memory to use for caching data payload in managed ledger. This
memory
# is allocated from JVM direct memory and it's shared across all the topics
# running in the same broker
diff --git
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerFactoryConfig.java
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerFactoryConfig.java
index 6f1f41d..81673f4 100644
---
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerFactoryConfig.java
+++
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerFactoryConfig.java
@@ -18,42 +18,26 @@
*/
package org.apache.bookkeeper.mledger;
+import lombok.Data;
+
/**
* Configuration for a {@link ManagedLedgerFactory}.
*/
+@Data
public class ManagedLedgerFactoryConfig {
private static final long MB = 1024 * 1024;
private long maxCacheSize = 128 * MB;
-private double cacheEvictionWatermark = 0.90;
-
-public long getMaxCacheSize() {
-return maxCacheSize;
-}
/**
- *
- * @param maxCacheSize
- * @return
+ * The cache eviction watermark is the percentage of the cache size to
reach when removing entries from the cache.
*/
-public ManagedLedgerFactoryConfig setMaxCacheSize(long maxCacheSize) {
-this.maxCacheSize = maxCacheSize;
-return this;
-}
+private double cacheEvictionWatermark = 0.90;
-public double getCacheEvictionWatermark() {
-return cacheEvictionWatermark;
-}
+private int numManagedLedgerWorkerThreads =
Runtime.getRuntime().availableProcessors();
+private int numManagedLedgerSchedulerThreads =
Runtime.getRuntime().availableProcessors();
-/**
- * The cache eviction watermark is the
[incubator-pulsar] branch master updated: [client] add properties to producer for cpp & python client (#2420)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 834104b [client] add properties to producer for cpp & python client
(#2420)
834104b is described below
commit 834104bba3eab1977114b58f455d735bda2b9fe3
Author: Sijie Guo
AuthorDate: Wed Aug 22 09:32:46 2018 -0700
[client] add properties to producer for cpp & python client (#2420)
* [client] add properties to producer for cpp & python client
### Motivation
This is a caught-up change to enable properties for producer as java
clients.
### Changes
Enable properties on producer for both cpp & python client
### Results
Properties are added as metadata for CommandProducer. However there is no
way
to verify the producer properties. so I didn't add any specific tests, just
adding properties for both cpp and python clients in the tests, that should
excerise the corresponding code path.
* Add `properties` to pydoc
---
.../include/pulsar/ProducerConfiguration.h | 34 ++
pulsar-client-cpp/lib/Commands.cc | 11 ++-
pulsar-client-cpp/lib/Commands.h | 3 +-
pulsar-client-cpp/lib/ProducerConfiguration.cc | 34 +-
pulsar-client-cpp/lib/ProducerConfigurationImpl.h | 1 +
pulsar-client-cpp/lib/ProducerImpl.cc | 3 +-
pulsar-client-cpp/python/pulsar/__init__.py| 11 ++-
pulsar-client-cpp/python/src/config.cc | 1 +
pulsar-client-cpp/python/test_producer.py | 6 +++-
pulsar-client-cpp/tests/BasicEndToEndTest.cc | 2 ++
10 files changed, 100 insertions(+), 6 deletions(-)
diff --git a/pulsar-client-cpp/include/pulsar/ProducerConfiguration.h
b/pulsar-client-cpp/include/pulsar/ProducerConfiguration.h
index 9f7bf1f..45154c5 100644
--- a/pulsar-client-cpp/include/pulsar/ProducerConfiguration.h
+++ b/pulsar-client-cpp/include/pulsar/ProducerConfiguration.h
@@ -127,6 +127,40 @@ class ProducerConfiguration {
bool isEncryptionEnabled() const;
ProducerConfiguration& addEncryptionKey(std::string key);
+/**
+ * Check whether the message has a specific property attached.
+ *
+ * @param name the name of the property to check
+ * @return true if the message has the specified property
+ * @return false if the property is not defined
+ */
+bool hasProperty(const std::string& name) const;
+
+/**
+ * Get the value of a specific property
+ *
+ * @param name the name of the property
+ * @return the value of the property or null if the property was not
defined
+ */
+const std::string& getProperty(const std::string& name) const;
+
+/**
+ * Get all the properties attached to this producer.
+ */
+std::map& getProperties() const;
+
+/**
+ * Sets a new property on a message.
+ * @param name the name of the property
+ * @param value the associated value
+ */
+ProducerConfiguration& setProperty(const std::string& name, const
std::string& value);
+
+/**
+ * Add all the properties in the provided map
+ */
+ProducerConfiguration& setProperties(const std::map& properties);
+
friend class PulsarWrapper;
private:
diff --git a/pulsar-client-cpp/lib/Commands.cc
b/pulsar-client-cpp/lib/Commands.cc
index 8bd0128..4d9f1be 100644
--- a/pulsar-client-cpp/lib/Commands.cc
+++ b/pulsar-client-cpp/lib/Commands.cc
@@ -22,6 +22,7 @@
#include "pulsar/MessageBuilder.h"
#include "PulsarApi.pb.h"
#include "LogUtils.h"
+#include "PulsarApi.pb.h"
#include "Utils.h"
#include "Url.h"
#include "checksum/ChecksumProvider.h"
@@ -231,13 +232,21 @@ SharedBuffer Commands::newUnsubscribe(uint64_t
consumerId, uint64_t requestId) {
}
SharedBuffer Commands::newProducer(const std::string& topic, uint64_t
producerId,
- const std::string& producerName, uint64_t
requestId) {
+ const std::string& producerName, uint64_t
requestId,
+ const std::map&
metadata) {
BaseCommand cmd;
cmd.set_type(BaseCommand::PRODUCER);
CommandProducer* producer = cmd.mutable_producer();
producer->set_topic(topic);
producer->set_producer_id(producerId);
producer->set_request_id(requestId);
+for (std::map::const_iterator it =
metadata.begin(); it != metadata.end();
+ it++) {
+proto::KeyValue* keyValue = proto::KeyValue().New();
+keyValue->set_key(it->first);
+keyValue->set_value(it->second);
+producer->mutable_metadata()-&
[incubator-pulsar] branch master updated: add slf4j handler to log web-server log (#2408)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 0bf3494 add slf4j handler to log web-server log (#2408)
0bf3494 is described below
commit 0bf3494dc2e941e53a495c92310f3847aa11fd09
Author: Rajan Dhabalia
AuthorDate: Mon Aug 20 13:47:48 2018 -0700
add slf4j handler to log web-server log (#2408)
---
.../apache/pulsar/functions/worker/rest/WorkerServer.java | 15 ++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git
a/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/rest/WorkerServer.java
b/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/rest/WorkerServer.java
index 3e9a426..1c26c7d 100644
---
a/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/rest/WorkerServer.java
+++
b/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/rest/WorkerServer.java
@@ -21,6 +21,8 @@ package org.apache.pulsar.functions.worker.rest;
import java.util.ArrayList;
import java.util.EnumSet;
import java.util.List;
+import java.util.TimeZone;
+
import javax.servlet.DispatcherType;
import lombok.extern.slf4j.Slf4j;
import org.apache.pulsar.broker.web.AuthenticationFilter;
@@ -32,6 +34,7 @@ import org.apache.pulsar.functions.worker.WorkerService;
import org.eclipse.jetty.server.Handler;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.ServerConnector;
+import org.eclipse.jetty.server.Slf4jRequestLog;
import java.net.BindException;
import java.net.URI;
@@ -39,6 +42,7 @@ import java.security.GeneralSecurityException;
import org.eclipse.jetty.server.handler.ContextHandlerCollection;
import org.eclipse.jetty.server.handler.DefaultHandler;
import org.eclipse.jetty.server.handler.HandlerCollection;
+import org.eclipse.jetty.server.handler.RequestLogHandler;
import org.eclipse.jetty.servlet.FilterHolder;
import org.eclipse.jetty.servlet.ServletContextHandler;
import org.eclipse.jetty.servlet.ServletHolder;
@@ -98,10 +102,19 @@ public class WorkerServer {
newServletContextHandler("/admin/v2", new
ResourceConfig(Resources.getApiResources()), workerService));
handlers.add(newServletContextHandler("/", new
ResourceConfig(Resources.getRootResources()), workerService));
+RequestLogHandler requestLogHandler = new RequestLogHandler();
+Slf4jRequestLog requestLog = new Slf4jRequestLog();
+requestLog.setExtended(true);
+requestLog.setLogTimeZone(TimeZone.getDefault().getID());
+requestLog.setLogLatency(true);
+requestLogHandler.setRequestLog(requestLog);
+handlers.add(0, new ContextHandlerCollection());
+handlers.add(requestLogHandler);
+
ContextHandlerCollection contexts = new ContextHandlerCollection();
contexts.setHandlers(handlers.toArray(new Handler[handlers.size()]));
HandlerCollection handlerCollection = new HandlerCollection();
-handlerCollection.setHandlers(new Handler[] { contexts, new
DefaultHandler() });
+handlerCollection.setHandlers(new Handler[] { contexts, new
DefaultHandler(), requestLogHandler });
server.setHandler(handlerCollection);
if (this.workerConfig.isTlsEnabled()) {
[incubator-pulsar] branch asf-site updated: Updated site at revision 90d86c0
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/asf-site by this push: new f3f96e5 Updated site at revision 90d86c0 f3f96e5 is described below commit f3f96e5bbfcf8e869b6c8de54a15ff9fa95d Author: jenkins AuthorDate: Sat Aug 18 08:38:42 2018 + Updated site at revision 90d86c0 --- content/api/admin/index-all.html | 4 ++ ...sarAdminException.ServerSideErrorException.html | 14 +++ .../pulsar/client/admin/PulsarAdminException.html | 12 ++ .../apache/pulsar/client/api/ClientBuilder.html| 2 +- .../pulsar/client/api/ClientConfiguration.html | 2 +- .../docs/latest/adaptors/PulsarSpark/index.html| 8 ++-- .../docs/latest/adaptors/PulsarStorm/index.html| 6 +-- content/docs/latest/admin-api/overview/index.html | 10 ++--- content/docs/latest/clients/Cpp/index.html | 8 ++-- content/docs/latest/clients/Java/index.html| 14 +++ content/docs/latest/clients/Python/index.html | 10 ++--- content/docs/latest/clients/WebSocket/index.html | 8 ++-- content/docs/latest/clients/go/index.html | 6 +-- .../docs/latest/cookbooks/Encryption/index.html| 6 +-- .../latest/cookbooks/PartitionedTopics/index.html | 14 +++ .../latest/cookbooks/RetentionExpiry/index.html| 14 +++ .../docs/latest/cookbooks/compaction/index.html| 6 +-- .../cookbooks/message-deduplication/index.html | 10 ++--- .../docs/latest/cookbooks/message-queue/index.html | 14 +++ .../latest/cookbooks/tiered-storage/index.html | 4 +- .../docs/latest/deployment/Kubernetes/index.html | 4 +- .../docs/latest/deployment/aws-cluster/index.html | 4 +- content/docs/latest/deployment/cluster/index.html | 4 +- content/docs/latest/deployment/instance/index.html | 4 +- .../ConceptsAndArchitecture/index.html | 4 +- .../latest/getting-started/LocalCluster/index.html | 4 +- .../latest/getting-started/Pulsar-2.0/index.html | 2 +- .../docs/latest/getting-started/docker/index.html | 4 +- .../docs/latest/project/BinaryProtocol/index.html | 4 +- content/docs/latest/project/CompileCpp/index.html | 8 ++-- .../docs/latest/project/SimulationTools/index.html | 2 +- .../docs/latest/project/schema-storage/index.html | 4 +- content/docs/latest/reference/CliTools/index.html | 18 - content/docs/latest/reference/RestApi/index.html | 5 +++ .../docs/latest/security/authorization/index.html | 10 ++--- content/docs/latest/security/encryption/index.html | 6 +-- content/ja/adaptors/PulsarSpark/index.html | 8 ++-- content/ja/adaptors/PulsarStorm/index.html | 6 +-- content/ja/admin/AdminInterface/index.html | 12 +++--- content/ja/admin/Authz/index.html | 12 +++--- content/ja/admin/ClustersBrokers/index.html| 6 +-- content/ja/admin/PropertiesNamespaces/index.html | 6 +-- content/ja/advanced/PartitionedTopics/index.html | 12 +++--- content/ja/advanced/RetentionExpiry/index.html | 12 +++--- content/ja/clients/Cpp/index.html | 6 +-- content/ja/clients/Java/index.html | 8 ++-- content/ja/clients/Python/index.html | 8 ++-- content/ja/clients/WebSocket/index.html| 8 ++-- content/ja/deployment/InstanceSetup/index.html | 6 +-- content/ja/deployment/Kubernetes/index.html| 4 +- .../ConceptsAndArchitecture/index.html | 2 +- content/ja/getting-started/LocalCluster/index.html | 4 +- content/ja/project/BinaryProtocol/index.html | 4 +- content/ja/project/SimulationTools/index.html | 2 +- content/ja/reference/CliTools/index.html | 18 - content/ja/reference/RestApi/index.html| 5 +++ content/js/pjax-api.min.js | 2 +- content/swagger/swagger.json | 47 -- 58 files changed, 250 insertions(+), 207 deletions(-) diff --git a/content/api/admin/index-all.html b/content/api/admin/index-all.html index ebff08d..7ae45af 100644 --- a/content/api/admin/index-all.html +++ b/content/api/admin/index-all.html @@ -1708,6 +1708,8 @@ PulsarAdminException(ServerErrorException, String) - Constructor for exception org.apache.pulsar.client.admin.PulsarAdminException +PulsarAdminException(WebApplicationException) - Constructor for exception org.apache.pulsar.client.admin.PulsarAdminException + PulsarAdminException(String, Throwable) - Constructor for exception org.apache.pulsar.client.admin.PulsarAdminException PulsarAdminException.ConflictException - Exception in org.apache.pulsar.client.admin @@ -1850,6 +1852,8 @@ secondary - Variable in class org.apache.pulsar.common.policies.data.NamespaceIsolationData +ServerSideErrorException(ServerErrorException
[incubator-pulsar] branch master updated: [build] Configure os extension plugin at root pom (#2396)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/master by this push: new 90d86c0 [build] Configure os extension plugin at root pom (#2396) 90d86c0 is described below commit 90d86c0fe31fd614c9aa9037f900305661ece879 Author: Sijie Guo AuthorDate: Fri Aug 17 13:32:10 2018 -0700 [build] Configure os extension plugin at root pom (#2396) ### Motivation os extension plugin is required by managed-ledger as well. ### Change move the extension to the root pom file. --- pom.xml | 5 + pulsar-client-schema/pom.xml | 7 --- pulsar-functions/proto-shaded/pom.xml | 7 --- pulsar-functions/proto/pom.xml| 7 --- 4 files changed, 5 insertions(+), 21 deletions(-) diff --git a/pom.xml b/pom.xml index f60421c..49edc8c 100644 --- a/pom.xml +++ b/pom.xml @@ -1256,6 +1256,11 @@ flexible messaging model and an intuitive client API. wagon-ssh-external 2.10 + +kr.motd.maven +os-maven-plugin +1.4.1.Final + diff --git a/pulsar-client-schema/pom.xml b/pulsar-client-schema/pom.xml index 7ee5646..477627e 100644 --- a/pulsar-client-schema/pom.xml +++ b/pulsar-client-schema/pom.xml @@ -86,13 +86,6 @@ true - - -kr.motd.maven -os-maven-plugin -1.4.1.Final - - diff --git a/pulsar-functions/proto-shaded/pom.xml b/pulsar-functions/proto-shaded/pom.xml index 2a4d68f..7cdc489 100644 --- a/pulsar-functions/proto-shaded/pom.xml +++ b/pulsar-functions/proto-shaded/pom.xml @@ -46,13 +46,6 @@ - - -kr.motd.maven -os-maven-plugin -1.4.1.Final - - org.apache.maven.plugins diff --git a/pulsar-functions/proto/pom.xml b/pulsar-functions/proto/pom.xml index 4249ec6..28a5e06 100644 --- a/pulsar-functions/proto/pom.xml +++ b/pulsar-functions/proto/pom.xml @@ -54,13 +54,6 @@ - - -kr.motd.maven -os-maven-plugin -1.4.1.Final - - org.xolstice.maven.plugins
[incubator-pulsar] branch asf-site updated: Updated site at revision 4f43a3f
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/asf-site by this push:
new e2bfd71 Updated site at revision 4f43a3f
e2bfd71 is described below
commit e2bfd71e746832798ab7be5235181546f8bb23df
Author: jenkins
AuthorDate: Fri Aug 17 16:18:36 2018 +
Updated site at revision 4f43a3f
---
content/en/versions.html | 2 +-
content/en/versions/index.html| 2 +-
content/fr/versions.html | 2 +-
content/fr/versions/index.html| 2 +-
content/ja/versions.html | 2 +-
content/ja/versions/index.html| 2 +-
content/swagger/swagger.json | 38 +++---
content/versions.html | 2 +-
content/versions/index.html | 2 +-
content/zh-CN/versions.html | 2 +-
content/zh-CN/versions/index.html | 2 +-
11 files changed, 29 insertions(+), 29 deletions(-)
diff --git a/content/en/versions.html b/content/en/versions.html
index 95a75c4..8292114 100644
--- a/content/en/versions.html
+++ b/content/en/versions.html
@@ -18,7 +18,7 @@
languagesDropDown.className = "hide";
}
});
- Apache Pulsar
VersionsLatest Stable VersionLatest
stable release of A [...]
+ Apache Pulsar
VersionsLatest Stable VersionLatest
stable release of A [...]
const community =
document.querySelector("a[href='#community']").parentNode;
const communityMenu =
'' +
diff --git a/content/en/versions/index.html b/content/en/versions/index.html
index 95a75c4..8292114 100644
--- a/content/en/versions/index.html
+++ b/content/en/versions/index.html
@@ -18,7 +18,7 @@
languagesDropDown.className = "hide";
}
});
- Apache Pulsar
VersionsLatest Stable VersionLatest
stable release of A [...]
+ Apache Pulsar
VersionsLatest Stable VersionLatest
stable release of A [...]
const community =
document.querySelector("a[href='#community']").parentNode;
const communityMenu =
'' +
diff --git a/content/fr/versions.html b/content/fr/versions.html
index 0f76fae..d3b4e56 100644
--- a/content/fr/versions.html
+++ b/content/fr/versions.html
@@ -18,7 +18,7 @@
languagesDropDown.className = "hide";
}
});
- Apache Pulsar
VersionsDernière version stableDernière
version stable [...]
+ Apache Pulsar
VersionsDernière version stableDernière
version stable [...]
const community =
document.querySelector("a[href='#community']").parentNode;
const communityMenu =
'' +
diff --git a/content/fr/versions/index.html b/content/fr/versions/index.html
index 0f76fae..d3b4e56 100644
--- a/content/fr/versions/index.html
+++ b/content/fr/versions/index.html
@@ -18,7 +18,7 @@
languagesDropDown.className = "hide";
}
});
- Apache Pulsar
VersionsDernière version stableDernière
version stable [...]
+ Apache Pulsar
VersionsDernière version stableDernière
version stable [...]
const community =
document.querySelector("a[href='#community']").parentNode;
const communityMenu =
'' +
diff --git a/content/ja/versions.html b/content/ja/versions.html
index 26f3c8b..6e5c7d2 100644
--- a/content/ja/versions.html
+++ b/content/ja/versions.html
@@ -18,7 +18,7 @@
languagesDropDown.className = "hide";
}
});
- Apache Pulsar
VersionsLatest Stable VersionLatest
stable release of A [...]
+ Apache Pulsar
VersionsLatest Stable VersionLatest
stable release of A [...]
const community =
document.querySelector("a[href='#community']").parentNode;
const communityMenu =
'' +
diff --git a/content/ja/versions/index.html b/content/ja/versions/index.html
index 26f3c8b..6e5c7d2 100644
--- a/content/ja/versions/index.html
+++ b/content/ja/versions/index.html
@@ -18,7 +18,7 @@
languagesDropDown.className = "hide";
}
});
- Apache Pulsar
VersionsLatest Stable VersionLatest
stable release of A [...]
+ Apache Pulsar
VersionsLatest Stable VersionLatest
stable release of A [...]
const community =
document.querySelector("a[href='#community']").parentNode;
const communityMenu =
'' +
diff --git a/content/swagger/swagger.json b/content/swagger/swagger.json
index cb4afa7..e1b2147 100644
--- a/content/swagger/swagger.json
+++ b/content/swagger/swagger.json
@@ -6690,11 +6690,28 @@
"cpu" : {
"$ref" : "#/definitions/ResourceUsage"
},
+"lastUpdate" : {
+ "type" : "integer",
+ "format" : "int64"
+
[incubator-pulsar] branch asf-site updated: Updated site at revision e1238ac
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/asf-site by this push: new 73448f6 Updated site at revision e1238ac 73448f6 is described below commit 73448f63b5208cc2cbd645734e9e5e6c3b3e4cf7 Author: jenkins AuthorDate: Fri Aug 17 09:29:39 2018 + Updated site at revision e1238ac --- .../docs/latest/adaptors/PulsarSpark/index.html| 2 +- .../docs/latest/adaptors/PulsarStorm/index.html| 2 +- content/docs/latest/admin-api/overview/index.html | 8 content/docs/latest/clients/Java/index.html| 2 +- .../latest/cookbooks/PartitionedTopics/index.html | 2 +- .../latest/cookbooks/RetentionExpiry/index.html| 2 +- .../docs/latest/cookbooks/compaction/index.html| 2 +- .../cookbooks/message-deduplication/index.html | 2 +- .../docs/latest/cookbooks/message-queue/index.html | 2 +- .../latest/cookbooks/tiered-storage/index.html | 2 +- .../docs/latest/deployment/aws-cluster/index.html | 2 +- content/docs/latest/deployment/cluster/index.html | 2 +- content/docs/latest/deployment/instance/index.html | 2 +- content/docs/latest/reference/CliTools/index.html | 4 ++-- .../docs/latest/security/authorization/index.html | 4 ++-- content/swagger/swagger.json | 22 +++--- 16 files changed, 31 insertions(+), 31 deletions(-) diff --git a/content/docs/latest/adaptors/PulsarSpark/index.html b/content/docs/latest/adaptors/PulsarSpark/index.html index c00dd49..a30d928 100644 --- a/content/docs/latest/adaptors/PulsarSpark/index.html +++ b/content/docs/latest/adaptors/PulsarSpark/index.html @@ -1673,9 +1673,9 @@ - The Pulsar admin interface + The Pulsar admin interface diff --git a/content/docs/latest/adaptors/PulsarStorm/index.html b/content/docs/latest/adaptors/PulsarStorm/index.html index a100055..3059cf5 100644 --- a/content/docs/latest/adaptors/PulsarStorm/index.html +++ b/content/docs/latest/adaptors/PulsarStorm/index.html @@ -1427,9 +1427,9 @@ - The Pulsar admin interface + The Pulsar admin interface diff --git a/content/docs/latest/admin-api/overview/index.html b/content/docs/latest/admin-api/overview/index.html index 994ad01..80a6f28 100644 --- a/content/docs/latest/admin-api/overview/index.html +++ b/content/docs/latest/admin-api/overview/index.html @@ -1189,9 +1189,9 @@ - The Pulsar admin interface + The Pulsar admin interface @@ -1439,9 +1439,9 @@ - The Pulsar admin interface + The Pulsar admin interface @@ -1685,9 +1685,9 @@ - The Pulsar admin interface + The Pulsar admin interface @@ -1941,9 +1941,9 @@ - The Pulsar admin interface + The Pulsar admin interface diff --git a/content/docs/latest/clients/Java/index.html b/content/docs/latest/clients/Java/index.html index 8ce7177..b938e5f 100644 --- a/content/docs/latest/clients/Java/index.html +++ b/content/docs/latest/clients/Java/index.html @@ -1437,9 +1437,9 @@ - The Pulsar admin interface + The Pulsar admin interface diff --git a/content/docs/latest/cookbooks/PartitionedTopics/index.html b/content/docs/latest/cookbooks/PartitionedTopics/index.html index 25ce2c6..c5c0499 100644 --- a/content/docs/latest/cookbooks/PartitionedTopics/index.html +++ b/content/docs/latest/cookbooks/PartitionedTopics/index.html @@ -1681,9 +1681,9 @@ - The Pulsar admin interface + The Pulsar admin interface diff --git a/content/docs/latest/cookbooks/RetentionExpiry/index.html b/content/docs/latest/cookbooks/RetentionExpiry/index.html index 0a51438..763a81c 100644 --- a/content/docs/latest/cookbooks/RetentionExpiry/index.html +++ b/content/docs/latest/cookbooks/RetentionExpiry/index.html @@ -1189,9 +1189,9 @@ - The Pulsar admin interface + The Pulsar admin interface diff --git a/content/docs/latest/cookbooks/compaction/index.html b/content/docs/latest/cookbooks
[incubator-pulsar] branch master updated: [documentation][deploy] Improve helm deployment script to deploy Pulsar to minikube (#2363)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 2bc4812 [documentation][deploy] Improve helm deployment script to
deploy Pulsar to minikube (#2363)
2bc4812 is described below
commit 2bc48129bfb4433182f1b05a9008faef76e387ac
Author: Sijie Guo
AuthorDate: Thu Aug 16 00:25:49 2018 -0700
[documentation][deploy] Improve helm deployment script to deploy Pulsar to
minikube (#2363)
* [documentation][deploy] Update deployment instructions for deploying to
Minikube
* Enable functions workers
* [documentation][deploy] Improve helm deployment script to deploy Pulsar
to minikube
### Changes
- update the helm scripts: bookie/autorecovery/broker pods should wait
until metadata is initialized
- disable `autoRecovery` on bookies since we start `AutoRecovery` in
separate pods
- enable function worker on brokers
- provide a values file for minikube
- update documentation for using helm chart to deploy a cluster to minikube
* move the service type definition to values file
---
deployment/kubernetes/helm/README.md | 54 +
.../pulsar/templates/autorecovery-deployment.yaml | 2 +-
.../pulsar/templates/bookkeeper-configmap.yaml | 2 +
.../pulsar/templates/bookkeeper-statefulset.yaml | 2 +-
.../helm/pulsar/templates/broker-configmap.yaml| 2 +
.../helm/pulsar/templates/broker-deployment.yaml | 3 +-
.../pulsar/templates/prometheus-deployment.yaml| 6 +
.../helm/pulsar/templates/proxy-deployment.yaml| 2 +-
.../helm/pulsar/templates/proxy-service.yaml | 2 +-
.../helm/pulsar/templates/zookeeper-metadata.yaml | 4 +
.../helm/pulsar/{values.yaml => values-mini.yaml} | 125 +
deployment/kubernetes/helm/pulsar/values.yaml | 9 +-
site2/docs/deploy-kubernetes.md| 35 +-
13 files changed, 166 insertions(+), 82 deletions(-)
diff --git a/deployment/kubernetes/helm/README.md
b/deployment/kubernetes/helm/README.md
index 627b0fc..36f16bc 100644
--- a/deployment/kubernetes/helm/README.md
+++ b/deployment/kubernetes/helm/README.md
@@ -21,3 +21,57 @@
This directory contains the Helm Chart required
to do a complete Pulsar deployment on Kubernetes.
+
+## Install Helm
+
+Before you start, you need to install helm.
+Following [helm
documentation](https://docs.helm.sh/using_helm/#installing-helm) to install it.
+
+## Deploy Pulsar
+
+### Minikube
+
+ Install Minikube
+
+[Install and configure
minikube](https://github.com/kubernetes/minikube#installation) with
+a [VM driver](https://github.com/kubernetes/minikube#requirements), e.g.
`kvm2` on Linux
+or `hyperkit` or `VirtualBox` on macOS.
+
+ Create a K8S cluster on Minikube
+
+```
+minikube start --memory=8192 --cpus=4 \
+--kubernetes-version=v1.10.5
+```
+
+ Set kubectl to use Minikube.
+
+```
+kubectl config use-context minikube
+```
+
+After you created a K8S cluster on Minikube, you can access its dashboard via
following command:
+
+```
+minikube dashboard
+```
+
+The command will automatically trigger open a webpage in your browser.
+
+ Install Pulsar Chart
+
+Assume you already cloned pulsar repo in `PULSAR_HOME` directory.
+
+1. Go to Pulsar helm chart directory
+```shell
+cd ${PULSAR_HOME}/deployment/kubernetes/helm
+```
+1. Install helm chart.
+```shell
+helm install --values pulsar/values-mini.yaml ./pulsar
+```
+
+Once the helm chart is completed on installation, you can access the cluster
via:
+
+- Web service url: `http://$(minikube ip):30001/`
+- Pulsar service url: `pulsar://$(minikube ip):30002/`
diff --git
a/deployment/kubernetes/helm/pulsar/templates/autorecovery-deployment.yaml
b/deployment/kubernetes/helm/pulsar/templates/autorecovery-deployment.yaml
index fb98e1b..fe1dd08 100644
--- a/deployment/kubernetes/helm/pulsar/templates/autorecovery-deployment.yaml
+++ b/deployment/kubernetes/helm/pulsar/templates/autorecovery-deployment.yaml
@@ -83,7 +83,7 @@ spec:
command: ["sh", "-c"]
args:
- >-
-until nslookup {{ template "pulsar.fullname" . }}-{{
.Values.zookeeper.component }}-{{ add (.Values.zookeeper.replicaCount | int) -1
}}.{{ template "pulsar.fullname" . }}-{{ .Values.zookeeper.component }}.{{
.Values.namespace }}; do
+until bin/pulsar zookeeper-shell -server {{ template
"pulsar.fullname" . }}-{{ .Values.zookeeper.component }} ls /admin/clusters/{{
template "pulsar.fullname" . }}; do
sleep 3;
done;
containers:
diff --git
a/deployment/kubernetes/helm/pulsar/templates/bookkeeper-configmap.yaml
b/deployment/kubernetes/helm/pulsar/templates/bookke
[incubator-pulsar] branch master updated: Avoid double executor on non-persistent topic send operation (#2351)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 722dac5 Avoid double executor on non-persistent topic send operation
(#2351)
722dac5 is described below
commit 722dac58cf5a15e79acc428b6b9978d0b0d6e85b
Author: Matteo Merli
AuthorDate: Wed Aug 15 11:54:02 2018 -0700
Avoid double executor on non-persistent topic send operation (#2351)
---
.../service/nonpersistent/NonPersistentTopic.java | 42 --
1 file changed, 15 insertions(+), 27 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/nonpersistent/NonPersistentTopic.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/nonpersistent/NonPersistentTopic.java
index e13549e..281d843 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/nonpersistent/NonPersistentTopic.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/nonpersistent/NonPersistentTopic.java
@@ -188,32 +188,24 @@ public class NonPersistentTopic implements Topic {
@Override
public void publishMessage(ByteBuf data, PublishContext callback) {
-AtomicInteger msgDeliveryCount = new AtomicInteger(2);
+callback.completed(null, 0L, 0L);
ENTRIES_ADDED_COUNTER_UPDATER.incrementAndGet(this);
-// retain data for sub/replication because io-thread will release
actual payload
-data.retain(2);
-this.executor.executeOrdered(topic, SafeRun.safeRun(() -> {
-subscriptions.forEach((name, subscription) -> {
-ByteBuf duplicateBuffer = data.retainedDuplicate();
-Entry entry = create(0L, 0L, duplicateBuffer);
-// entry internally retains data so, duplicateBuffer should be
release here
-duplicateBuffer.release();
-if (subscription.getDispatcher() != null) {
-
subscription.getDispatcher().sendMessages(Lists.newArrayList(entry));
-} else {
-// it happens when subscription is created but dispatcher
is not created as consumer is not added
-// yet
-entry.release();
-}
-});
-data.release();
-if (msgDeliveryCount.decrementAndGet() == 0) {
-callback.completed(null, 0L, 0L);
+subscriptions.forEach((name, subscription) -> {
+ByteBuf duplicateBuffer = data.retainedDuplicate();
+Entry entry = create(0L, 0L, duplicateBuffer);
+// entry internally retains data so, duplicateBuffer should be
release here
+duplicateBuffer.release();
+if (subscription.getDispatcher() != null) {
+
subscription.getDispatcher().sendMessages(Collections.singletonList(entry));
+} else {
+// it happens when subscription is created but dispatcher is
not created as consumer is not added
+// yet
+entry.release();
}
-}));
+});
-this.executor.executeOrdered(topic, SafeRun.safeRun(() -> {
+if (!replicators.isEmpty()) {
replicators.forEach((name, replicator) -> {
ByteBuf duplicateBuffer = data.retainedDuplicate();
Entry entry = create(0L, 0L, duplicateBuffer);
@@ -221,11 +213,7 @@ public class NonPersistentTopic implements Topic {
duplicateBuffer.release();
((NonPersistentReplicator) replicator).sendMessage(entry);
});
-data.release();
-if (msgDeliveryCount.decrementAndGet() == 0) {
-callback.completed(null, 0L, 0L);
-}
-}));
+}
}
@Override
[incubator-pulsar] branch master updated: [cpp] receiver queue size config acorss partitions in multi-topics-consumer (#2311)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/master by this push: new 296830a [cpp] receiver queue size config acorss partitions in multi-topics-consumer (#2311) 296830a is described below commit 296830a37d82de65d30dfd74d9a5893033056f18 Author: Jia Zhai AuthorDate: Tue Aug 14 06:05:59 2018 +0800 [cpp] receiver queue size config acorss partitions in multi-topics-consumer (#2311) * catch up receiver queue size support in multi topics consumer * add python config --- pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc | 5 - pulsar-client-cpp/python/pulsar/__init__.py| 13 pulsar-client-cpp/python/src/config.cc | 4 .../client/impl/MultiTopicsConsumerImpl.java | 23 +- 4 files changed, 35 insertions(+), 10 deletions(-) diff --git a/pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc b/pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc index 7be197c..6750273 100644 --- a/pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc +++ b/pulsar-client-cpp/lib/MultiTopicsConsumerImpl.cc @@ -160,9 +160,12 @@ void MultiTopicsConsumerImpl::subscribeTopicPartitions(const Result result, config.setBrokerConsumerStatsCacheTimeInMs(conf_.getBrokerConsumerStatsCacheTimeInMs()); config.setMessageListener( boost::bind(::messageReceived, shared_from_this(), _1, _2)); -config.setReceiverQueueSize(conf_.getReceiverQueueSize()); int numPartitions = partitionMetadata->getPartitions() >= 1 ? partitionMetadata->getPartitions() : 1; +// Apply total limit of receiver queue size across partitions +config.setReceiverQueueSize( +std::min(conf_.getReceiverQueueSize(), + (int)(conf_.getMaxTotalReceiverQueueSizeAcrossPartitions() / numPartitions))); Lock lock(mutex_); topicsPartitions_.insert(std::make_pair(topicName->toString(), numPartitions)); diff --git a/pulsar-client-cpp/python/pulsar/__init__.py b/pulsar-client-cpp/python/pulsar/__init__.py index 434fb07..f3b560b 100644 --- a/pulsar-client-cpp/python/pulsar/__init__.py +++ b/pulsar-client-cpp/python/pulsar/__init__.py @@ -315,6 +315,7 @@ class Client: send_timeout_millis=3, compression_type=CompressionType.NONE, max_pending_messages=1000, +max_pending_messages_across_partitions=5, block_if_queue_full=False, batching_enabled=False, batching_max_messages=1000, @@ -352,6 +353,9 @@ class Client: * `max_pending_messages`: Set the max size of the queue holding the messages pending to receive an acknowledgment from the broker. +* `max_pending_messages_across_partitions`: + Set the max size of the queue holding the messages pending to receive + an acknowledgment across partitions from the broker. * `block_if_queue_full`: Set whether `send_async` operations should block when the outgoing message queue is full. * `message_routing_mode`: @@ -364,6 +368,7 @@ class Client: _check_type(int, send_timeout_millis, 'send_timeout_millis') _check_type(CompressionType, compression_type, 'compression_type') _check_type(int, max_pending_messages, 'max_pending_messages') +_check_type(int, max_pending_messages_across_partitions, 'max_pending_messages_across_partitions') _check_type(bool, block_if_queue_full, 'block_if_queue_full') _check_type(bool, batching_enabled, 'batching_enabled') _check_type(int, batching_max_messages, 'batching_max_messages') @@ -374,6 +379,7 @@ class Client: conf.send_timeout_millis(send_timeout_millis) conf.compression_type(compression_type) conf.max_pending_messages(max_pending_messages) + conf.max_pending_messages_across_partitions(max_pending_messages_across_partitions) conf.block_if_queue_full(block_if_queue_full) conf.batching_enabled(batching_enabled) conf.batching_max_messages(batching_max_messages) @@ -392,6 +398,7 @@ class Client: consumer_type=ConsumerType.Exclusive, message_listener=None, receiver_queue_size=1000, + max_total_receiver_queue_size_across_partitions=5, consumer_name=None, unacked_messages_timeout_ms=None, broker_consumer_stats_cache_time_ms=3, @@ -434,6 +441,9 @@ class Client: should not be interrupted when the consumer queue size is zero. The default value is 1000 messages and should work well for most use
[incubator-pulsar] branch master updated: [docker] Publish pulsar-all image to docker hub (#2361)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 777223d [docker] Publish pulsar-all image to docker hub (#2361)
777223d is described below
commit 777223d8031233ef03f61c41b8b99916c268b819
Author: Sijie Guo
AuthorDate: Mon Aug 13 09:07:44 2018 -0700
[docker] Publish pulsar-all image to docker hub (#2361)
### Motivation
We have `pulsar-all` image since 2.1.0 release. However this image is not
pushed to docker hub.
### Changes
Publish `pulsar-all` image to docker hub.
---
docker/publish.sh | 4
1 file changed, 4 insertions(+)
diff --git a/docker/publish.sh b/docker/publish.sh
index 411fd0d..f73bdac 100755
--- a/docker/publish.sh
+++ b/docker/publish.sh
@@ -63,18 +63,22 @@ set -x
set -e
docker tag pulsar:latest ${docker_registry_org}/pulsar:latest
+docker tag pulsar-all:latest ${docker_registry_org}/pulsar-all:latest
docker tag pulsar-grafana:latest ${docker_registry_org}/pulsar-grafana:latest
docker tag pulsar-dashboard:latest
${docker_registry_org}/pulsar-dashboard:latest
docker tag pulsar:latest ${docker_registry_org}/pulsar:$MVN_VERSION
+docker tag pulsar-all:latest ${docker_registry_org}/pulsar-all:$MVN_VERSION
docker tag pulsar-grafana:latest
${docker_registry_org}/pulsar-grafana:$MVN_VERSION
docker tag pulsar-dashboard:latest
${docker_registry_org}/pulsar-dashboard:$MVN_VERSION
# Push all images and tags
docker push ${docker_registry_org}/pulsar:latest
+docker push ${docker_registry_org}/pulsar-all:latest
docker push ${docker_registry_org}/pulsar-grafana:latest
docker push ${docker_registry_org}/pulsar-dashboard:latest
docker push ${docker_registry_org}/pulsar:$MVN_VERSION
+docker push ${docker_registry_org}/pulsar-all:$MVN_VERSION
docker push ${docker_registry_org}/pulsar-grafana:$MVN_VERSION
docker push ${docker_registry_org}/pulsar-dashboard:$MVN_VERSION
[incubator-pulsar] branch asf-site updated: Updated site at revision c0a5072
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/asf-site by this push:
new a50747e Updated site at revision c0a5072
a50747e is described below
commit a50747ec201e8731e13490af21be514ccdc3a309
Author: jenkins
AuthorDate: Mon Aug 13 08:35:56 2018 +
Updated site at revision c0a5072
---
content/swagger/swagger.json | 48 ++--
1 file changed, 24 insertions(+), 24 deletions(-)
diff --git a/content/swagger/swagger.json b/content/swagger/swagger.json
index 5a2eb43..c21c847 100644
--- a/content/swagger/swagger.json
+++ b/content/swagger/swagger.json
@@ -6364,10 +6364,10 @@
"type" : "string"
}
},
-"clientVersion" : {
+"connectedSince" : {
"type" : "string"
},
-"connectedSince" : {
+"clientVersion" : {
"type" : "string"
},
"address" : {
@@ -6687,20 +6687,32 @@
"type" : "number",
"format" : "double"
},
-"msgThroughputIn" : {
- "type" : "number",
- "format" : "double"
+"underLoaded" : {
+ "type" : "boolean"
+},
+"overLoaded" : {
+ "type" : "boolean"
+},
+"loadReportType" : {
+ "type" : "string"
},
"msgThroughputOut" : {
"type" : "number",
"format" : "double"
},
+"cpu" : {
+ "$ref" : "#/definitions/ResourceUsage"
+},
+"directMemory" : {
+ "$ref" : "#/definitions/ResourceUsage"
+},
"lastUpdate" : {
"type" : "integer",
"format" : "int64"
},
-"bandwidthIn" : {
- "$ref" : "#/definitions/ResourceUsage"
+"msgThroughputIn" : {
+ "type" : "number",
+ "format" : "double"
},
"bandwidthOut" : {
"$ref" : "#/definitions/ResourceUsage"
@@ -6708,20 +6720,8 @@
"memory" : {
"$ref" : "#/definitions/ResourceUsage"
},
-"cpu" : {
- "$ref" : "#/definitions/ResourceUsage"
-},
-"directMemory" : {
+"bandwidthIn" : {
"$ref" : "#/definitions/ResourceUsage"
-},
-"underLoaded" : {
- "type" : "boolean"
-},
-"overLoaded" : {
- "type" : "boolean"
-},
-"loadReportType" : {
- "type" : "string"
}
}
},
@@ -6881,13 +6881,13 @@
"type" : "number",
"format" : "double"
},
-"clientVersion" : {
+"producerName" : {
"type" : "string"
},
"connectedSince" : {
"type" : "string"
},
-"producerName" : {
+"clientVersion" : {
"type" : "string"
},
"address" : {
@@ -7585,13 +7585,13 @@
"type" : "string"
}
},
-"clientVersion" : {
+"producerName" : {
"type" : "string"
},
"connectedSince" : {
"type" : "string"
},
-"producerName" : {
+"clientVersion" : {
"type" : "string"
},
"address" : {
[incubator-pulsar] branch asf-site updated: Updated site at revision d248cee
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/asf-site by this push: new e31ed67 Updated site at revision d248cee e31ed67 is described below commit e31ed675c8283b12550922fb5df13df9c3ca657b Author: jenkins AuthorDate: Sat Aug 11 00:20:03 2018 + Updated site at revision d248cee --- .../docs/latest/adaptors/PulsarSpark/index.html| 8 ++--- .../docs/latest/adaptors/PulsarStorm/index.html| 6 ++-- content/docs/latest/admin-api/overview/index.html | 10 +++--- content/docs/latest/clients/Cpp/index.html | 8 ++--- content/docs/latest/clients/Java/index.html| 14 content/docs/latest/clients/Python/index.html | 10 +++--- content/docs/latest/clients/WebSocket/index.html | 8 ++--- content/docs/latest/clients/go/index.html | 6 ++-- .../docs/latest/cookbooks/Encryption/index.html| 6 ++-- .../latest/cookbooks/PartitionedTopics/index.html | 14 .../latest/cookbooks/RetentionExpiry/index.html| 14 .../docs/latest/cookbooks/compaction/index.html| 6 ++-- .../cookbooks/message-deduplication/index.html | 10 +++--- .../docs/latest/cookbooks/message-queue/index.html | 14 .../latest/cookbooks/tiered-storage/index.html | 4 +-- .../docs/latest/deployment/Kubernetes/index.html | 4 +-- .../docs/latest/deployment/aws-cluster/index.html | 4 +-- content/docs/latest/deployment/cluster/index.html | 4 +-- content/docs/latest/deployment/instance/index.html | 4 +-- .../ConceptsAndArchitecture/index.html | 4 +-- .../latest/getting-started/LocalCluster/index.html | 4 +-- .../latest/getting-started/Pulsar-2.0/index.html | 2 +- .../docs/latest/getting-started/docker/index.html | 4 +-- .../docs/latest/project/BinaryProtocol/index.html | 4 +-- content/docs/latest/project/CompileCpp/index.html | 8 ++--- .../docs/latest/project/SimulationTools/index.html | 2 +- .../docs/latest/project/schema-storage/index.html | 4 +-- content/docs/latest/reference/CliTools/index.html | 18 +- .../docs/latest/security/authorization/index.html | 10 +++--- content/docs/latest/security/encryption/index.html | 6 ++-- content/ja/adaptors/PulsarSpark/index.html | 8 ++--- content/ja/adaptors/PulsarStorm/index.html | 6 ++-- content/ja/admin/AdminInterface/index.html | 12 +++ content/ja/admin/Authz/index.html | 12 +++ content/ja/admin/ClustersBrokers/index.html| 6 ++-- content/ja/admin/PropertiesNamespaces/index.html | 6 ++-- content/ja/advanced/PartitionedTopics/index.html | 12 +++ content/ja/advanced/RetentionExpiry/index.html | 12 +++ content/ja/clients/Cpp/index.html | 6 ++-- content/ja/clients/Java/index.html | 8 ++--- content/ja/clients/Python/index.html | 8 ++--- content/ja/clients/WebSocket/index.html| 8 ++--- content/ja/deployment/InstanceSetup/index.html | 6 ++-- content/ja/deployment/Kubernetes/index.html| 4 +-- .../ConceptsAndArchitecture/index.html | 2 +- content/ja/getting-started/LocalCluster/index.html | 4 +-- content/ja/project/BinaryProtocol/index.html | 4 +-- content/ja/project/SimulationTools/index.html | 2 +- content/ja/reference/CliTools/index.html | 18 +- content/swagger/swagger.json | 40 +++--- 50 files changed, 202 insertions(+), 202 deletions(-) diff --git a/content/docs/latest/adaptors/PulsarSpark/index.html b/content/docs/latest/adaptors/PulsarSpark/index.html index a30d928..d130108 100644 --- a/content/docs/latest/adaptors/PulsarSpark/index.html +++ b/content/docs/latest/adaptors/PulsarSpark/index.html @@ -1079,9 +1079,9 @@ + Spark Streaming Pulsar receiver - Spark Streaming Pulsar receiver @@ -1325,9 +1325,9 @@ + Spark Streaming Pulsar receiver - Spark Streaming Pulsar receiver @@ -1537,8 +1537,6 @@ - - The Pulsar Java client @@ -1617,6 +1615,8 @@ + + Authentication and authorization in Pulsar diff --git a/content/docs/latest/adaptors/PulsarStorm/index.html b/content/docs/latest/adaptors/PulsarStorm/index.html index 3059cf5..67a4256 100644 --- a/content/docs/latest/adaptors/PulsarStorm/index.html +++ b/content/docs/latest/adaptors/PulsarStorm/index.html @@ -1083,9 +1083,9
[incubator-pulsar] branch master updated: C++ HTTP lookup works with TLS (#2314) (#2350)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new b6feac8 C++ HTTP lookup works with TLS (#2314) (#2350)
b6feac8 is described below
commit b6feac8180c978e3c59dfbcd4ef44494dd73b385
Author: Ivan Kelly
AuthorDate: Fri Aug 10 20:10:49 2018 +0200
C++ HTTP lookup works with TLS (#2314) (#2350)
* C++ HTTP lookup works with TLS (#2314)
Previously it did not take TLS into account at all. Now, if useTls is
configured, or the client it configured with a https:// url, the http
lookup will be configured to use TLS.
This patch also makes some changes to make sure the URL the broker
advertises for TLS starts with pulsar+ssl://.
* format
---
.../org/apache/pulsar/broker/PulsarService.java| 2 +-
pulsar-client-cpp/lib/ClientConnection.cc | 2 +-
pulsar-client-cpp/lib/ClientImpl.cc| 5 ++-
pulsar-client-cpp/lib/HTTPLookupService.cc | 43 +++---
pulsar-client-cpp/lib/HTTPLookupService.h | 3 ++
pulsar-client-cpp/lib/LookupDataResult.h | 8 ++--
pulsar-client-cpp/tests/AuthPluginTest.cc | 22 +++
.../pulsar/discovery/service/DiscoveryService.java | 2 +-
8 files changed, 73 insertions(+), 14 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java
index 93a045e..ff224bc 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java
@@ -799,7 +799,7 @@ public class PulsarService implements AutoCloseable {
public static String brokerUrlTls(ServiceConfiguration config) {
if (config.isTlsEnabled()) {
-return "pulsar://" + advertisedAddress(config) + ":" +
config.getBrokerServicePortTls();
+return "pulsar+ssl://" + advertisedAddress(config) + ":" +
config.getBrokerServicePortTls();
} else {
return "";
}
diff --git a/pulsar-client-cpp/lib/ClientConnection.cc
b/pulsar-client-cpp/lib/ClientConnection.cc
index da30707..7bafdf8 100644
--- a/pulsar-client-cpp/lib/ClientConnection.cc
+++ b/pulsar-client-cpp/lib/ClientConnection.cc
@@ -813,7 +813,7 @@ void ClientConnection::handleIncomingCommand() {
lookupResultPtr->setBrokerUrl(lookupTopicResponse.brokerserviceurl());
}
-
lookupResultPtr->setBrokerUrlSsl(lookupTopicResponse.brokerserviceurltls());
+
lookupResultPtr->setBrokerUrlTls(lookupTopicResponse.brokerserviceurltls());
lookupResultPtr->setAuthoritative(lookupTopicResponse.authoritative());
lookupResultPtr->setRedirect(lookupTopicResponse.response() ==
CommandLookupTopicResponse::Redirect);
diff --git a/pulsar-client-cpp/lib/ClientImpl.cc
b/pulsar-client-cpp/lib/ClientImpl.cc
index c7242c2..c31cf8e 100644
--- a/pulsar-client-cpp/lib/ClientImpl.cc
+++ b/pulsar-client-cpp/lib/ClientImpl.cc
@@ -391,8 +391,9 @@ Future
ClientImpl::getConnection(const std::str
void ClientImpl::handleLookup(Result result, LookupDataResultPtr data,
Promise
promise) {
if (data) {
-LOG_DEBUG("Getting connection to broker: " << data->getBrokerUrl());
-const std::string& logicalAddress = data->getBrokerUrl();
+const std::string& logicalAddress =
+clientConfiguration_.isUseTls() ? data->getBrokerUrlTls() :
data->getBrokerUrl();
+LOG_DEBUG("Getting connection to broker: " << logicalAddress);
const std::string& physicalAddress =
data->shouldProxyThroughServiceUrl() ? serviceUrl_ :
logicalAddress;
Future future =
diff --git a/pulsar-client-cpp/lib/HTTPLookupService.cc
b/pulsar-client-cpp/lib/HTTPLookupService.cc
index fe27e8d..b3cb023 100644
--- a/pulsar-client-cpp/lib/HTTPLookupService.cc
+++ b/pulsar-client-cpp/lib/HTTPLookupService.cc
@@ -39,7 +39,10 @@ HTTPLookupService::HTTPLookupService(const std::string
,
const AuthenticationPtr )
:
executorProvider_(boost::make_shared(NUMBER_OF_LOOKUP_THREADS)),
authenticationPtr_(authData),
-
lookupTimeoutInSeconds_(clientConfiguration.getOperationTimeoutSeconds()) {
+
lookupTimeoutInSeconds_(clientConfiguration.getOperationTimeoutSeconds()),
+ isUseTls_(clientConfiguration.isUseTls()),
+ tlsAllowInsecure_(clientConfiguration.isTlsAl
[incubator-pulsar] branch master updated: Fix NPE when splitting and unloading bundle (#2348)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new a850934 Fix NPE when splitting and unloading bundle (#2348)
a850934 is described below
commit a8509341cc8373e849a52bffdb140184371509a3
Author: massakam
AuthorDate: Fri Aug 10 13:38:52 2018 +0900
Fix NPE when splitting and unloading bundle (#2348)
---
.../pulsar/broker/namespace/NamespaceService.java | 6 ++--
.../broker/namespace/NamespaceServiceTest.java | 42 ++
2 files changed, 45 insertions(+), 3 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java
index bf44519..d9a1ccf 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java
@@ -593,7 +593,7 @@ public class NamespaceService {
boolean unload,
AtomicInteger counter,
CompletableFuture unloadFuture) {
-CompletableFuture updateFuture = new
CompletableFuture<>();
+CompletableFuture> updateFuture = new
CompletableFuture<>();
final Pair> splittedBundles =
bundleFactory.splitBundles(bundle,
2 /* by default split into 2 */);
@@ -622,7 +622,7 @@ public class NamespaceService {
// namespace bundle
bundleFactory.invalidateBundleCache(bundle.getNamespaceObject());
-updateFuture.complete(splittedBundles.getLeft());
+updateFuture.complete(splittedBundles.getRight());
} else if (rc == Code.BADVERSION.intValue()) {
KeeperException keeperException =
KeeperException.create(KeeperException.Code.get(rc));
String msg = format("failed to update namespace
policies [%s], NamespaceBundle: %s " +
@@ -680,7 +680,7 @@ public class NamespaceService {
if (unload) {
// unload new split bundles
-r.getBundles().forEach(splitBundle -> {
+r.forEach(splitBundle -> {
try {
unloadNamespaceBundle(splitBundle);
} catch (Exception e) {
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/namespace/NamespaceServiceTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/namespace/NamespaceServiceTest.java
index a2ef397..2c51e16 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/namespace/NamespaceServiceTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/namespace/NamespaceServiceTest.java
@@ -429,6 +429,48 @@ public class NamespaceServiceTest extends BrokerTestBase {
}
+@Test
+public void testRemoveOwnershipAndSplitBundle() throws Exception {
+OwnershipCache ownershipCache =
spy(pulsar.getNamespaceService().getOwnershipCache());
+
doNothing().when(ownershipCache).disableOwnership(any(NamespaceBundle.class));
+
+Field ownership =
NamespaceService.class.getDeclaredField("ownershipCache");
+ownership.setAccessible(true);
+ownership.set(pulsar.getNamespaceService(), ownershipCache);
+
+NamespaceService namespaceService = pulsar.getNamespaceService();
+NamespaceName nsname = NamespaceName.get("pulsar/global/ns1");
+TopicName topicName =
TopicName.get("persistent://pulsar/global/ns1/topic-1");
+NamespaceBundles bundles =
namespaceService.getNamespaceBundleFactory().getBundles(nsname);
+NamespaceBundle originalBundle = bundles.findBundle(topicName);
+
+CompletableFuture result1 =
namespaceService.splitAndOwnBundle(originalBundle, false);
+try {
+result1.get();
+} catch (Exception e) {
+fail("split bundle faild", e);
+}
+
+NamespaceBundles updatedNsBundles =
namespaceService.getNamespaceBundleFactory().getBundles(nsname);
+assertNotNull(updatedNsBundles);
+NamespaceBundle splittedBundle =
updatedNsBundles.findBundle(topicName);
+
+updatedNsBundles.getBundles().stream().filter(bundle ->
!bundle.equals(splittedBundle)).forEach(bundle -> {
+try {
+ownershipCache.removeOwnership(bundle).get();
+} catch (Exception e) {
+fail("faild to remove ownership", e);
+}
+});
+
+CompletableFuture result2 =
namespa
[incubator-pulsar] branch master updated: Re-added tracking id to new website (#2349)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 270132c Re-added tracking id to new website (#2349)
270132c is described below
commit 270132c55eaa6170edcb3448efe6472d5b184ac8
Author: Matteo Merli
AuthorDate: Thu Aug 9 12:45:09 2018 -0700
Re-added tracking id to new website (#2349)
---
site2/website/siteConfig.js | 1 +
1 file changed, 1 insertion(+)
diff --git a/site2/website/siteConfig.js b/site2/website/siteConfig.js
index b5f3e50..6a3196c 100644
--- a/site2/website/siteConfig.js
+++ b/site2/website/siteConfig.js
@@ -118,6 +118,7 @@ const siteConfig = {
facetFilters: ['language:LANGUAGE', 'version:VERSION'],
},
},
+ gaTrackingId: 'UA-102219959-1',
/* colors for website */
colors: {
[incubator-pulsar] branch asf-site updated: Updated site at revision c8e6fce
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/asf-site by this push: new fc62255 Updated site at revision c8e6fce fc62255 is described below commit fc62255d2cf7f6e9d56d197eccd03716a84b4bff Author: jenkins AuthorDate: Wed Aug 8 02:16:40 2018 + Updated site at revision c8e6fce --- content/blog/2018/08/06/Apache-Pulsar-2-1-0.html | 12 .../blog/2018/08/06/Apache-Pulsar-2-1-0/index.html | 12 content/docs/en/next/client-libraries-cpp.html | 8 ++--- .../docs/en/next/client-libraries-cpp/index.html | 8 ++--- content/docs/ja/next/client-libraries-cpp.html | 8 ++--- .../docs/ja/next/client-libraries-cpp/index.html | 8 ++--- .../docs/latest/adaptors/PulsarSpark/index.html| 10 +++--- .../docs/latest/adaptors/PulsarStorm/index.html| 8 ++--- content/docs/latest/admin-api/overview/index.html | 18 +-- content/docs/latest/clients/Cpp/index.html | 8 ++--- content/docs/latest/clients/Java/index.html| 16 +- content/docs/latest/clients/Python/index.html | 10 +++--- content/docs/latest/clients/WebSocket/index.html | 8 ++--- content/docs/latest/clients/go/index.html | 6 ++-- .../docs/latest/cookbooks/Encryption/index.html| 6 ++-- .../latest/cookbooks/PartitionedTopics/index.html | 16 +- .../latest/cookbooks/RetentionExpiry/index.html| 16 +- .../docs/latest/cookbooks/compaction/index.html| 8 ++--- .../cookbooks/message-deduplication/index.html | 12 .../docs/latest/cookbooks/message-queue/index.html | 16 +- .../latest/cookbooks/tiered-storage/index.html | 6 ++-- .../docs/latest/deployment/Kubernetes/index.html | 4 +-- .../docs/latest/deployment/aws-cluster/index.html | 6 ++-- content/docs/latest/deployment/cluster/index.html | 6 ++-- content/docs/latest/deployment/instance/index.html | 6 ++-- .../ConceptsAndArchitecture/index.html | 4 +-- .../latest/getting-started/LocalCluster/index.html | 4 +-- .../latest/getting-started/Pulsar-2.0/index.html | 2 +- .../docs/latest/getting-started/docker/index.html | 4 +-- .../docs/latest/project/BinaryProtocol/index.html | 4 +-- content/docs/latest/project/CompileCpp/index.html | 8 ++--- .../docs/latest/project/SimulationTools/index.html | 2 +- .../docs/latest/project/schema-storage/index.html | 4 +-- content/docs/latest/reference/CliTools/index.html | 22 ++--- .../docs/latest/security/authorization/index.html | 14 - content/docs/latest/security/encryption/index.html | 6 ++-- content/docs/zh-CN/next/client-libraries-cpp.html | 8 ++--- .../zh-CN/next/client-libraries-cpp/index.html | 8 ++--- content/ja/adaptors/PulsarSpark/index.html | 8 ++--- content/ja/adaptors/PulsarStorm/index.html | 6 ++-- content/ja/admin/AdminInterface/index.html | 12 content/ja/admin/Authz/index.html | 12 content/ja/admin/ClustersBrokers/index.html| 6 ++-- content/ja/admin/PropertiesNamespaces/index.html | 6 ++-- content/ja/advanced/PartitionedTopics/index.html | 12 content/ja/advanced/RetentionExpiry/index.html | 12 content/ja/clients/Cpp/index.html | 6 ++-- content/ja/clients/Java/index.html | 8 ++--- content/ja/clients/Python/index.html | 8 ++--- content/ja/clients/WebSocket/index.html| 8 ++--- content/ja/deployment/InstanceSetup/index.html | 6 ++-- content/ja/deployment/Kubernetes/index.html| 4 +-- .../ConceptsAndArchitecture/index.html | 2 +- content/ja/getting-started/LocalCluster/index.html | 4 +-- content/ja/project/BinaryProtocol/index.html | 4 +-- content/ja/project/SimulationTools/index.html | 2 +- content/ja/reference/CliTools/index.html | 18 +-- content/swagger/swagger.json | 36 +++--- 58 files changed, 256 insertions(+), 256 deletions(-) diff --git a/content/blog/2018/08/06/Apache-Pulsar-2-1-0.html b/content/blog/2018/08/06/Apache-Pulsar-2-1-0.html index e0a6f8d..b0e8804 100644 --- a/content/blog/2018/08/06/Apache-Pulsar-2-1-0.html +++ b/content/blog/2018/08/06/Apache-Pulsar-2-1-0.html @@ -75,19 +75,19 @@ When the cluster starts to run out of space, you just add another storage node, pickup the new storage nodes and start using them without rebalancing partitions. However, this can start to get expensive after a while. Pulsar mitigates this cost/size trade-off by providing Tiered Storage. Tiered Storage turns your Pulsar topics into real infinite streams, by offloading older segments into a long term storage, such as AWS S3, GCS and HDFS, which is designed for storing cold data
[incubator-pulsar] branch asf-site updated: Updated site at revision 14765b2
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/asf-site by this push: new 55347f8 Updated site at revision 14765b2 55347f8 is described below commit 55347f88eea9faffe33c2e9010036c169040c792 Author: jenkins AuthorDate: Tue Aug 7 08:45:31 2018 + Updated site at revision 14765b2 --- .../docs/latest/adaptors/PulsarSpark/index.html| 8 ++-- .../docs/latest/adaptors/PulsarStorm/index.html| 6 +-- content/docs/latest/admin-api/overview/index.html | 10 ++--- content/docs/latest/clients/Cpp/index.html | 8 ++-- content/docs/latest/clients/Java/index.html| 14 +++ content/docs/latest/clients/Python/index.html | 10 ++--- content/docs/latest/clients/WebSocket/index.html | 8 ++-- content/docs/latest/clients/go/index.html | 6 +-- .../docs/latest/cookbooks/Encryption/index.html| 6 +-- .../latest/cookbooks/PartitionedTopics/index.html | 14 +++ .../latest/cookbooks/RetentionExpiry/index.html| 14 +++ .../docs/latest/cookbooks/compaction/index.html| 6 +-- .../cookbooks/message-deduplication/index.html | 10 ++--- .../docs/latest/cookbooks/message-queue/index.html | 14 +++ .../latest/cookbooks/tiered-storage/index.html | 4 +- .../docs/latest/deployment/Kubernetes/index.html | 4 +- .../docs/latest/deployment/aws-cluster/index.html | 4 +- content/docs/latest/deployment/cluster/index.html | 4 +- content/docs/latest/deployment/instance/index.html | 4 +- .../ConceptsAndArchitecture/index.html | 4 +- .../latest/getting-started/LocalCluster/index.html | 4 +- .../latest/getting-started/Pulsar-2.0/index.html | 2 +- .../docs/latest/getting-started/docker/index.html | 4 +- .../docs/latest/project/BinaryProtocol/index.html | 4 +- content/docs/latest/project/CompileCpp/index.html | 8 ++-- .../docs/latest/project/SimulationTools/index.html | 2 +- .../docs/latest/project/schema-storage/index.html | 4 +- content/docs/latest/reference/CliTools/index.html | 18 - .../docs/latest/security/authorization/index.html | 10 ++--- content/docs/latest/security/encryption/index.html | 6 +-- content/ja/adaptors/PulsarSpark/index.html | 8 ++-- content/ja/adaptors/PulsarStorm/index.html | 6 +-- content/ja/admin/AdminInterface/index.html | 12 +++--- content/ja/admin/Authz/index.html | 12 +++--- content/ja/admin/ClustersBrokers/index.html| 6 +-- content/ja/admin/PropertiesNamespaces/index.html | 6 +-- content/ja/advanced/PartitionedTopics/index.html | 12 +++--- content/ja/advanced/RetentionExpiry/index.html | 12 +++--- content/ja/clients/Cpp/index.html | 6 +-- content/ja/clients/Java/index.html | 8 ++-- content/ja/clients/Python/index.html | 8 ++-- content/ja/clients/WebSocket/index.html| 8 ++-- content/ja/deployment/InstanceSetup/index.html | 6 +-- content/ja/deployment/Kubernetes/index.html| 4 +- .../ConceptsAndArchitecture/index.html | 2 +- content/ja/getting-started/LocalCluster/index.html | 4 +- content/ja/project/BinaryProtocol/index.html | 4 +- content/ja/project/SimulationTools/index.html | 2 +- content/ja/reference/CliTools/index.html | 18 - content/swagger/swagger.json | 44 +++--- content/zh-CN/download.html| 2 +- content/zh-CN/download/index.html | 2 +- content/zh-CN/events.html | 2 +- content/zh-CN/events/index.html| 2 +- content/zh-CN/resources.html | 2 +- content/zh-CN/resources/index.html | 2 +- 56 files changed, 210 insertions(+), 210 deletions(-) diff --git a/content/docs/latest/adaptors/PulsarSpark/index.html b/content/docs/latest/adaptors/PulsarSpark/index.html index a30d928..d130108 100644 --- a/content/docs/latest/adaptors/PulsarSpark/index.html +++ b/content/docs/latest/adaptors/PulsarSpark/index.html @@ -1079,9 +1079,9 @@ + Spark Streaming Pulsar receiver - Spark Streaming Pulsar receiver @@ -1325,9 +1325,9 @@ + Spark Streaming Pulsar receiver - Spark Streaming Pulsar receiver @@ -1537,8 +1537,6 @@ - - The Pulsar Java client @@ -1617,6 +1615,8 @@ + + Authentication and authorization in Pulsar diff --git
[incubator-pulsar] branch master updated: Fixed refcounting when processing entries from sql (#2316)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 6b7a76f Fixed refcounting when processing entries from sql (#2316)
6b7a76f is described below
commit 6b7a76fba4b1c08bf845409c6e4cabf48a454a5e
Author: Matteo Merli
AuthorDate: Tue Aug 7 09:12:47 2018 +0900
Fixed refcounting when processing entries from sql (#2316)
---
.../java/org/apache/pulsar/client/impl/MessageParser.java| 5 -
.../org/apache/pulsar/sql/presto/PulsarRecordCursor.java | 12 +---
2 files changed, 5 insertions(+), 12 deletions(-)
diff --git
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/MessageParser.java
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/MessageParser.java
index 8cf0328..b95b22d 100644
---
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/MessageParser.java
+++
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/MessageParser.java
@@ -95,15 +95,10 @@ public class MessageParser {
if (numMessages == 1 && !msgMetadata.hasNumMessagesInBatch()) {
final MessageImpl message = new MessageImpl<>(msgId,
msgMetadata, uncompressedPayload, null, null);
processor.process(msgId, message, uncompressedPayload);
-
-uncompressedPayload.release();
-
} else {
// handle batch message enqueuing; uncompressed payload has
all messages in batch
receiveIndividualMessagesFromBatch(msgMetadata,
uncompressedPayload, messageId, null, -1, processor);
-uncompressedPayload.release();
}
-
} finally {
if (uncompressedPayload != null) {
uncompressedPayload.release();
diff --git
a/pulsar-sql/presto-pulsar/src/main/java/org/apache/pulsar/sql/presto/PulsarRecordCursor.java
b/pulsar-sql/presto-pulsar/src/main/java/org/apache/pulsar/sql/presto/PulsarRecordCursor.java
index 618b33b..3523228 100644
---
a/pulsar-sql/presto-pulsar/src/main/java/org/apache/pulsar/sql/presto/PulsarRecordCursor.java
+++
b/pulsar-sql/presto-pulsar/src/main/java/org/apache/pulsar/sql/presto/PulsarRecordCursor.java
@@ -52,7 +52,6 @@ import java.util.List;
import java.util.Map;
import java.util.Queue;
import java.util.concurrent.TimeUnit;
-import java.util.function.Consumer;
import static com.facebook.presto.spi.StandardErrorCode.NOT_SUPPORTED;
import static com.facebook.presto.spi.type.BigintType.BIGINT;
@@ -202,10 +201,9 @@ public class PulsarRecordCursor implements RecordCursor {
throw new RuntimeException(e);
}
-newEntries.forEach(new Consumer() {
-@Override
-public void accept(Entry entry) {
-completedBytes += entry.getData().length;
+newEntries.forEach(entry -> {
+try {
+completedBytes += entry.getDataBuffer().readableBytes();
// filter entries that is not part of my split
if (((PositionImpl)
entry.getPosition()).compareTo(pulsarSplit.getEndPosition()) < 0) {
try {
@@ -217,9 +215,9 @@ public class PulsarRecordCursor implements RecordCursor {
log.error(e, "Failed to parse message from pulsar
topic %s", topicName.toString());
throw new RuntimeException(e);
}
-} else {
-entry.release();
}
+} finally {
+entry.release();
}
});
}
[incubator-pulsar] branch master updated: Attempt to fix flakyness of BrokerBkEnsemblesTests.testSkipCorruptDataLedger (#2318)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 9213e58 Attempt to fix flakyness of
BrokerBkEnsemblesTests.testSkipCorruptDataLedger (#2318)
9213e58 is described below
commit 9213e58a74dc26096cc132eec709ae3d95ade095
Author: Matteo Merli
AuthorDate: Tue Aug 7 09:11:28 2018 +0900
Attempt to fix flakyness of
BrokerBkEnsemblesTests.testSkipCorruptDataLedger (#2318)
---
.../broker/service/BrokerBkEnsemblesTests.java | 22 +++---
1 file changed, 15 insertions(+), 7 deletions(-)
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerBkEnsemblesTests.java
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerBkEnsemblesTests.java
index ce023c3..94b2226 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerBkEnsemblesTests.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerBkEnsemblesTests.java
@@ -19,6 +19,9 @@
package org.apache.pulsar.broker.service;
import static
org.apache.pulsar.broker.auth.MockedPulsarServiceBaseTest.retryStrategically;
+import static org.testng.Assert.assertEquals;
+
+import com.google.common.collect.Sets;
import java.lang.reflect.Field;
import java.net.URL;
@@ -54,8 +57,6 @@ import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;
-import com.google.common.collect.Sets;
-
/**
*/
public class BrokerBkEnsemblesTests {
@@ -236,8 +237,11 @@ public class BrokerBkEnsemblesTests {
*
* @throws Exception
*/
-@Test(timeOut = 6000)
+@Test
public void testSkipCorruptDataLedger() throws Exception {
+// Ensure intended state for autoSkipNonRecoverableData
+
admin.brokers().updateDynamicConfiguration("autoSkipNonRecoverableData",
"false");
+
PulsarClient client =
PulsarClient.builder().serviceUrl(adminUrl.toString()).statsInterval(0,
TimeUnit.SECONDS)
.build();
@@ -246,9 +250,13 @@ public class BrokerBkEnsemblesTests {
final int totalDataLedgers = 5;
final int entriesPerLedger = totalMessages / totalDataLedgers;
-admin.namespaces().createNamespace(ns1);
+try {
+admin.namespaces().createNamespace(ns1);
+} catch (Exception e) {
-final String topic1 = "persistent://" + ns1 + "/my-topic";
+}
+
+final String topic1 = "persistent://" + ns1 + "/my-topic-" +
System.currentTimeMillis();
// Create subscription
Consumer consumer =
client.newConsumer().topic(topic1).subscriptionName("my-subscriber-name")
@@ -287,6 +295,7 @@ public class BrokerBkEnsemblesTests {
// (2) delete first 4 data-ledgers
ledgerInfo.entrySet().forEach(entry -> {
if (!entry.equals(lastLedger)) {
+assertEquals(entry.getValue().getEntries(), entriesPerLedger);
try {
bookKeeper.deleteLedger(entry.getKey());
} catch (Exception e) {
@@ -322,7 +331,7 @@ public class BrokerBkEnsemblesTests {
// (5) consumer will be able to consume 20 messages from last
non-deleted ledger
consumer =
client.newConsumer().topic(topic1).subscriptionName("my-subscriber-name").subscribe();
for (int i = 0; i < entriesPerLedger; i++) {
-msg = consumer.receive(5, TimeUnit.SECONDS);
+msg = consumer.receive();
System.out.println(i);
consumer.acknowledge(msg);
}
@@ -330,7 +339,6 @@ public class BrokerBkEnsemblesTests {
producer.close();
consumer.close();
client.close();
-
}
private static final Logger LOG =
LoggerFactory.getLogger(BrokerBkEnsemblesTests.class);
[incubator-pulsar] branch asf-site updated: Updated site at revision 3f8944f
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/asf-site by this push:
new f54a79b Updated site at revision 3f8944f
f54a79b is described below
commit f54a79b4fad5e6211917ff7fab9b5be4dcd1622a
Author: jenkins
AuthorDate: Mon Aug 6 06:45:40 2018 +
Updated site at revision 3f8944f
---
content/swagger/swagger.json | 14 +++---
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/content/swagger/swagger.json b/content/swagger/swagger.json
index 3e8f88a..8598603 100644
--- a/content/swagger/swagger.json
+++ b/content/swagger/swagger.json
@@ -6418,19 +6418,16 @@
"type" : "number",
"format" : "double"
},
-"memory" : {
- "$ref" : "#/definitions/ResourceUsage"
-},
-"cpu" : {
+"bandwidthIn" : {
"$ref" : "#/definitions/ResourceUsage"
},
-"directMemory" : {
+"bandwidthOut" : {
"$ref" : "#/definitions/ResourceUsage"
},
-"bandwidthIn" : {
+"memory" : {
"$ref" : "#/definitions/ResourceUsage"
},
-"bandwidthOut" : {
+"directMemory" : {
"$ref" : "#/definitions/ResourceUsage"
},
"lastUpdate" : {
@@ -6451,6 +6448,9 @@
"overLoaded" : {
"type" : "boolean"
},
+"cpu" : {
+ "$ref" : "#/definitions/ResourceUsage"
+},
"loadReportType" : {
"type" : "string"
}
[incubator-pulsar] branch asf-site updated: Updated site at revision 3f8944f
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/asf-site by this push: new b346af5 Updated site at revision 3f8944f b346af5 is described below commit b346af5a5264cff1f412afa0dd26e39f206608a7 Author: jenkins AuthorDate: Mon Aug 6 00:52:42 2018 + Updated site at revision 3f8944f --- content/docs/en/client-libraries.html | 42 +- content/docs/en/client-libraries/index.html| 42 +- content/docs/en/next/client-libraries.html | 19 +- content/docs/en/next/client-libraries/index.html | 19 +- content/docs/ja/client-libraries.html | 42 +- content/docs/ja/client-libraries/index.html| 42 +- content/docs/ja/next/client-libraries.html | 19 +- content/docs/ja/next/client-libraries/index.html | 19 +- content/docs/zh-CN/client-libraries.html | 42 +- content/docs/zh-CN/client-libraries/index.html | 42 +- content/docs/zh-CN/next/client-libraries.html | 19 +- .../docs/zh-CN/next/client-libraries/index.html| 19 +- content/swagger/swagger.json | 20 +-- 13 files changed, 364 insertions(+), 22 deletions(-) diff --git a/content/docs/en/client-libraries.html b/content/docs/en/client-libraries.html index 8d6f07c..07e8e16 100644 --- a/content/docs/en/client-libraries.html +++ b/content/docs/en/client-libraries.html @@ -60,7 +60,47 @@ The Pulsar C++ client. There are also http://www.stack.nl/~dimitri/doxygen/;>Doxygen-generated API docs for the C++ client here. -← Start a standalone cluster with DockerPulsar Overview →Java clientGo clientPython clientC++ clientFeatureJavaC++GoPythonWebSocket + + +Partitioned topics✅ +Batching✅ +Compression✅ +TLS✅ +Authentication✅ +Reader API✅ +Proxy Support✅ +Effectively-Once❌ +Schema✅❌ +Consumer seek✅✅✅❌ +Multi-topics consumer✅❌ +Topics regex consumer✅❌ +Compacted topics✅✅✅❌ +User defined properties producer/consumer✅❌ +Reader hasMessageAvailable✅✅✅❌ +Hostname verification✅❌ + + +https://github.com/Comcast/pulsar-client-go;>pulsar-client-gohttps://github.com/Comcast;>Comcasthttps://opensource.org/licenses/Apache-2.0;>https://img.shields.io/badge/License-Apache%202.0-blue.svg; alt="License">A native golang client +Gohttps://github.com/t2y/go-pulsar;>go-pulsarhttps://github.com/t2y;>t2yhttps://opensource.org/licenses/Apache-2.0;>https://img.shields.io/badge/License-Apache%202.0-blue.svg; alt="License"> +Scalahttps://github.com/sksamuel/pulsar4s;>pulsar4shttps://github.com/sksamuel;>sksamuelhttps://opensource.org/licenses/Apache-2.0;>https://img.shields.io/badge/License-Apache%202.0-blue.svg; alt="License">Idomatic, typesafe, and reactive Scala client for Apache Pulsar + + +← Start a standalone cluster with DockerPulsar Overview →Java clientGo clientPython clientC++ client' + diff --git a/content/docs/en/client-libraries/index.html b/content/docs/en/client-libraries/index.html index 8d6f07c..07e8e16 100644 --- a/content/docs/en/client-libraries/index.html +++ b/content/docs/en/client-libraries/index.html @@ -60,7 +60,47 @@ The Pulsar C++ client. There are also http://www.stack.nl/~dimitri/doxygen/;>Doxygen-generated API docs for the C++ client here. -← Start a standalone cluster with DockerPulsar Overview →Java clientGo clientPython clientC++ clientFeatureJavaC++GoPythonWebSocket + + +Partitioned topics✅ +Batching✅ +Compression✅ +TLS✅ +Authentication✅ +Reader API✅ +Proxy Support✅ +Effectively-Once❌ +Schema✅❌ +Consumer seek✅✅✅❌ +Multi-topics consumer✅❌ +Topics regex consumer✅❌ +Compacted topics✅✅✅❌ +User defined properties producer/consumer✅❌ +Reader hasMessageAvailable✅✅✅❌ +Hostname verification✅❌ + + +https://github.com/Comcast/pulsar-client-go;>pulsar-client-gohttps://github.com/Comcast;>Comcasthttps://opensource.org/licenses/Apache-2.0;>https://img.shields.io/badge/License-Apache%202.0-blue.svg; alt="License">A native golang client +Gohttps://github.com/t2y/go-pulsar;>go-pulsarhttps://github.com/t2y;>t2yhttps://opensource.org/licenses/Apache-2.0;>https://img.shields.io/badge/License-Apache%202.0-blue.svg; alt="License"> +Scalahttps://github.com/sksamuel/pulsar4s;>pulsar4shttps://github.com/sksamuel;>sksamuelhttps://opensource.org/licenses/Apache-2.0;>https://img.shields.io/badge/License-Apache%202.0-blue.svg; alt="License">Idomatic, typesafe, and reactive Scala client for Apache Pulsar + + +← Start a standalone cluster with DockerPulsar Overview →Java clientGo clientPython client
[incubator-pulsar] branch asf-site updated: Updated site at revision 41a973b
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/asf-site by this push: new c8990fb Updated site at revision 41a973b c8990fb is described below commit c8990fb6bbd3903b0a96ceb658bf0d12e5183b5d Author: jenkins AuthorDate: Mon Aug 6 00:18:59 2018 + Updated site at revision 41a973b --- content/contact.html | 2 +- content/contact/index.html | 2 +- .../docs/latest/adaptors/PulsarSpark/index.html| 8 ++--- .../docs/latest/adaptors/PulsarStorm/index.html| 6 ++-- content/docs/latest/admin-api/overview/index.html | 10 +++--- content/docs/latest/clients/Cpp/index.html | 8 ++--- content/docs/latest/clients/Java/index.html| 14 content/docs/latest/clients/Python/index.html | 10 +++--- content/docs/latest/clients/WebSocket/index.html | 8 ++--- content/docs/latest/clients/go/index.html | 6 ++-- .../docs/latest/cookbooks/Encryption/index.html| 6 ++-- .../latest/cookbooks/PartitionedTopics/index.html | 14 .../latest/cookbooks/RetentionExpiry/index.html| 14 .../docs/latest/cookbooks/compaction/index.html| 6 ++-- .../cookbooks/message-deduplication/index.html | 10 +++--- .../docs/latest/cookbooks/message-queue/index.html | 14 .../latest/cookbooks/tiered-storage/index.html | 4 +-- .../docs/latest/deployment/Kubernetes/index.html | 4 +-- .../docs/latest/deployment/aws-cluster/index.html | 4 +-- content/docs/latest/deployment/cluster/index.html | 4 +-- content/docs/latest/deployment/instance/index.html | 4 +-- .../ConceptsAndArchitecture/index.html | 4 +-- .../latest/getting-started/LocalCluster/index.html | 4 +-- .../latest/getting-started/Pulsar-2.0/index.html | 2 +- .../docs/latest/getting-started/docker/index.html | 4 +-- .../docs/latest/project/BinaryProtocol/index.html | 4 +-- content/docs/latest/project/CompileCpp/index.html | 8 ++--- .../docs/latest/project/SimulationTools/index.html | 2 +- .../docs/latest/project/schema-storage/index.html | 4 +-- content/docs/latest/reference/CliTools/index.html | 18 +- .../docs/latest/security/authorization/index.html | 10 +++--- content/docs/latest/security/encryption/index.html | 6 ++-- content/download.html | 2 +- content/download/index.html| 2 +- content/en/contact.html| 2 +- content/en/contact/index.html | 2 +- content/en/download.html | 2 +- content/en/download/index.html | 2 +- content/en/versions.html | 2 +- content/en/versions/index.html | 2 +- content/ja/adaptors/PulsarSpark/index.html | 8 ++--- content/ja/adaptors/PulsarStorm/index.html | 6 ++-- content/ja/admin/AdminInterface/index.html | 12 +++ content/ja/admin/Authz/index.html | 12 +++ content/ja/admin/ClustersBrokers/index.html| 6 ++-- content/ja/admin/PropertiesNamespaces/index.html | 6 ++-- content/ja/advanced/PartitionedTopics/index.html | 12 +++ content/ja/advanced/RetentionExpiry/index.html | 12 +++ content/ja/clients/Cpp/index.html | 6 ++-- content/ja/clients/Java/index.html | 8 ++--- content/ja/clients/Python/index.html | 8 ++--- content/ja/clients/WebSocket/index.html| 8 ++--- content/ja/contact.html| 2 +- content/ja/contact/index.html | 2 +- content/ja/deployment/InstanceSetup/index.html | 6 ++-- content/ja/deployment/Kubernetes/index.html| 4 +-- content/ja/download.html | 2 +- content/ja/download/index.html | 2 +- .../ConceptsAndArchitecture/index.html | 2 +- content/ja/getting-started/LocalCluster/index.html | 4 +-- content/ja/project/BinaryProtocol/index.html | 4 +-- content/ja/project/SimulationTools/index.html | 2 +- content/ja/reference/CliTools/index.html | 18 +- content/ja/versions.html | 2 +- content/ja/versions/index.html | 2 +- content/swagger/swagger.json | 42 +++--- content/versions.html | 2 +- content/versions/index.html| 2 +- content/zh-CN/contact.html | 2 +- content/zh-CN/contact/index.html | 2 +- content/zh-CN/download.html| 2 +- content/zh-CN/download/index.html | 2 +- content/zh-CN/versions.html
[incubator-pulsar] branch asf-site updated: Updated site at revision 28fe0a1
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/asf-site by this push:
new cf09773 Updated site at revision 28fe0a1
cf09773 is described below
commit cf0977389f922345ca20dadf598467652559949f
Author: jenkins
AuthorDate: Sat Aug 4 08:33:16 2018 +
Updated site at revision 28fe0a1
---
.../api/cpp/classpulsar_1_1_client-members.html| 6 +-
content/api/cpp/classpulsar_1_1_client.html| 80 +-
...spulsar_1_1_consumer_configuration-members.html | 30
.../classpulsar_1_1_consumer_configuration.html| 28
content/api/cpp/functions.html | 9 +++
content/api/cpp/functions_func.html| 9 +++
content/api/cpp/search/all_c.js| 5 +-
content/api/cpp/search/functions_b.js | 5 +-
content/css/main.css | 2 +-
.../docs/latest/adaptors/PulsarSpark/index.html| 8 +--
.../docs/latest/adaptors/PulsarStorm/index.html| 6 +-
content/docs/latest/admin-api/overview/index.html | 10 +--
content/docs/latest/clients/Cpp/index.html | 8 +--
content/docs/latest/clients/Java/index.html| 14 ++--
content/docs/latest/clients/Python/index.html | 10 +--
content/docs/latest/clients/WebSocket/index.html | 8 +--
content/docs/latest/clients/go/index.html | 6 +-
.../docs/latest/cookbooks/Encryption/index.html| 6 +-
.../latest/cookbooks/PartitionedTopics/index.html | 14 ++--
.../latest/cookbooks/RetentionExpiry/index.html| 14 ++--
.../docs/latest/cookbooks/compaction/index.html| 6 +-
.../cookbooks/message-deduplication/index.html | 10 +--
.../docs/latest/cookbooks/message-queue/index.html | 14 ++--
.../latest/cookbooks/tiered-storage/index.html | 4 +-
.../docs/latest/deployment/Kubernetes/index.html | 4 +-
.../docs/latest/deployment/aws-cluster/index.html | 4 +-
content/docs/latest/deployment/cluster/index.html | 4 +-
content/docs/latest/deployment/instance/index.html | 4 +-
.../ConceptsAndArchitecture/index.html | 4 +-
.../latest/getting-started/LocalCluster/index.html | 4 +-
.../latest/getting-started/Pulsar-2.0/index.html | 2 +-
.../docs/latest/getting-started/docker/index.html | 4 +-
.../docs/latest/project/BinaryProtocol/index.html | 4 +-
content/docs/latest/project/CompileCpp/index.html | 8 +--
.../docs/latest/project/SimulationTools/index.html | 2 +-
.../docs/latest/project/schema-storage/index.html | 4 +-
content/docs/latest/reference/CliTools/index.html | 18 ++---
.../docs/latest/security/authorization/index.html | 10 +--
content/docs/latest/security/encryption/index.html | 6 +-
content/ja/adaptors/PulsarSpark/index.html | 8 +--
content/ja/adaptors/PulsarStorm/index.html | 6 +-
content/ja/admin/AdminInterface/index.html | 12 ++--
content/ja/admin/Authz/index.html | 12 ++--
content/ja/admin/ClustersBrokers/index.html| 6 +-
content/ja/admin/PropertiesNamespaces/index.html | 6 +-
content/ja/advanced/PartitionedTopics/index.html | 12 ++--
content/ja/advanced/RetentionExpiry/index.html | 12 ++--
content/ja/clients/Cpp/index.html | 6 +-
content/ja/clients/Java/index.html | 8 +--
content/ja/clients/Python/index.html | 8 +--
content/ja/clients/WebSocket/index.html| 8 +--
content/ja/deployment/InstanceSetup/index.html | 6 +-
content/ja/deployment/Kubernetes/index.html| 4 +-
.../ConceptsAndArchitecture/index.html | 2 +-
content/ja/getting-started/LocalCluster/index.html | 4 +-
content/ja/project/BinaryProtocol/index.html | 4 +-
content/ja/project/SimulationTools/index.html | 2 +-
content/ja/reference/CliTools/index.html | 18 ++---
content/swagger/swagger.json | 28
59 files changed, 350 insertions(+), 216 deletions(-)
diff --git a/content/api/cpp/classpulsar_1_1_client-members.html
b/content/api/cpp/classpulsar_1_1_client-members.html
index 6fd4fb8..e44348f 100644
--- a/content/api/cpp/classpulsar_1_1_client-members.html
+++ b/content/api/cpp/classpulsar_1_1_client-members.html
@@ -110,12 +110,16 @@ var searchBox = new SearchBox("searchBox",
"search",false,'Search');
shutdown()
(defined in pulsar::Client)pulsar::Client
subscribe(const std::string
topic, const std::string consumerName, Consumer consumer)
(defined in pulsar::Client)pulsar::Client
subscribe(const
std::string topic, const std::string consumerName, const
ConsumerConfiguration conf, Consumer consumer) (defined in pulsar::Client)pulsar::Client
- subscribe(const
std::vector std::string topics, const std::string
subscriptionName, Consumer consumer
[incubator-pulsar] branch asf-site updated: Updated site at revision 5c2e88c
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/asf-site by this push: new 7820c74 Updated site at revision 5c2e88c 7820c74 is described below commit 7820c74c8df8854612841ef36eb83bf479ed6c6d Author: jenkins AuthorDate: Sat Aug 4 03:02:01 2018 + Updated site at revision 5c2e88c --- .../docs/latest/adaptors/PulsarSpark/index.html| 8 +++--- .../docs/latest/adaptors/PulsarStorm/index.html| 6 ++--- content/docs/latest/admin-api/overview/index.html | 10 content/docs/latest/clients/Cpp/index.html | 8 +++--- content/docs/latest/clients/Java/index.html| 14 +- content/docs/latest/clients/Python/index.html | 10 content/docs/latest/clients/WebSocket/index.html | 8 +++--- content/docs/latest/clients/go/index.html | 6 ++--- .../docs/latest/cookbooks/Encryption/index.html| 6 ++--- .../latest/cookbooks/PartitionedTopics/index.html | 14 +- .../latest/cookbooks/RetentionExpiry/index.html| 14 +- .../docs/latest/cookbooks/compaction/index.html| 6 ++--- .../cookbooks/message-deduplication/index.html | 10 .../docs/latest/cookbooks/message-queue/index.html | 14 +- .../latest/cookbooks/tiered-storage/index.html | 4 +-- .../docs/latest/deployment/Kubernetes/index.html | 4 +-- .../docs/latest/deployment/aws-cluster/index.html | 4 +-- content/docs/latest/deployment/cluster/index.html | 4 +-- content/docs/latest/deployment/instance/index.html | 4 +-- .../ConceptsAndArchitecture/index.html | 4 +-- .../latest/getting-started/LocalCluster/index.html | 4 +-- .../latest/getting-started/Pulsar-2.0/index.html | 2 +- .../docs/latest/getting-started/docker/index.html | 4 +-- .../docs/latest/project/BinaryProtocol/index.html | 4 +-- content/docs/latest/project/CompileCpp/index.html | 8 +++--- .../docs/latest/project/SimulationTools/index.html | 2 +- .../docs/latest/project/schema-storage/index.html | 4 +-- content/docs/latest/reference/CliTools/index.html | 18 ++--- .../docs/latest/security/authorization/index.html | 10 content/docs/latest/security/encryption/index.html | 6 ++--- content/docs/zh-CN/client-libraries-cpp.html | 4 +-- content/docs/zh-CN/client-libraries-cpp/index.html | 4 +-- content/docs/zh-CN/next/client-libraries-cpp.html | 4 +-- .../zh-CN/next/client-libraries-cpp/index.html | 4 +-- content/ja/adaptors/PulsarSpark/index.html | 8 +++--- content/ja/adaptors/PulsarStorm/index.html | 6 ++--- content/ja/admin/AdminInterface/index.html | 12 - content/ja/admin/Authz/index.html | 12 - content/ja/admin/ClustersBrokers/index.html| 6 ++--- content/ja/admin/PropertiesNamespaces/index.html | 6 ++--- content/ja/advanced/PartitionedTopics/index.html | 12 - content/ja/advanced/RetentionExpiry/index.html | 12 - content/ja/clients/Cpp/index.html | 6 ++--- content/ja/clients/Java/index.html | 8 +++--- content/ja/clients/Python/index.html | 8 +++--- content/ja/clients/WebSocket/index.html| 8 +++--- content/ja/deployment/InstanceSetup/index.html | 6 ++--- content/ja/deployment/Kubernetes/index.html| 4 +-- .../ConceptsAndArchitecture/index.html | 2 +- content/ja/getting-started/LocalCluster/index.html | 4 +-- content/ja/project/BinaryProtocol/index.html | 4 +-- content/ja/project/SimulationTools/index.html | 2 +- content/ja/reference/CliTools/index.html | 18 ++--- content/swagger/swagger.json | 30 +++--- 54 files changed, 205 insertions(+), 205 deletions(-) diff --git a/content/docs/latest/adaptors/PulsarSpark/index.html b/content/docs/latest/adaptors/PulsarSpark/index.html index d130108..a30d928 100644 --- a/content/docs/latest/adaptors/PulsarSpark/index.html +++ b/content/docs/latest/adaptors/PulsarSpark/index.html @@ -1079,9 +1079,9 @@ - Spark Streaming Pulsar receiver + Spark Streaming Pulsar receiver @@ -1325,9 +1325,9 @@ - Spark Streaming Pulsar receiver + Spark Streaming Pulsar receiver @@ -1537,6 +1537,8 @@ + + The Pulsar Java client @@ -1615,8 +1617,6 @@ - - Authentication and authorization in Pulsar diff --git a/content/docs/latest
[incubator-pulsar] branch master updated: [website] update client references (#2284)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new cb4e6f8 [website] update client references (#2284)
cb4e6f8 is described below
commit cb4e6f80fb9e1aa720bcadaac557199c1a1fd681
Author: Sijie Guo
AuthorDate: Wed Aug 1 16:22:48 2018 -0700
[website] update client references (#2284)
### Changes
- add downloads for PRM and DEB packages
- specify version for python client (via `pip install`) and go client (via
`dep`)
---
site2/docs/client-libraries-cpp.md| 38 +++
site2/docs/client-libraries-go.md | 13 +++-
site2/docs/client-libraries-python.md | 2 +-
site2/docs/getting-started-clients.md | 2 +-
site2/website/scripts/replace.js | 32 ++---
5 files changed, 81 insertions(+), 6 deletions(-)
diff --git a/site2/docs/client-libraries-cpp.md
b/site2/docs/client-libraries-cpp.md
index e788d2f..5f536e5 100644
--- a/site2/docs/client-libraries-cpp.md
+++ b/site2/docs/client-libraries-cpp.md
@@ -10,6 +10,44 @@ The Pulsar C++ client has been successfully tested on
**MacOS** and **Linux**.
## Linux
+### Install
+
+> Since 2.1.0 release, Pulsar ships pre-built RPM and Debian packages. You can
choose download
+> and install those packages instead of building them yourself.
+
+ RPM
+
+| Link | Crypto files |
+|--|--|
+| [client]({{pulsar:rpm:client}}) | [asc]({{pulsar:rpm:client}}.asc),
[sha1]({{pulsar:rpm:client}}.sha1), [sha512]({{pulsar:rpm:client}}.sha512) |
+| [client-debuginfo]({{pulsar:rpm:client-debuginfo}}) |
[asc]({{pulsar:rpm:client-debuginfo}}.asc),
[sha1]({{pulsar:rpm:client-debuginfo}}.sha1),
[sha512]({{pulsar:rpm:client-debuginfo}}.sha512) |
+| [client-devel]({{pulsar:rpm:client-devel}}) |
[asc]({{pulsar:rpm:client-devel}}.asc),
[sha1]({{pulsar:rpm:client-devel}}.sha1),
[sha512]({{pulsar:rpm:client-devel}}.sha512) |
+
+To install a RPM package, down the RPM packages and install them using
following command:
+
+```bash
+$ rpm -ivh apache-pulsar-client*.rpm
+```
+
+ DEB
+
+| Link | Crypto files |
+|--|--|
+| [client]({{pulsar:deb:client}}) | [asc]({{pulsar:deb:client}}.asc),
[sha1]({{pulsar:deb:client}}.sha1), [sha512]({{pulsar:deb:client}}.sha512) |
+| [client-devel]({{pulsar:deb:client-devel}}) |
[asc]({{pulsar:deb:client-devel}}.asc),
[sha1]({{pulsar:deb:client-devel}}.sha1),
[sha512]({{pulsar:deb:client-devel}}.sha512) |
+
+To install a RPM package, down the DEB packages and install them using
following command:
+
+```bash
+$ apt-install apache-pulsar-client*.deb
+```
+
+### Build
+
+> If you want to build RPM and Debian packages off latest master, you can
follow the instructions
+> below to do so. All the instructions are run at the root directory of your
cloned Pulsar
+> repo.
+
There are recipes that build RPM and Debian packages containing a
statically linked `libpulsar.so` / `libpulsar.a` with all the required
dependencies.
diff --git a/site2/docs/client-libraries-go.md
b/site2/docs/client-libraries-go.md
index eeae027..910ecae 100644
--- a/site2/docs/client-libraries-go.md
+++ b/site2/docs/client-libraries-go.md
@@ -16,16 +16,27 @@ The Pulsar Go client can be used to create Pulsar
[producers](#producers), [cons
Pulsar Go client library is based on the C++ client library. Follow
the instructions for [C++ library](client-libraries-cpp.md) for installing the
binaries
-through RPM, Deb or Homebrew packages.
+through [RPM](client-libraries-cpp.md#rpm), [Deb](client-libraries-cpp.md#deb)
or [Homebrew packages](client-libraries-cpp.md#macos).
### Installing go package
You can install the `pulsar` library locally using `go get`:
+> NOTE
+>
+> `go get` doesn't support fetching a specific tag. so it will always pull in
pulsar go client
+> from latest master. You need to make sure you have installed the right
pulsar cpp client library.
+
```bash
$ go get -u github.com/apache/incubator-pulsar/pulsar-client-go/pulsar
```
+Or you can use [dep](https://github.com/golang/dep) for managing the
dependencies.
+
+```bash
+$ dep ensure -add
github.com/apache/incubator-pulsar/pulsar-client-go/pulsar@v{{pulsar.version}}
+```
+
Once installed locally, you can import it into your project:
```go
diff --git a/site2/docs/client-libraries-python.md
b/site2/docs/client-libraries-python.md
index fff755e..cbb8e01 100644
--- a/site2/docs/client-libraries-python.md
+++ b/site2/docs/client-libraries-python.md
@@ -15,7 +15,7 @@ You can install the
[`pulsar-client`](https://pypi.python.org/pypi/pulsar-client
To install the `pulsar-client` library as a pre-built package using the
[pip](https://pip.pypa.io/en/stable/) package manager:
```shell
-$ pip install pulsar-client --upgrade
+
[incubator-pulsar] branch master updated: Open site links in new windows (#2267)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new f7adc3c Open site links in new windows (#2267)
f7adc3c is described below
commit f7adc3cf276d2ae15b608562cc3a22b39388be92
Author: Matteo Merli
AuthorDate: Tue Jul 31 10:47:52 2018 -0700
Open site links in new windows (#2267)
---
site2/website/core/Footer.js | 13 +++--
site2/website/pages/en/contact.js | 22 +++---
site2/website/static/js/custom.js | 18 +-
3 files changed, 27 insertions(+), 26 deletions(-)
diff --git a/site2/website/core/Footer.js b/site2/website/core/Footer.js
index 622072b..40f21c4 100644
--- a/site2/website/core/Footer.js
+++ b/site2/website/core/Footer.js
@@ -125,16 +125,17 @@ class Footer extends React.Component {
const communityMenuJs = `
const community =
document.querySelector("a[href='#community']").parentNode;
- const communityMenu =
+ const communityMenu =
'' +
-'Community' +
+'Community ▼' +
'' +
'' +
-'Contant' +
+'Contact' +
'Events' +
-'Twitter' +
-'Wiki' +
-'Issue tracking' +
+'Twitter
' +
+'Wiki ' +
+'Issue tracking
' +
+'' +
'Resources' +
'Team' +
'' +
diff --git a/site2/website/pages/en/contact.js
b/site2/website/pages/en/contact.js
index 1bda91e..7935924 100644
--- a/site2/website/pages/en/contact.js
+++ b/site2/website/pages/en/contact.js
@@ -68,12 +68,12 @@ class Contact extends React.Component {
-There are many ways to get help from the Apache Pulsar community.
-The mailing lists are the primary place where all Pulsar
committers are present.
-Bugs and feature requests can either be discussed on the dev
mailing list or
-by opening an issue on GitHub.
+There are many ways to get help from the Apache Pulsar community.
+The mailing lists are the primary place where all Pulsar
committers are present.
+Bugs and feature requests can either be discussed on the dev
mailing list or
+by opening an issue on https://github.com/apache/incubator-pulsar/; target="_blank">GitHub.
-
+
Mailing Lists
@@ -102,12 +102,12 @@ class Contact extends React.Component {
Slack
There is a Pulsar slack channel that is used for informal
discussions for devs and users.
-
- The Slack instance is at
[https://apache-pulsar.slack.com/](https://apache-pulsar.slack.com/)
-
-
- You can self-register at
[https://apache-pulsar.herokuapp.com/](https://apache-pulsar.herokuapp.com/)
-
+
+The Slack instance is at https://apache-pulsar.slack.com/; target="_blank">
+https://apache-pulsar.slack.com/
+
+You can self-register at https://apache-pulsar.herokuapp.com/; target="_blank">
+https://apache-pulsar.herokuapp.com/
diff --git a/site2/website/static/js/custom.js
b/site2/website/static/js/custom.js
index 96f5e55..15eaa9a 100644
--- a/site2/website/static/js/custom.js
+++ b/site2/website/static/js/custom.js
@@ -4,9 +4,9 @@ window.addEventListener('load', function() {
// setup apache menu items in nav bar
/*const community =
document.querySelector("a[href='#community']").parentNode;
- const communityMenu =
+ const communityMenu =
'' +
-'Community' +
+'Community' +
'' +
'' +
'Contant' +
@@ -36,16 +36,16 @@ window.addEventListener('load', function() {
// setup apache menu items in nav bar
const apache = document.querySelector("a[href='#apache']").parentNode;
- const apacheMenu =
+ const apacheMenu =
'' +
-'Apache' +
+'Apache ▼' +
'' +
'' +
-'https://www.apache.org/;>Foundation' +
-'https://www.apache.org/licenses/;>License' +
-'https://www.apache.org/foundation/sponsorship.html;>Sponsorship'
+
-'https://www.apache.org/foundation/thanks.html;>Thanks' +
-'https://www.apache.org/security;>Security' +
+'https://www.apache.org/; target="_blank" >Foundation
' +
+'https://www.apache.org/licenses/;
target="_blank">License ' +
+'https://www.apache.org/foundation/sponsorship.html;
target="_blank">Sponsorship ' +
+'https://www.apache.org/foundation/thanks.html;
target="_blank">Thanks ' +
+'https://www.apache.org/security;
target="_blank">Security ' +
'' +
'' +
'';
[incubator-pulsar] branch asf-site updated: Updated site at revision 5ebd4f5
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/asf-site by this push:
new cdee25d Updated site at revision 5ebd4f5
cdee25d is described below
commit cdee25d570aee3e024785506fe23e6ce354defa2
Author: jenkins
AuthorDate: Tue Jul 31 09:24:00 2018 +
Updated site at revision 5ebd4f5
---
content/docs/latest/functions/metrics/index.html | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/content/docs/latest/functions/metrics/index.html
b/content/docs/latest/functions/metrics/index.html
index 2488677..f995045 100644
--- a/content/docs/latest/functions/metrics/index.html
+++ b/content/docs/latest/functions/metrics/index.html
@@ -1013,7 +1013,7 @@
public class MetricRecordingFunction implements
FunctionString, Void {
@Override
-public void apply(String
input, Context context)
{
+public Void process(String
input, Context context)
{
context.recordMetric("number-of-characters", input.length());
return null;
}
[incubator-pulsar] branch asf-site updated: Updated site at revision d0350fc
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/asf-site by this push: new 77ea3f1 Updated site at revision d0350fc 77ea3f1 is described below commit 77ea3f100d1ca2186069f199754c853a3f776d8a Author: jenkins AuthorDate: Tue Jul 31 08:34:53 2018 + Updated site at revision d0350fc --- content/docs/latest/adaptors/PulsarSpark/index.html| 8 content/docs/latest/adaptors/PulsarStorm/index.html| 6 +++--- content/docs/latest/admin-api/overview/index.html | 10 +- content/docs/latest/clients/Cpp/index.html | 8 content/docs/latest/clients/Java/index.html| 14 +++--- content/docs/latest/clients/Python/index.html | 10 +- content/docs/latest/clients/WebSocket/index.html | 8 content/docs/latest/clients/go/index.html | 6 +++--- content/docs/latest/cookbooks/Encryption/index.html| 6 +++--- .../docs/latest/cookbooks/PartitionedTopics/index.html | 14 +++--- .../docs/latest/cookbooks/RetentionExpiry/index.html | 14 +++--- content/docs/latest/cookbooks/compaction/index.html| 6 +++--- .../latest/cookbooks/message-deduplication/index.html | 10 +- content/docs/latest/cookbooks/message-queue/index.html | 14 +++--- .../docs/latest/cookbooks/tiered-storage/index.html| 4 ++-- content/docs/latest/deployment/Kubernetes/index.html | 4 ++-- content/docs/latest/deployment/aws-cluster/index.html | 4 ++-- content/docs/latest/deployment/cluster/index.html | 4 ++-- content/docs/latest/deployment/instance/index.html | 4 ++-- .../getting-started/ConceptsAndArchitecture/index.html | 4 ++-- .../latest/getting-started/LocalCluster/index.html | 4 ++-- .../docs/latest/getting-started/Pulsar-2.0/index.html | 2 +- content/docs/latest/getting-started/docker/index.html | 4 ++-- content/docs/latest/project/BinaryProtocol/index.html | 4 ++-- content/docs/latest/project/CompileCpp/index.html | 8 content/docs/latest/project/SimulationTools/index.html | 2 +- content/docs/latest/project/schema-storage/index.html | 4 ++-- content/docs/latest/reference/CliTools/index.html | 18 +- content/docs/latest/reference/RestApi/index.html | 4 ++-- content/docs/latest/security/authorization/index.html | 10 +- content/docs/latest/security/encryption/index.html | 6 +++--- content/ja/adaptors/PulsarSpark/index.html | 8 content/ja/adaptors/PulsarStorm/index.html | 6 +++--- content/ja/admin/AdminInterface/index.html | 12 ++-- content/ja/admin/Authz/index.html | 12 ++-- content/ja/admin/ClustersBrokers/index.html| 6 +++--- content/ja/admin/PropertiesNamespaces/index.html | 6 +++--- content/ja/advanced/PartitionedTopics/index.html | 12 ++-- content/ja/advanced/RetentionExpiry/index.html | 12 ++-- content/ja/clients/Cpp/index.html | 6 +++--- content/ja/clients/Java/index.html | 8 content/ja/clients/Python/index.html | 8 content/ja/clients/WebSocket/index.html| 8 content/ja/deployment/InstanceSetup/index.html | 6 +++--- content/ja/deployment/Kubernetes/index.html| 4 ++-- .../getting-started/ConceptsAndArchitecture/index.html | 2 +- content/ja/getting-started/LocalCluster/index.html | 4 ++-- content/ja/project/BinaryProtocol/index.html | 4 ++-- content/ja/project/SimulationTools/index.html | 2 +- content/ja/reference/CliTools/index.html | 18 +- content/ja/reference/RestApi/index.html| 4 ++-- content/staging/docs/deploy-bare-metal.html| 8 content/staging/docs/deploy-bare-metal/index.html | 8 content/staging/docs/standalone.html | 6 +++--- content/staging/docs/standalone/index.html | 6 +++--- content/staging/download.html | 4 ++-- content/staging/download/index.html| 4 ++-- content/staging/en/download.html | 4 ++-- content/staging/en/download/index.html | 4 ++-- content/staging/en/index.html | 2 +- content/staging/index.html | 2 +- 61 files changed, 210 insertions(+), 210 deletions(-) diff --git a/content/docs/latest/adaptors/PulsarSpark/index.html b/content/docs/latest/adaptors/PulsarSpark/index.html index 4dc895f..afa2d4a 100644 --- a/content/docs/latest/adaptors/PulsarSpark/index.html +++ b/content/docs/latest/adaptors/PulsarSpark
[incubator-pulsar] branch master updated: Fix pulsar:version vars in site2 docs. (#2254)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 5f30d04 Fix pulsar:version vars in site2 docs. (#2254)
5f30d04 is described below
commit 5f30d048f91191366412136dff2ef9290ab6761a
Author: cckellogg
AuthorDate: Mon Jul 30 15:20:26 2018 -0700
Fix pulsar:version vars in site2 docs. (#2254)
---
site2/docs/deploy-bare-metal.md | 8
site2/docs/getting-started-standalone.md | 6 +++---
2 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/site2/docs/deploy-bare-metal.md b/site2/docs/deploy-bare-metal.md
index 792aa39..0c9236b 100644
--- a/site2/docs/deploy-bare-metal.md
+++ b/site2/docs/deploy-bare-metal.md
@@ -48,20 +48,20 @@ For machines running a bookie and a Pulsar broker, we
recommend using more power
To get started deploying a Pulsar cluster on bare metal, you'll need to
download a binary tarball release in one of the following ways:
* By clicking on the link directly below, which will automatically trigger a
download:
- * Pulsar pulsar:version binary
release
+ * Pulsar {{pulsar:version}}
binary release
* From the Pulsar [downloads page](pulsar:download_page_url)
* From the Pulsar [releases
page](https://github.com/apache/incubator-pulsar/releases/latest) on
[GitHub](https://github.com)
* Using [wget](https://www.gnu.org/software/wget):
```bash
-$ wget
http://archive.apache.org/dist/incubator/pulsar/pulsar-pulsar:version/apache-pulsar-pulsar:version-bin.tar.gz
+$ wget pulsar:binary_release_url
```
Once you've downloaded the tarball, untar it and `cd` into the resulting
directory:
```bash
-$ tar xvzf apache-pulsar-pulsar:version-bin.tar.gz
-$ cd apache-pulsar-pulsar:version
+$ tar xvzf apache-pulsar-{{pulsar:version}}-bin.tar.gz
+$ cd apache-pulsar-{{pulsar:version}}
```
The untarred directory contains the following subdirectories:
diff --git a/site2/docs/getting-started-standalone.md
b/site2/docs/getting-started-standalone.md
index 99bd923..ebe20b1 100644
--- a/site2/docs/getting-started-standalone.md
+++ b/site2/docs/getting-started-standalone.md
@@ -20,7 +20,7 @@ To get started running Pulsar, download a binary tarball
release in one of the f
* by clicking the link below and downloading the release from an Apache mirror:
- * Pulsar pulsar:version binary
release
+ * Pulsar {{pulsar:version}}
binary release
* from the Pulsar [downloads page](pulsar:download_page_url)
* from the Pulsar [releases
page](https://github.com/apache/incubator-pulsar/releases/latest)
@@ -33,8 +33,8 @@ To get started running Pulsar, download a binary tarball
release in one of the f
Once the tarball is downloaded, untar it and `cd` into the resulting directory:
```bash
-$ tar xvfz apache-pulsar-pulsar:version-bin.tar.gz
-$ cd apache-pulsar-pulsar:version
+$ tar xvfz apache-pulsar-{{pulsar:version}}-bin.tar.gz
+$ cd apache-pulsar-{{pulsar:version}}
```
## What your package contains
[incubator-pulsar] branch asf-site updated: Updated site at revision fd1b459
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/asf-site by this push: new 6cc7861 Updated site at revision fd1b459 6cc7861 is described below commit 6cc7861ee480ef74fe643799283c407cfe28ca09 Author: jenkins AuthorDate: Fri Jul 27 18:10:14 2018 + Updated site at revision fd1b459 --- .../docs/latest/adaptors/PulsarSpark/index.html| 10 +- .../docs/latest/adaptors/PulsarStorm/index.html| 8 content/docs/latest/admin-api/overview/index.html | 18 +- content/docs/latest/clients/Cpp/index.html | 8 content/docs/latest/clients/Java/index.html| 16 content/docs/latest/clients/Python/index.html | 10 +- content/docs/latest/clients/WebSocket/index.html | 8 content/docs/latest/clients/go/index.html | 6 +++--- .../docs/latest/cookbooks/Encryption/index.html| 6 +++--- .../latest/cookbooks/PartitionedTopics/index.html | 16 .../latest/cookbooks/RetentionExpiry/index.html| 16 .../docs/latest/cookbooks/compaction/index.html| 8 .../cookbooks/message-deduplication/index.html | 12 ++-- .../docs/latest/cookbooks/message-queue/index.html | 16 .../latest/cookbooks/tiered-storage/index.html | 6 +++--- .../docs/latest/deployment/Kubernetes/index.html | 4 ++-- .../docs/latest/deployment/aws-cluster/index.html | 6 +++--- content/docs/latest/deployment/cluster/index.html | 6 +++--- content/docs/latest/deployment/instance/index.html | 6 +++--- .../ConceptsAndArchitecture/index.html | 4 ++-- .../latest/getting-started/LocalCluster/index.html | 4 ++-- .../latest/getting-started/Pulsar-2.0/index.html | 2 +- .../docs/latest/getting-started/docker/index.html | 4 ++-- .../docs/latest/project/BinaryProtocol/index.html | 4 ++-- content/docs/latest/project/CompileCpp/index.html | 8 .../docs/latest/project/SimulationTools/index.html | 2 +- .../docs/latest/project/schema-storage/index.html | 4 ++-- content/docs/latest/reference/CliTools/index.html | 22 +++--- .../docs/latest/security/authorization/index.html | 14 +++--- content/docs/latest/security/encryption/index.html | 6 +++--- content/ja/adaptors/PulsarSpark/index.html | 8 content/ja/adaptors/PulsarStorm/index.html | 6 +++--- content/ja/admin/AdminInterface/index.html | 12 ++-- content/ja/admin/Authz/index.html | 12 ++-- content/ja/admin/ClustersBrokers/index.html| 6 +++--- content/ja/admin/PropertiesNamespaces/index.html | 6 +++--- content/ja/advanced/PartitionedTopics/index.html | 12 ++-- content/ja/advanced/RetentionExpiry/index.html | 12 ++-- content/ja/clients/Cpp/index.html | 6 +++--- content/ja/clients/Java/index.html | 8 content/ja/clients/Python/index.html | 8 content/ja/clients/WebSocket/index.html| 8 content/ja/deployment/InstanceSetup/index.html | 6 +++--- content/ja/deployment/Kubernetes/index.html| 4 ++-- .../ConceptsAndArchitecture/index.html | 2 +- content/ja/getting-started/LocalCluster/index.html | 4 ++-- content/ja/project/BinaryProtocol/index.html | 4 ++-- content/ja/project/SimulationTools/index.html | 2 +- content/ja/reference/CliTools/index.html | 18 +- content/staging/download.html | 4 +--- content/staging/download/index.html| 4 +--- content/staging/en/download.html | 4 +--- content/staging/en/download/index.html | 4 +--- content/staging/en/release-notes.html | 2 +- content/staging/en/release-notes/index.html| 2 +- content/staging/en/versions.html | 2 +- content/staging/en/versions/index.html | 2 +- content/staging/release-notes.html | 2 +- content/staging/release-notes/index.html | 2 +- content/staging/versions.html | 2 +- content/staging/versions/index.html| 2 +- 61 files changed, 214 insertions(+), 222 deletions(-) diff --git a/content/docs/latest/adaptors/PulsarSpark/index.html b/content/docs/latest/adaptors/PulsarSpark/index.html index 6bd0b47..afa2d4a 100644 --- a/content/docs/latest/adaptors/PulsarSpark/index.html +++ b/content/docs/latest/adaptors/PulsarSpark/index.html @@ -1072,9 +1072,9 @@ - Spark Streaming Pulsar receiver + Spark Streaming Pulsar receiver @@ -1316,9
[incubator-pulsar] branch master updated: [website] Add versions page (#2247)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new fd1b459 [website] Add versions page (#2247)
fd1b459 is described below
commit fd1b4593d9f993a3450a151e8aa7d1ce671f215e
Author: Sijie Guo
AuthorDate: Fri Jul 27 10:31:39 2018 -0700
[website] Add versions page (#2247)
* Update release notes title
* [website] Add versions page
### Motivation
We need to link back to the documentation for the old versions.
### Changes
Update `versions` page to include links back to old versions.
### NOTES
This change doesn't include cutting a version for 2.1.0 release.
This will be done after 2.1.0 release is cut.
---
site2/website/oldversions.json | 8 ++
site2/website/pages/en/download.js | 21 ++--
site2/website/pages/en/release-notes.js | 2 +-
site2/website/pages/en/versions.js | 171 +---
site2/website/siteConfig.js | 2 +-
5 files changed, 113 insertions(+), 91 deletions(-)
diff --git a/site2/website/oldversions.json b/site2/website/oldversions.json
new file mode 100644
index 000..a83f27f
--- /dev/null
+++ b/site2/website/oldversions.json
@@ -0,0 +1,8 @@
+[
+"2.0.1-incubating",
+"2.0.0-rc1-incubating",
+"1.22.0-incubating",
+"1.21.0-incubating",
+"1.20.0-incubating",
+"1.19.0-incubating"
+]
diff --git a/site2/website/pages/en/download.js
b/site2/website/pages/en/download.js
index 2484874..6d34397 100644
--- a/site2/website/pages/en/download.js
+++ b/site2/website/pages/en/download.js
@@ -89,15 +89,20 @@ class Download extends React.Component {
Release notes
-
- [Release notes](/release-notes) for all Pulsar's versions
-
+
+
+Release
notes for all Pulsar's versions
+
+
Getting started
-
- Once you've downloaded a Pulsar release, instructions on getting
up and running with a standalone cluster
- that you can run on your laptop can be found in the [Run Pulsar
locally](/docs/standalone) tutorial.
-
+
+
+Once you've downloaded a Pulsar release, instructions on
getting up and running with a standalone cluster
+that you can run on your laptop can be found in the{' '}
+Run Pulsar
locally tutorial.
+
+
If you need to connect to an existing Pulsar cluster or instance
using an officially supported client,
see the client docs for these languages:
@@ -160,7 +165,7 @@ class Download extends React.Component {
sha512)
- Release Notes
+ Release Notes
)
diff --git a/site2/website/pages/en/release-notes.js
b/site2/website/pages/en/release-notes.js
index 09643e3..076d6a6 100644
--- a/site2/website/pages/en/release-notes.js
+++ b/site2/website/pages/en/release-notes.js
@@ -19,7 +19,7 @@ class ReleaseNotes extends React.Component {
- Apache Pulsar downloads
+ Apache Pulsar Release Notes
diff --git a/site2/website/pages/en/versions.js
b/site2/website/pages/en/versions.js
index 5aabeb1..c730e20 100644
--- a/site2/website/pages/en/versions.js
+++ b/site2/website/pages/en/versions.js
@@ -8,90 +8,99 @@ const GridBlock = CompLibrary.GridBlock;
const CWD = process.cwd();
const siteConfig = require(`${CWD}/siteConfig.js`);
-//const versions = require(CWD + '/versions.json');
+// versions post docusaurus
+// const versions = require(`${CWD}/versions.json`);
+// versions pre docusaurus
+const oldversions = require(`${CWD}/oldversions.json`);
-/*
-class Versions extends React.Component {
- render() {
-const latestVersion = versions[0];
+function Versions(props) {
+const latestStableVersion = oldversions[0];
+const repoUrl = `https://github.com/${siteConfig.organizationName}/${
+ siteConfig.projectName
+}`;
return (
-
-
-
-
- {siteConfig.title + ' Versions'}
-
-New versions of this project are released every so often.
-Current version (Stable)
-
-
-
- {latestVersion}
-
-Documentation
-
[incubator-pulsar] branch master updated: add stats entry for active consumer name and endpoint to get leader of functions cluster (#2234)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/master by this push: new 10ec02e add stats entry for active consumer name and endpoint to get leader of functions cluster (#2234) 10ec02e is described below commit 10ec02eabe0670aab7d60d49e6d242469a8a11f0 Author: Boyang Jerry Peng AuthorDate: Thu Jul 26 15:51:53 2018 -0700 add stats entry for active consumer name and endpoint to get leader of functions cluster (#2234) * add stats entry for active consumer name and endpoint to get leader of functions cluster * fixing bug in stats * removing line --- .../pulsar/broker/admin/impl/FunctionsBase.java| 14 .../service/persistent/PersistentSubscription.java | 7 ++ .../common/policies/data/SubscriptionStats.java| 5 - .../pulsar/functions/worker/MembershipManager.java | 25 ++ .../functions/worker/rest/api/FunctionsImpl.java | 18 .../worker/rest/api/v2/FunctionApiV2Resource.java | 7 ++ 6 files changed, 75 insertions(+), 1 deletion(-) diff --git a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/FunctionsBase.java b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/FunctionsBase.java index f97f180..16d8252 100644 --- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/FunctionsBase.java +++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/FunctionsBase.java @@ -215,6 +215,20 @@ public class FunctionsBase extends AdminResource implements Supplier
[incubator-pulsar] branch master updated: [build] Add `yarn.lock` into gitignore file (#2242)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git The following commit(s) were added to refs/heads/master by this push: new 8e62a46 [build] Add `yarn.lock` into gitignore file (#2242) 8e62a46 is described below commit 8e62a468471187036d5d6c3df825b5144c70ed85 Author: Sijie Guo AuthorDate: Thu Jul 26 15:15:01 2018 -0700 [build] Add `yarn.lock` into gitignore file (#2242) ### Motivation We introduced docusaurus in #2206 for building the new documentation site. docusaurus is using [yarn](https://yarnpkg.com/en/docs/install#mac-stable) for building. It will leave `yarn.lock` file after builds. ### Changes Add `yarn.lock` into gitignore file --- .gitignore | 4 1 file changed, 4 insertions(+) diff --git a/.gitignore b/.gitignore index 8808c8f..3e3c4e8 100644 --- a/.gitignore +++ b/.gitignore @@ -67,3 +67,7 @@ pulsar-client-cpp/python/wheelhouse # CI generated files .repository docker.debug-info + +# Yarn + +**/yarn.lock
