[GitHub] hrsakai commented on issue #672: Support setting private key content in athenz client plugin
hrsakai commented on issue #672: Support setting private key content in athenz client plugin URL: https://github.com/apache/incubator-pulsar/pull/672#issuecomment-321749998 OK. We will consider it. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] hrsakai commented on issue #620: added new entrypoint for reader to websocket proxy
hrsakai commented on issue #620: added new entrypoint for reader to websocket proxy URL: https://github.com/apache/incubator-pulsar/pull/620#issuecomment-321749147 @rdhabalia Sorry for my late reply, I added warn log and call receiveMessage() on every readNext(). Please check them. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #672: Support setting private key content in athenz client plugin
merlimat commented on issue #672: Support setting private key content in athenz client plugin URL: https://github.com/apache/incubator-pulsar/pull/672#issuecomment-321729674 > How about adding support for URL schemes (e.g. file:, data:, env:, exec:) to privateKeyPath config param? If we provided the reader and plugins used it, the plugins would get the same benefit when we support new scheme (new data source), plus it can be applicable for any configuration parameters. +1 for this. In container deployements line Kubernetes, it is common to have private key in files that are mounted as "secrets" and seen as files by the application. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #672: Support setting private key content in athenz client plugin
merlimat commented on issue #672: Support setting private key content in athenz client plugin URL: https://github.com/apache/incubator-pulsar/pull/672#issuecomment-321729674 > How about adding support for URL schemes (e.g. file:, data:, env:, exec:) to privateKeyPath config param? If we provided the reader and plugins used it, the plugins would get the same benefit when we support new scheme (new data source), plus it can be applicable for any configuration parameters. +1 for this. In container deployements line Kubernetes, is common to have private key in files that are mounted as "secrets" and seen as files by the application. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] maskit commented on issue #672: Support setting private key content in athenz client plugin
maskit commented on issue #672: Support setting private key content in athenz client plugin URL: https://github.com/apache/incubator-pulsar/pull/672#issuecomment-321729113 OK, that makes sense. Since it's just a plugin I don't object introducing its own unique way, however, it sounds like not athenz specific issue. The same issue will appear if somebody tried to use TLS auth, and they would need to re-implement the logic to read secrets from somewhere. Also, all users who read it from environment variables need to implement the logic even though it's not an unique way. How about adding support for URL schemes (e.g. file:, data:, env:, exec:) to privateKeyPath config param? If we provided the reader and plugins used it, the plugins would get the same benefit when we support new scheme (new data source), plus it can be applicable for any configuration parameters. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #673: Fix #657: Use exceptions instead of assert() to do runtime parameter ?
merlimat commented on issue #673: Fix #657: Use exceptions instead of assert() to do runtime parameter ? URL: https://github.com/apache/incubator-pulsar/pull/673#issuecomment-321709114 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat closed pull request #676: remove topicName-dataProvider from testIncrementPartitionsOfTopic test
merlimat closed pull request #676: remove topicName-dataProvider from testIncrementPartitionsOfTopic test URL: https://github.com/apache/incubator-pulsar/pull/676 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rdhabalia commented on issue #635: Intermittent test failures in AdminApiTest.testIncrementPartitionsOfTopic
rdhabalia commented on issue #635: Intermittent test failures in
AdminApiTest.testIncrementPartitionsOfTopic
URL:
https://github.com/apache/incubator-pulsar/issues/635#issuecomment-321716697
Actually, even in last build, test is not failed due to assertion error but
it failed with 500-error because broker gets shutdown during the test and topic
is closed.
```
persistent://prop-xyz/use/ns1/topic_+&*%{}() \/$@#^%
org.apache.pulsar.broker.service.BrokerServiceException$PersistenceException:
org.apache.bookkeeper.mledger.ManagedLedgerException: ManagedLedger
prop-xyz/use/ns1/persistent/topic_%2B%26*%25%7B%7D%28%29+%5C%2F%24%40%23%5E%25-partition-4
has already been closed
at
org.apache.pulsar.broker.service.persistent.PersistentTopic$2.openCursorFailed(PersistentTopic.java:456)
at
org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.asyncOpenCursor(ManagedLedgerImpl.java:584)
at
org.apache.pulsar.broker.service.persistent.PersistentTopic.getDurableSubscription(PersistentTopic.java:441)
at
org.apache.pulsar.broker.service.persistent.PersistentTopic.createSubscription(PersistentTopic.java:494)
at
org.apache.pulsar.broker.admin.PersistentTopics$3.lambda$2(PersistentTopics.java:1322)
at
java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822)
at
java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:834)
at
java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2155)
at
org.apache.pulsar.broker.admin.PersistentTopics$3.lambda$0(PersistentTopics.java:1316)
at java.util.ArrayList.forEach(ArrayList.java:1249)
at
org.apache.pulsar.broker.admin.PersistentTopics$3.operationComplete(PersistentTopics
```
`AdminApitTest` creates 2 pulsar instance and shutdown and
`testIncrementPartitionsOfTopic` runs twice with different topic name. and
according to log: broker shutdowns while test is running second time.
```
486298 [main] INFO org.apache.pulsar.broker.service.BrokerService - Shutting
down Pulsar Broker service
486298 [main] INFO org.apache.pulsar.zookeeper.ZooKeeperDataCache -
[MockZookeeper] Received ZooKeeper watch event: WatchedEvent
state:SyncConnected type:NodeDeleted path:/loadbalance/brokers/localhost:15413
```
So, created #676 , and hopefully it should fix it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] merlimat opened a new pull request #675: Interrupt the Python consumer when receiving SIGINT
merlimat opened a new pull request #675: Interrupt the Python consumer when receiving SIGINT URL: https://github.com/apache/incubator-pulsar/pull/675 ### Motivation When a Python consumer is blocked on a `consumer.receive()` call, it will ignore the SIGINT from a `Ctrl-C` interruption. The consumer appears to be frozen, and the Python process is stuck, waiting for more messages. We need to make sure to respect the SIGINT, at least for the `consumer.receive()` which block undefinitely. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #673: Fix #657: Use exceptions instead of assert() to do runtime parameter ?
merlimat commented on issue #673: Fix #657: Use exceptions instead of assert() to do runtime parameter ? URL: https://github.com/apache/incubator-pulsar/pull/673#issuecomment-321709114 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #673: Fix #657: Use exceptions instead of assert() to do runtime parameter ?
merlimat commented on issue #673: Fix #657: Use exceptions instead of assert() to do runtime parameter ? URL: https://github.com/apache/incubator-pulsar/pull/673#issuecomment-321709106 ping @jai1 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] msb-at-yahoo commented on issue #672: Support setting private key content in athenz client plugin
msb-at-yahoo commented on issue #672: Support setting private key content in athenz client plugin URL: https://github.com/apache/incubator-pulsar/pull/672#issuecomment-321704775 what about getting rid of the privateKeyPath field, and during init load the content of the file if privateKeyPath is in the config? then getZtsClient can lose the branch. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat opened a new issue #635: Intermittent test failures in AdminApiTest.testIncrementPartitionsOfTopic
merlimat opened a new issue #635: Intermittent test failures in AdminApiTest.testIncrementPartitionsOfTopic URL: https://github.com/apache/incubator-pulsar/issues/635 As seen in https://builds.apache.org/job/pulsar-pull-request/org.apache.pulsar$pulsar-broker/59/testReport/junit/org.apache.pulsar.broker.admin/AdminApiTest/testIncrementPartitionsOfTopic/ ``` java.lang.AssertionError: expected [8] but found [12] at org.apache.pulsar.broker.admin.AdminApiTest.testIncrementPartitionsOfTopic(AdminApiTest.java:1761) ``` Apparently, this failure is causing other tests in `AdminApiTests` to fail afterwards: ``` org.apache.pulsar.broker.admin.AdminApiTest.testIncrementPartitionsOfTopic org.apache.pulsar.broker.admin.AdminApiTest.testIncrementPartitionsOfTopic org.apache.pulsar.broker.admin.AdminApiTest.testNamespaceSplitBundle org.apache.pulsar.broker.admin.AdminApiTest.testNamespaceUnloadBundle org.apache.pulsar.broker.admin.AdminApiTest.testPersistentTopicsExpireMessages org.apache.pulsar.broker.admin.AdminApiTest.testPulsarAdminForUriAndUrlEncoding org.apache.pulsar.broker.admin.AdminApiTest.testPulsarAdminForUriAndUrlEncoding ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #674: Assertion failure in RocksDB when opening the index db
merlimat commented on issue #674: Assertion failure in RocksDB when opening the index db URL: https://github.com/apache/incubator-pulsar/issues/674#issuecomment-321675966 Created facebook/rocksdb#2717 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat opened a new issue #674: Assertion failure in RocksDB when opening the index db
frame #13: 0x00013d90f3ca librocksdbjni1259198247081385980.jnilib`rocksdb::DBImpl::Recover(std::__1::vector<rocksdb::ColumnFamilyDescriptor, std::__1::allocator > const&, bool, bool, bool) + 1370 frame #14: 0x00013d91371e librocksdbjni1259198247081385980.jnilib`rocksdb::DB::Open(rocksdb::DBOptions const&, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, std::__1::vector<rocksdb::ColumnFamilyDescriptor, std::__1::allocator > const&, std::__1::vector<rocksdb::ColumnFamilyHandle*, std::__1::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**) + 2238 frame #15: 0x00013d912c27 librocksdbjni1259198247081385980.jnilib`rocksdb::DB::Open(rocksdb::Options const&, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, rocksdb::DB**) + 1351 frame #16: 0x00013d83610c librocksdbjni1259198247081385980.jnilib`___lldb_unnamed_symbol106$$librocksdbjni1259198247081385980.jnilib + 28 frame #17: 0x00013d8302a3 librocksdbjni1259198247081385980.jnilib`rocksdb_open_helper(JNIEnv_*, long, _jstring*, std::__1::function, std::__1::allocator > const&, rocksdb::DB**)>) + 147 frame #18: 0x00013d8303a9 librocksdbjni1259198247081385980.jnilib`Java_org_rocksdb_RocksDB_open__JLjava_lang_String_2 + 73 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat closed pull request #666: Added documentation on how to start the standalone service with Docker
merlimat closed pull request #666: Added documentation on how to start the standalone service with Docker URL: https://github.com/apache/incubator-pulsar/pull/666 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat closed pull request #670: Don't run Javadoc and Swagger builds on regular mvn package
merlimat closed pull request #670: Don't run Javadoc and Swagger builds on regular mvn package URL: https://github.com/apache/incubator-pulsar/pull/670 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on a change in pull request #666: Added documentation on how to start the standalone service with Docker
merlimat commented on a change in pull request #666: Added documentation on how
to start the standalone service with Docker
URL: https://github.com/apache/incubator-pulsar/pull/666#discussion_r132488932
##
File path: site/docs/latest/getting-started/docker.md
##
@@ -0,0 +1,162 @@
+---
+title: Start a standalone cluster with Docker
+lead: Quickly start a Pulsar service in a single Docker container
+tags:
+- standalone
+- local
+- docker
+next: ../Clients
+---
+
+
+For the purposes of local development and testing, you can run Pulsar in {%
popover standalone %}
+mode on your own machine within a Docker container.
+
+If you don't have Docker installed, you can download the [Community
edition](https://www.docker.com/community-edition)
+and follow the instructions for your OS.
+
+## Starting Pulsar inside Docker
+
+```shell
+$ docker run -it \
+-p 6650:6650 \
+-p 8080:8080 \
+-v $PWD/data:/pulsar/data \
+apachepulsar/pulsar:{{site.current_version}} \
Review comment:
Differently from the kubernetes yaml definitiions, this is part of the
website, so it's getting converted into html at the website build time. At that
point, Jekyll will substitute the variables with theyr values.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rdhabalia commented on issue #634: PIP-3 : Introduce message-dispatch rate limiting
rdhabalia commented on issue #634: PIP-3 : Introduce message-dispatch rate limiting URL: https://github.com/apache/incubator-pulsar/pull/634#issuecomment-321617192 @merlimat I addressed all the comments. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #666: Added documentation on how to start the standalone service with Docker
merlimat commented on issue #666: Added documentation on how to start the standalone service with Docker URL: https://github.com/apache/incubator-pulsar/pull/666#issuecomment-321615429 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat closed pull request #668: Fix Python package version
merlimat closed pull request #668: Fix Python package version URL: https://github.com/apache/incubator-pulsar/pull/668 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] hrsakai commented on issue #672: Support setting private key content in athenz client plugin
hrsakai commented on issue #672: Support setting private key content in athenz client plugin URL: https://github.com/apache/incubator-pulsar/pull/672#issuecomment-321609624 In case of PCF, we should not deploy private key as a file in terms of security and auto scale, we should set private key content to Environment variable. so we added this modification and we think this is useful in the similar case. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] maskit commented on issue #638: Dead link on Getting started
maskit commented on issue #638: Dead link on Getting started URL: https://github.com/apache/incubator-pulsar/issues/638#issuecomment-321588197 Fix confirmed, thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on a change in pull request #666: Added documentation on how to start the standalone service with Docker
merlimat commented on a change in pull request #666: Added documentation on how
to start the standalone service with Docker
URL: https://github.com/apache/incubator-pulsar/pull/666#discussion_r132488932
##
File path: site/docs/latest/getting-started/docker.md
##
@@ -0,0 +1,162 @@
+---
+title: Start a standalone cluster with Docker
+lead: Quickly start a Pulsar service in a single Docker container
+tags:
+- standalone
+- local
+- docker
+next: ../Clients
+---
+
+
+For the purposes of local development and testing, you can run Pulsar in {%
popover standalone %}
+mode on your own machine within a Docker container.
+
+If you don't have Docker installed, you can download the [Community
edition](https://www.docker.com/community-edition)
+and follow the instructions for your OS.
+
+## Starting Pulsar inside Docker
+
+```shell
+$ docker run -it \
+-p 6650:6650 \
+-p 8080:8080 \
+-v $PWD/data:/pulsar/data \
+apachepulsar/pulsar:{{site.current_version}} \
Review comment:
Differently from the kubernetes yaml definitiions, this is part of the
website, so it's getting converted into html at the website build time. At that
point, Jekyll will substitute the variables with they're values.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] maskit commented on issue #672: Support setting private key content in athenz client plugin
maskit commented on issue #672: Support setting private key content in athenz client plugin URL: https://github.com/apache/incubator-pulsar/pull/672#issuecomment-321582633 I don't know much about athenz but why is this needed? What would be the benefits? If we provide multiple ways to achieve something, we should provide their pros and cons too so that users can make a right decision. Otherwise it would introduce confusion. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] maskit commented on issue #671: Should authParamsString for Authentication support another format ?
maskit commented on issue #671: Should authParamsString for Authentication support another format ? URL: https://github.com/apache/incubator-pulsar/issues/671#issuecomment-321577504 I think the entire parameter value should be parsed on plugins side. Then plugins can choose any format they want. Some plugin may need just a keyword, some other may need binary data. This would break compatibility though. I'd say the current interface could be better. (When can we make incompatible changes btw?) If we need to keep compatibility as possible as we can, 4th option would be "Encode values" (== don't allow to use ":" and ","). Any encodings which doesn't use ":" and "," (e.g. base64, URL encode) are fine, and we can (should) delegate the decoding to plugins. ---- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rdhabalia commented on a change in pull request #666: Added documentation on how to start the standalone service with Docker
rdhabalia commented on a change in pull request #666: Added documentation on
how to start the standalone service with Docker
URL: https://github.com/apache/incubator-pulsar/pull/666#discussion_r132374595
##
File path: site/docs/latest/getting-started/docker.md
##
@@ -0,0 +1,162 @@
+---
+title: Start a standalone cluster with Docker
+lead: Quickly start a Pulsar service in a single Docker container
+tags:
+- standalone
+- local
+- docker
+next: ../Clients
+---
+
+
+For the purposes of local development and testing, you can run Pulsar in {%
popover standalone %}
+mode on your own machine within a Docker container.
+
+If you don't have Docker installed, you can download the [Community
edition](https://www.docker.com/community-edition)
+and follow the instructions for your OS.
+
+## Starting Pulsar inside Docker
+
+```shell
+$ docker run -it \
+-p 6650:6650 \
+-p 8080:8080 \
+-v $PWD/data:/pulsar/data \
+apachepulsar/pulsar:{{site.current_version}} \
Review comment:
instead `{{site.current_version}}`, should we just keep `latest` so, anyone
can copy this text and execute it?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] hrsakai opened a new pull request #672: Support setting private key content in athenz client plugin
hrsakai opened a new pull request #672: Support setting private key content in athenz client plugin URL: https://github.com/apache/incubator-pulsar/pull/672 ### Motivation We can't set private key content, but it can only be set file path in athenz client plugin. ### Modifications Supports setting private key content in athenz client plugin. ### Result We can set private key content instead of file path. Next, we plan to modify c++ athenz client plugin. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rdhabalia commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting
rdhabalia commented on a change in pull request #634: PIP-3 : Introduce
message-dispatch rate limiting
URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r132353530
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/Namespaces.java
##
@@ -810,6 +811,42 @@ public void splitNamespaceBundle(@PathParam("property")
String property, @PathPa
}
}
+@PUT
Review comment:
have updated with `POST` but, [REST
standard](http://restcookbook.com/HTTP%20Methods/put-vs-post) suggests `PUT`
for resource-update and `POST` for create.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rdhabalia closed pull request #669: centralize policies znode name
rdhabalia closed pull request #669: centralize policies znode name URL: https://github.com/apache/incubator-pulsar/pull/669 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] yush1ga opened a new issue #671: Should authParamsString for Authentication support another format.
yush1ga opened a new issue #671: Should authParamsString for Authentication
support another format.
URL: https://github.com/apache/incubator-pulsar/issues/671
Now, when parsing authParamsString, keys or values containing ":" or ","
cannot been parsed correctly.
However, C++ Athenz Plugin requires for URL (containing ":") as a parameter.
Moreover, other 3rd party or official auth plugins may require parameters
containing ":" in near future.
I have 3 ideas to resolve this problem.
### 1. Enable users to set delimiters.
Users can set delimiters freely if they don't want to use ":" and ",".
### 2. Support JSON
In addition to current format, enable JSON to be used.
### 3. Ignore second and subsequent ":"
e.g.
When the string like `url:http://hoge.com:4080/` is gotten, it is processed
like following.
```java
paramString = "url:http://hoge.com:4080/;
String[] kv = paramString.split(":")
paramMap[kv[0]] = "";
for (int i = 1; i < kv.size(); i++) {
paramMap[kv[0]] += kv[i];
}
```
? In this case, parameter containing "," cannot been parsed.
--------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] merlimat opened a new pull request #670: Don't run Javadoc and Swagger builds on regular mvn package
merlimat opened a new pull request #670: Don't run Javadoc and Swagger builds on regular mvn package URL: https://github.com/apache/incubator-pulsar/pull/670 ### Motivation Javadoc and Swagger builds are only needed when building the website and should not run for PRs builds. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rdhabalia opened a new pull request #669: centralize policies znode name
rdhabalia opened a new pull request #669: centralize policies znode name URL: https://github.com/apache/incubator-pulsar/pull/669 ### Motivation Right now, znode name of `policies` is not centralized and every class uses it's own reference. making it centralized help to create a single literal of `policies` znode and useful for future refactoring or readability. ### Result No functional change. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat opened a new pull request #668: Fix Python package version
merlimat opened a new pull request #668: Fix Python package version URL: https://github.com/apache/incubator-pulsar/pull/668 ### Motivation Small fix to the scripts that obtain the version to be compliant with PyPI. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat closed pull request #665: Use Docker images from apachepulsar hub org
merlimat closed pull request #665: Use Docker images from apachepulsar hub org URL: https://github.com/apache/incubator-pulsar/pull/665 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat closed pull request #664: Fixed broken links to admin-api/overview
merlimat closed pull request #664: Fixed broken links to admin-api/overview URL: https://github.com/apache/incubator-pulsar/pull/664 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on a change in pull request #664: Fixed broken links to admin-api/overview
merlimat commented on a change in pull request #664: Fixed broken links to admin-api/overview URL: https://github.com/apache/incubator-pulsar/pull/664#discussion_r132264650 ## File path: site/_includes/explanations/partitioned-topic-admin.md ## @@ -1,4 +1,4 @@ -You can use Pulsar's [admin API](../../admin/AdminInterface) to create and manage partitioned topics. +You can use Pulsar's [admin API](../../admin-api/overview) to create and manage partitioned topics. Review comment: Oh, I think I misunderstood your comment then. Are you saying to add the link to `/docs/latest/advanced/PartitionedTopics` ? Can we do it later though? Let's just fix the 404 errors for now. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rdhabalia commented on a change in pull request #664: Fixed broken links to admin-api/overview
rdhabalia commented on a change in pull request #664: Fixed broken links to admin-api/overview URL: https://github.com/apache/incubator-pulsar/pull/664#discussion_r132251410 ## File path: site/_includes/explanations/partitioned-topic-admin.md ## @@ -1,4 +1,4 @@ -You can use Pulsar's [admin API](../../admin/AdminInterface) to create and manage partitioned topics. +You can use Pulsar's [admin API](../../admin-api/overview) to create and manage partitioned topics. Review comment: instead of overview -> `partitioned-topics`? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #639: Getting started page should provide instructions on how to build from source
merlimat commented on issue #639: Getting started page should provide instructions on how to build from source URL: https://github.com/apache/incubator-pulsar/issues/639#issuecomment-321336061 Fixed in #648 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat closed issue #639: Getting started page should provide instructions on how to build from source
merlimat closed issue #639: Getting started page should provide instructions on how to build from source URL: https://github.com/apache/incubator-pulsar/issues/639 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on a change in pull request #665: Use Docker images from apachepulsar hub org
merlimat commented on a change in pull request #665: Use Docker images from apachepulsar hub org URL: https://github.com/apache/incubator-pulsar/pull/665#discussion_r132259125 ## File path: kubernetes/generic/bookie.yaml ## @@ -57,7 +57,7 @@ spec: spec: containers: - name: bookie -image: streamlio/pulsar:latest +image: apachepulsar/pulsar:latest Review comment: (sent previous comment before getting your second one) > I can see both latest/pulsar:1.19.0-incubating present at hub so, if we always push latest > then it would be better to keep latest as it won't require code change.? Yes the `latest` tag is special for Docker in that it's also the default tag that gets used when no version is specified. The script to publish the images makes sure to always push both tags to hub.docker.com : https://github.com/apache/incubator-pulsar/blob/master/docker/publish.sh#L66 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rdhabalia commented on a change in pull request #665: Use Docker images from apachepulsar hub org
rdhabalia commented on a change in pull request #665: Use Docker images from apachepulsar hub org URL: https://github.com/apache/incubator-pulsar/pull/665#discussion_r132257041 ## File path: kubernetes/generic/bookie.yaml ## @@ -57,7 +57,7 @@ spec: spec: containers: - name: bookie -image: streamlio/pulsar:latest +image: apachepulsar/pulsar:latest Review comment: I can see both `latest/pulsar:1.19.0-incubating` present at [hub](https://hub.docker.com/r/apachepulsar/pulsar/tags/) so, if we always push latest then it would be better to keep `latest` as it won't require code change.? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #665: Use Docker images from apachepulsar hub org
merlimat commented on issue #665: Use Docker images from apachepulsar hub org URL: https://github.com/apache/incubator-pulsar/pull/665#issuecomment-321319738 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #664: Fixed broken links to admin-api/overview
merlimat commented on issue #664: Fixed broken links to admin-api/overview URL: https://github.com/apache/incubator-pulsar/pull/664#issuecomment-321326060 > instead of overview -> non-persistent-topics? Sure, I just did a `sed` to update the broken links, but it makes sense to link to the right sections now. Will fix that. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat closed pull request #660: Updated download links to Apache mirrors and prepared for multiple ve?
merlimat closed pull request #660: Updated download links to Apache mirrors and prepared for multiple ve? URL: https://github.com/apache/incubator-pulsar/pull/660 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rdhabalia commented on a change in pull request #664: Fixed broken links to admin-api/overview
rdhabalia commented on a change in pull request #664: Fixed broken links to admin-api/overview URL: https://github.com/apache/incubator-pulsar/pull/664#discussion_r132251584 ## File path: site/docs/latest/admin-api/partitioned-topics.md ## @@ -2,7 +2,7 @@ title: Managing partitioned topics --- -You can use Pulsar's [admin API](../../admin/AdminInterface) to create and manage partitioned topics. +You can use Pulsar's [admin API](../../admin-api/overview) to create and manage partitioned topics. Review comment: same ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #659: Pulsar connectors
merlimat commented on issue #659: Pulsar connectors URL: https://github.com/apache/incubator-pulsar/pull/659#issuecomment-321323049 @cckellogg There are 2 files missing headers: [WARNING] Missing header in: /home/jenkins/jenkins-slave/workspace/pulsar-pull-request/pulsar-connectors/core/src/main/java/org/apache/pulsar/connect/util/ConfigUtils.java [WARNING] Missing header in: /home/jenkins/jenkins-slave/workspace/pulsar-pull-request/pulsar-connectors/core/src/main/java/org/apache/pulsar/connect/util/Bytes.java This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #665: Use Docker images from apachepulsar hub org
merlimat commented on issue #665: Use Docker images from apachepulsar hub org URL: https://github.com/apache/incubator-pulsar/pull/665#issuecomment-321319738 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #663: When publishing website, we should not remove files that were not gen?
merlimat commented on issue #663: When publishing website, we should not remove files that were not gen? URL: https://github.com/apache/incubator-pulsar/pull/663#issuecomment-321306891 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on a change in pull request #660: Updated download links to Apache mirrors and prepared for multiple ve?
merlimat commented on a change in pull request #660: Updated download links to Apache mirrors and prepared for multiple ve? URL: https://github.com/apache/incubator-pulsar/pull/660#discussion_r132069835 ## File path: site/_config.yml ## @@ -25,9 +25,8 @@ pulsar_repo: https://github.com/apache/incubator-pulsar/tree/master baseurl: / destination: ../generated-site/content -versions: -- 1.18 -latest: 1.18 +current_version: 1.19.0-incubating Review comment: That's correct. Well, in addition to "snapshot" the docs in the `asf-site` branch, by copying the `docs/latest` directory into `docs/v1.20.0`. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rdhabalia commented on a change in pull request #660: Updated download links to Apache mirrors and prepared for multiple ve?
rdhabalia commented on a change in pull request #660: Updated download links to Apache mirrors and prepared for multiple ve? URL: https://github.com/apache/incubator-pulsar/pull/660#discussion_r132068359 ## File path: site/_config.yml ## @@ -25,9 +25,8 @@ pulsar_repo: https://github.com/apache/incubator-pulsar/tree/master baseurl: / destination: ../generated-site/content -versions: -- 1.18 -latest: 1.18 +current_version: 1.19.0-incubating Review comment: does it mean on next new release: we only need to change this file with new version? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #660: Updated download links to Apache mirrors and prepared for multiple ve?
merlimat commented on issue #660: Updated download links to Apache mirrors and prepared for multiple ve? URL: https://github.com/apache/incubator-pulsar/pull/660#issuecomment-321120532 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat opened a new pull request #660: Updated download links to Apache mirrors and prepared for multiple ve?
merlimat opened a new pull request #660: Updated download links to Apache mirrors and prepared for multiple ve? URL: https://github.com/apache/incubator-pulsar/pull/660 ?rsions docs ### Motivation With the 1.19.0-incubating release we will be storing the artifacts on the ASF dist servers, therefore we need to update the website with the right download links. Also, added links in the top-right bar to access documentation for multiple versions. NOTE: Do not merge this PR until the artifacts are already mirrored (will take several hours). cc/ @lucperkins This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cckellogg opened a new pull request #659: Pulsar connectors
cckellogg opened a new pull request #659: Pulsar connectors URL: https://github.com/apache/incubator-pulsar/pull/659 ### Motivation Start a framework work for building source and sink connectors. ### Modifications Added a base framework for building pulsar source and sink connectors. ### Result After this change you will be able to dump a topic to google cloud storage. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat closed pull request #632: Splitted Admin API reference into multiple pages
merlimat closed pull request #632: Splitted Admin API reference into multiple pages URL: https://github.com/apache/incubator-pulsar/pull/632 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting
merlimat commented on a change in pull request #634: PIP-3 : Introduce
message-dispatch rate limiting
URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r132040863
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/AdminResource.java
##
@@ -245,6 +245,19 @@ protected Policies getNamespacePolicies(String property,
String cluster, String
}
}
+protected LocalPolicies getNamespaceLocalPolicies(String property, String
cluster, String namespace) {
Review comment:
Though BundleData is different. It is not configuration. It is written back,
by the brokers themselves, into the local ZK because we want each cluster to be
able to split the bundles independently.
For rate limiting, we are talking about proper configuration.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rdhabalia commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting
rdhabalia commented on a change in pull request #634: PIP-3 : Introduce
message-dispatch rate limiting
URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r132036537
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/AdminResource.java
##
@@ -245,6 +245,19 @@ protected Policies getNamespacePolicies(String property,
String cluster, String
}
}
+protected LocalPolicies getNamespaceLocalPolicies(String property, String
cluster, String namespace) {
Review comment:
Ok. But I think we introduced local-policies to keep cluster-level policy
information such as `BundleData` and it keeps global-policies clean.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] merlimat commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting
merlimat commented on a change in pull request #634: PIP-3 : Introduce
message-dispatch rate limiting
URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r132034288
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java
##
@@ -1366,5 +1388,80 @@ public void markBatchMessagePublished() {
this.hasBatchMessagePublished = true;
}
+public RateLimiter getDispatchRateLimiter() {
+return dispatchRateLimiter;
+}
+
+public DispatchRateType getDispatchRateType() {
+return dispatchRateType;
+}
+
+public boolean tryDispatchPermit(long permits) {
+return permits <= 0 || dispatchRateLimiter == null
+// acquiring permits must be < configured msg-rate;
+|| dispatchRateLimiter.tryAcquire(permits);
+}
+
+public boolean hasMessageDispatchPermit() {
+return dispatchRateLimiter == null ||
dispatchRateLimiter.getAvailablePermits() > 0;
+}
+
+private void updateDispatchRate(LocalZooKeeperCacheService localZKCache) {
+final String path = joinPath(LOCAL_POLICIES_ROOT,
DestinationName.get(this.topic).getNamespace());
+try {
+Optional policies =
localZKCache.policiesCache().getAsync(path).get(cacheTimeOutInSec,
+SECONDS);
+if (policies.isPresent() && policies.get().dispatchRate != null
+&& policies.get().dispatchRate.topicDispatchRate > 0) {
+updateDispatchRate(policies.get().dispatchRate);
+log.info("[{}] configured message-dispatch rate configured at
policy {}}", this.topic,
+policies.get().dispatchRate);
+return;
+}
+} catch (Exception e) {
+log.warn("Failed to get message-rate for {}", this.topic, e);
+}
+DispatchRate dispatchRate =
brokerService.pulsar().getConfiguration().getDispatchRatePerTopicInMsg() > 0
Review comment:
It would require 2 rate limiters only if both limits are set. Otherwise one
of them, or both, will be null.
--------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] merlimat commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting
merlimat commented on a change in pull request #634: PIP-3 : Introduce
message-dispatch rate limiting
URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r132033540
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/AdminResource.java
##
@@ -245,6 +245,19 @@ protected Policies getNamespacePolicies(String property,
String cluster, String
}
}
+protected LocalPolicies getNamespaceLocalPolicies(String property, String
cluster, String namespace) {
Review comment:
That's fine, but it can be quite confusing for a user given that other
methods are changing the settings globally.
I think it could either be explicitely set for a particular cluster, or for
all the clusters. In order to clarify the scope. Eg:
```json
{
"rateLimit" : {
"dispatchRatePerTopicInMsg" : 1000.0,
"dispatchRatePerTopicInBytes" : 100.0,
"clusters" : {
"small-cluster" : {
"dispatchRatePerTopicInMsg" : 10.0,
"dispatchRatePerTopicInBytes" : 1000.0,
},
"large-cluster" : {
"dispatchRatePerTopicInMsg" : 100.0,
"dispatchRatePerTopicInBytes" : 10.0,
}
}
}
}
```
The advantage is that you can also see all the limit in a single place.
--------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rdhabalia commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting
rdhabalia commented on a change in pull request #634: PIP-3 : Introduce
message-dispatch rate limiting
URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r132031203
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java
##
@@ -1366,5 +1388,80 @@ public void markBatchMessagePublished() {
this.hasBatchMessagePublished = true;
}
+public RateLimiter getDispatchRateLimiter() {
+return dispatchRateLimiter;
+}
+
+public DispatchRateType getDispatchRateType() {
+return dispatchRateType;
+}
+
+public boolean tryDispatchPermit(long permits) {
+return permits <= 0 || dispatchRateLimiter == null
+// acquiring permits must be < configured msg-rate;
+|| dispatchRateLimiter.tryAcquire(permits);
+}
+
+public boolean hasMessageDispatchPermit() {
+return dispatchRateLimiter == null ||
dispatchRateLimiter.getAvailablePermits() > 0;
+}
+
+private void updateDispatchRate(LocalZooKeeperCacheService localZKCache) {
+final String path = joinPath(LOCAL_POLICIES_ROOT,
DestinationName.get(this.topic).getNamespace());
+try {
+Optional policies =
localZKCache.policiesCache().getAsync(path).get(cacheTimeOutInSec,
+SECONDS);
+if (policies.isPresent() && policies.get().dispatchRate != null
+&& policies.get().dispatchRate.topicDispatchRate > 0) {
+updateDispatchRate(policies.get().dispatchRate);
+log.info("[{}] configured message-dispatch rate configured at
policy {}}", this.topic,
+policies.get().dispatchRate);
+return;
+}
+} catch (Exception e) {
+log.warn("Failed to get message-rate for {}", this.topic, e);
+}
+DispatchRate dispatchRate =
brokerService.pulsar().getConfiguration().getDispatchRatePerTopicInMsg() > 0
Review comment:
> Actually, that could just be done inside DispatchRate class
Actually, using admin-api, we always define
`DispatchRateType=msgRate/byteRate` and throttling-value in `DispatchRate`
class.
However, for dynamic configuration we can't store complex-type so, created
two variables `dispatchRatePerTopicInMsg` and `dispatchRatePerTopicInByte`,
therefore, we need this logic when we have value set for both configuration.
> Ideally, if both limits are set, both of them should be applied.
This will require 2 `RateLimiter` objects for every topic which we use
across all the subscriptions. Do you think considering both limits would be
useful?
----
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] merlimat commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting
merlimat commented on a change in pull request #634: PIP-3 : Introduce
message-dispatch rate limiting
URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r132027792
##
File path:
pulsar-common/src/main/java/org/apache/pulsar/common/util/RateLimiter.java
##
@@ -0,0 +1,175 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.common.util;
+
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.ScheduledThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+import static com.google.common.base.Preconditions.checkArgument;
+
+/**
+ *
+ * A Rate Limiter that distributes permits at a configurable rate. Each {@link
#acquire()} blocks if necessary until a
+ * permit is available, and then takes it. Each {@link #tryAcquire()} tries to
acquire permits from available permits,
+ * it returns true if it succeed else returns false. Rate limiter release
configured permits at every configured rate
+ * time, so, on next ticket new fresh permits will be available.
+ *
+ * For example: if RateLimiter is configured to release 10 permits at every 1
second then RateLimiter will allow to
+ * acquire 10 permits at any time with in that 1 second.
+ *
+ */
+public class RateLimiter {
Review comment:
Ok, I found out the gist with the explanation and it makes sense. Can you
include the full explanation here as well? Also, would it make sense to have
the `1sec` period to be configurable as well? Eg: only rate limit if it's
exceeding the rate for 10 seconds instead of 1.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rdhabalia commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting
rdhabalia commented on a change in pull request #634: PIP-3 : Introduce
message-dispatch rate limiting
URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r132026790
##
File path:
pulsar-common/src/main/java/org/apache/pulsar/common/util/RateLimiter.java
##
@@ -0,0 +1,175 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.common.util;
+
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.ScheduledThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+import static com.google.common.base.Preconditions.checkArgument;
+
+/**
+ *
+ * A Rate Limiter that distributes permits at a configurable rate. Each {@link
#acquire()} blocks if necessary until a
+ * permit is available, and then takes it. Each {@link #tryAcquire()} tries to
acquire permits from available permits,
+ * it returns true if it succeed else returns false. Rate limiter release
configured permits at every configured rate
+ * time, so, on next ticket new fresh permits will be available.
+ *
+ * For example: if RateLimiter is configured to release 10 permits at every 1
second then RateLimiter will allow to
+ * acquire 10 permits at any time with in that 1 second.
+ *
+ */
+public class RateLimiter {
Review comment:
I have started with `Guava-RateLimiter` but ended up with custom one due to
2 main reasons:
(1) achieve per-second rate-limiting and (2) custom is faster compare to
guava. I have added both [Explanation
here](https://gist.github.com/rdhabalia/324519648b8a1008ef49d30c8f4a8bf2)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] merlimat closed pull request #658: Use $UID instead of $USER in docker build
merlimat closed pull request #658: Use $UID instead of $USER in docker build URL: https://github.com/apache/incubator-pulsar/pull/658 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #658: Use $UID instead of $USER in docker build
merlimat commented on issue #658: Use $UID instead of $USER in docker build URL: https://github.com/apache/incubator-pulsar/pull/658#issuecomment-321064613 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #632: Splitted Admin API reference into multiple pages
merlimat commented on issue #632: Splitted Admin API reference into multiple pages URL: https://github.com/apache/incubator-pulsar/pull/632#issuecomment-321056047 @rdhabalia updated This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting
merlimat commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r131990002 ## File path: conf/broker.conf ## @@ -106,6 +106,14 @@ maxUnackedMessagesPerBroker=0 # limit/2 messages maxUnackedMessagesPerSubscriptionOnBrokerBlocked=0.16 +# Default number of message dispatching throttling-limit for every topic. Using a value of 0, is disabling default +# message dispatch-throttling +dispatchRatePerTopicInMsg=0 Review comment: Can we include "throttling" in the var name? Like `dispatchThrottlingRatePerTopicInMsg` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting
merlimat commented on a change in pull request #634: PIP-3 : Introduce
message-dispatch rate limiting
URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r131991850
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/AdminResource.java
##
@@ -245,6 +245,19 @@ protected Policies getNamespacePolicies(String property,
String cluster, String
}
}
+protected LocalPolicies getNamespaceLocalPolicies(String property, String
cluster, String namespace) {
Review comment:
Shouldn't we configuring the throttling policies on the global zk?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] merlimat commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting
merlimat commented on a change in pull request #634: PIP-3 : Introduce
message-dispatch rate limiting
URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r132003921
##
File path:
pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/DispatchRate.java
##
@@ -0,0 +1,74 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.common.policies.data;
+
+public class DispatchRate {
+
+public long topicDispatchRate = -1;
+public DispatchRateType dispatchRateTye = DispatchRateType.messageRate;
+
+public enum DispatchRateType {
+messageRate, byteRate;
+}
+
+public DispatchRate() {
+this.topicDispatchRate = -1;
+this.dispatchRateTye = DispatchRateType.messageRate;
+}
+
+public DispatchRate(long topicMessageRate, DispatchRateType
dispatchRateTye) {
+super();
+this.topicDispatchRate = topicMessageRate;
+this.dispatchRateTye = dispatchRateTye;
+}
+
+@Override
+public int hashCode() {
+final int prime = 31;
+int result = 1;
+result = prime * result + ((dispatchRateTye == null) ? 0 :
dispatchRateTye.hashCode());
+result = prime * result + (int) (topicDispatchRate ^
(topicDispatchRate >>> 32));
+return result;
+}
+
+@Override
+public boolean equals(Object obj) {
+if (this == obj) {
+return true;
+}
+if (obj == null) {
+return false;
+}
+if (getClass() != obj.getClass()) {
+return false;
+}
+DispatchRate other = (DispatchRate) obj;
+if (dispatchRateTye != other.dispatchRateTye)
+return false;
+if (topicDispatchRate != other.topicDispatchRate)
+return false;
+return true;
+}
+
+@Override
+public String toString() {
Review comment:
```java
return Objects.toString(this);
```
?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] merlimat commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting
merlimat commented on a change in pull request #634: PIP-3 : Introduce
message-dispatch rate limiting
URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r131992335
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/Namespaces.java
##
@@ -810,6 +811,42 @@ public void splitNamespaceBundle(@PathParam("property")
String property, @PathPa
}
}
+@PUT
Review comment:
Should this `POST` rather than `PUT`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] merlimat commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting
merlimat commented on a change in pull request #634: PIP-3 : Introduce
message-dispatch rate limiting
URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r131996619
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
##
@@ -575,5 +578,31 @@ private void clearUnAckedMsgs(Consumer consumer) {
subscription.addUnAckedMessages(-unaAckedMsgs);
}
+public static class SendMessageInfo {
Review comment:
How is this information used?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] merlimat commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting
merlimat commented on a change in pull request #634: PIP-3 : Introduce
message-dispatch rate limiting
URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r132004308
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java
##
@@ -1366,5 +1388,80 @@ public void markBatchMessagePublished() {
this.hasBatchMessagePublished = true;
}
+public RateLimiter getDispatchRateLimiter() {
+return dispatchRateLimiter;
+}
+
+public DispatchRateType getDispatchRateType() {
+return dispatchRateType;
+}
+
+public boolean tryDispatchPermit(long permits) {
+return permits <= 0 || dispatchRateLimiter == null
+// acquiring permits must be < configured msg-rate;
+|| dispatchRateLimiter.tryAcquire(permits);
+}
+
+public boolean hasMessageDispatchPermit() {
+return dispatchRateLimiter == null ||
dispatchRateLimiter.getAvailablePermits() > 0;
+}
+
+private void updateDispatchRate(LocalZooKeeperCacheService localZKCache) {
+final String path = joinPath(LOCAL_POLICIES_ROOT,
DestinationName.get(this.topic).getNamespace());
+try {
+Optional policies =
localZKCache.policiesCache().getAsync(path).get(cacheTimeOutInSec,
+SECONDS);
+if (policies.isPresent() && policies.get().dispatchRate != null
+&& policies.get().dispatchRate.topicDispatchRate > 0) {
+updateDispatchRate(policies.get().dispatchRate);
+log.info("[{}] configured message-dispatch rate configured at
policy {}}", this.topic,
+policies.get().dispatchRate);
+return;
+}
+} catch (Exception e) {
+log.warn("Failed to get message-rate for {}", this.topic, e);
+}
+DispatchRate dispatchRate =
brokerService.pulsar().getConfiguration().getDispatchRatePerTopicInMsg() > 0
Review comment:
Actually, that could just be done inside `DispatchRate` class
--------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] merlimat commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting
merlimat commented on a change in pull request #634: PIP-3 : Introduce
message-dispatch rate limiting
URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r132003620
##
File path:
pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/DispatchRate.java
##
@@ -0,0 +1,74 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.common.policies.data;
+
+public class DispatchRate {
+
+public long topicDispatchRate = -1;
+public DispatchRateType dispatchRateTye = DispatchRateType.messageRate;
+
+public enum DispatchRateType {
+messageRate, byteRate;
+}
+
+public DispatchRate() {
+this.topicDispatchRate = -1;
+this.dispatchRateTye = DispatchRateType.messageRate;
+}
+
+public DispatchRate(long topicMessageRate, DispatchRateType
dispatchRateTye) {
+super();
+this.topicDispatchRate = topicMessageRate;
+this.dispatchRateTye = dispatchRateTye;
+}
+
+@Override
+public int hashCode() {
Review comment:
Unless `hashCode()` is used in a very critical place, just use
`Objects.hashCode(var1, var2)`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] merlimat commented on a change in pull request #634: PIP-3 : Introduce message-dispatch rate limiting
merlimat commented on a change in pull request #634: PIP-3 : Introduce
message-dispatch rate limiting
URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r132003259
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java
##
@@ -1366,5 +1388,80 @@ public void markBatchMessagePublished() {
this.hasBatchMessagePublished = true;
}
+public RateLimiter getDispatchRateLimiter() {
+return dispatchRateLimiter;
+}
+
+public DispatchRateType getDispatchRateType() {
+return dispatchRateType;
+}
+
+public boolean tryDispatchPermit(long permits) {
+return permits <= 0 || dispatchRateLimiter == null
+// acquiring permits must be < configured msg-rate;
+|| dispatchRateLimiter.tryAcquire(permits);
+}
+
+public boolean hasMessageDispatchPermit() {
+return dispatchRateLimiter == null ||
dispatchRateLimiter.getAvailablePermits() > 0;
+}
+
+private void updateDispatchRate(LocalZooKeeperCacheService localZKCache) {
+final String path = joinPath(LOCAL_POLICIES_ROOT,
DestinationName.get(this.topic).getNamespace());
+try {
+Optional policies =
localZKCache.policiesCache().getAsync(path).get(cacheTimeOutInSec,
+SECONDS);
+if (policies.isPresent() && policies.get().dispatchRate != null
+&& policies.get().dispatchRate.topicDispatchRate > 0) {
+updateDispatchRate(policies.get().dispatchRate);
+log.info("[{}] configured message-dispatch rate configured at
policy {}}", this.topic,
+policies.get().dispatchRate);
+return;
+}
+} catch (Exception e) {
+log.warn("Failed to get message-rate for {}", this.topic, e);
+}
+DispatchRate dispatchRate =
brokerService.pulsar().getConfiguration().getDispatchRatePerTopicInMsg() > 0
Review comment:
Ideally, if both limits are set, both of them should be applied.
--------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] merlimat commented on issue #658: Use $UID instead of $USER in docker build
merlimat commented on issue #658: Use $UID instead of $USER in docker build URL: https://github.com/apache/incubator-pulsar/pull/658#issuecomment-321050797 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #658: Use $UID instead of $USER in docker build
merlimat commented on issue #658: Use $UID instead of $USER in docker build URL: https://github.com/apache/incubator-pulsar/pull/658#issuecomment-321050797 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on a change in pull request #632: Splitted Admin API reference into multiple pages
merlimat commented on a change in pull request #632: Splitted Admin API
reference into multiple pages
URL: https://github.com/apache/incubator-pulsar/pull/632#discussion_r132002230
##
File path: site/docs/latest/admin-api/permissions.md
##
@@ -1,155 +1,6 @@
-Pulsar {% popover namespaces %} are logical groupings of {% popover topics %}.
-
-Namespaces can be managed via:
-
-* The [`namespaces`](../../reference/CliTools#pulsar-admin-clusters) command
of the [`pulsar-admin`](../../reference/CliTools#pulsar-admin) tool
-* The `/admin/namespaces` endpoint of the admin [REST
API](../../reference/RestApi)
-* The `namespaces` method of the {% javadoc PulsarAdmin admin
org.apache.pulsar.client.admin.PulsarAdmin %} object in the [Java
API](../../applications/JavaClient)
-
-### Create
-
-You can create new namespaces under a given {% popover property %} and within
a Pulsar {% popover cluster %}.
-
- pulsar-admin
-
-Use the [`create`](../../reference/CliTools#pulsar-admin-namespaces-create)
subcommand and specify the namespace by name:
-
-```shell
-$ pulsar-admin namespaces create test-property/cl1/ns1
-```
-
- REST API
-
-{% endpoint PUT /admin/namespaces/:property/:cluster/:namespace %}
-
-[More
info](../../reference/RestApi#/admin/namespaces/:property/:cluster/:namespace)
-
- Java
-
-```java
-admin.namespaces().createNamespace(namespace);
-```
-
-### Get policies
-
-You can fetch the current policies associated with a namespace at any time.
-
- pulsar-admin
-
-Use the
[`policies`](../../reference/CliTools#pulsar-admin-namespaces-policies)
subcommand and specify the namespace:
-
-```shell
-$ pulsar-admin namespaces policies test-property/cl1/ns1
-{
- "auth_policies": {
-"namespace_auth": {},
-"destination_auth": {}
- },
- "replication_clusters": [],
- "bundles_activated": true,
- "bundles": {
-"boundaries": [
- "0x",
- "0x"
-],
-"numBundles": 1
- },
- "backlog_quota_map": {},
- "persistence": null,
- "latency_stats_sample_rate": {},
- "message_ttl_in_seconds": 0,
- "retention_policies": null,
- "deleted": false
-}
-```
-
- REST API
-
-{% endpoint GET /admin/namespaces/:property/:cluster/:namespace %}
-
-[More
info](../../reference/RestApi#/admin/namespaces/:property/:cluster/:namespace)
-
- Java
-
-```java
-admin.namespaces().getPolicies(namespace);
-```
-
-### List namespaces within a property
-
-You can list all namespaces within a given Pulsar {% popover property %}.
-
- pulsar-admin
-
-Use the [`list`](../../reference/CliTools#pulsar-admin-namespaces-list)
subcommand and specify the property:
-
-```shell
-$ pulsar-admin namespaces list test-property
-test-property/cl1/ns1
-test-property/cl2/ns2
-```
-
- REST API
-
-{% endpoint GET /admin/namespaces/:property %}
-
-[More info](../../reference/RestApi#/admin/namespaces/:property)
-
- Java
-
-```java
-admin.namespaces().getNamespaces(property);
-```
-
-### List namespaces within a cluster
-
-You can list all namespaces within a given Pulsar {% popover cluster %}.
-
- pulsar-admin
-
-Use the
[`list-cluster`](../../reference/CliTools#pulsar-admin-namespaces-list-cluster)
subcommand and specify the cluster:
-
-```shell
-$ pulsar-admin namespaces list-cluster test-property/cl1
-test-property/cl1/ns1
-test-property/cl1/ns1
-```
-
- REST API
-
-{% endpoint GET /admin/namespaces/:property/:cluster %}
-
-[More info](../../reference/RestApi#/admin/namespaces/:property/:cluster)
-
- Java
-
-```java
-admin.namespaces().getNamespaces(property, cluster);
-```
-
-### Delete
-
-You can delete existing namespaces from a property/cluster.
-
- pulsar-admin
-
-Use the [`delete`](../../reference/CliTools#pulsar-admin-namespaces-delete)
subcommand and specify the namespace:
-
-```shell
-$ pulsar-admin namespaces delete test-property/cl1/ns1
-```
-
- REST
-
-{% endpoint DELETE /admin/namespaces/:property/:cluster/:namespace %}
-
-[More
info](../../reference/RestApi#/admin/namespaces/:property/:cluster/:namespace)
-
- Java
-
-```java
-admin.namespaces().deleteNamespace(namespace);
-```
+---
+title: Managing permissions
+---
Review comment:
You're right, I broke it down in the wrong place. Put all these operations
under namespace again.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] merlimat commented on a change in pull request #632: Splitted Admin API reference into multiple pages
merlimat commented on a change in pull request #632: Splitted Admin API
reference into multiple pages
URL: https://github.com/apache/incubator-pulsar/pull/632#discussion_r132001213
##
File path: site/docs/latest/admin-api/permissions.md
##
@@ -1,155 +1,6 @@
-Pulsar {% popover namespaces %} are logical groupings of {% popover topics %}.
-
-Namespaces can be managed via:
-
-* The [`namespaces`](../../reference/CliTools#pulsar-admin-clusters) command
of the [`pulsar-admin`](../../reference/CliTools#pulsar-admin) tool
-* The `/admin/namespaces` endpoint of the admin [REST
API](../../reference/RestApi)
-* The `namespaces` method of the {% javadoc PulsarAdmin admin
org.apache.pulsar.client.admin.PulsarAdmin %} object in the [Java
API](../../applications/JavaClient)
-
-### Create
-
-You can create new namespaces under a given {% popover property %} and within
a Pulsar {% popover cluster %}.
-
- pulsar-admin
-
-Use the [`create`](../../reference/CliTools#pulsar-admin-namespaces-create)
subcommand and specify the namespace by name:
-
-```shell
-$ pulsar-admin namespaces create test-property/cl1/ns1
-```
-
- REST API
-
-{% endpoint PUT /admin/namespaces/:property/:cluster/:namespace %}
-
-[More
info](../../reference/RestApi#/admin/namespaces/:property/:cluster/:namespace)
-
- Java
-
-```java
-admin.namespaces().createNamespace(namespace);
-```
-
-### Get policies
-
-You can fetch the current policies associated with a namespace at any time.
-
- pulsar-admin
-
-Use the
[`policies`](../../reference/CliTools#pulsar-admin-namespaces-policies)
subcommand and specify the namespace:
-
-```shell
-$ pulsar-admin namespaces policies test-property/cl1/ns1
-{
- "auth_policies": {
-"namespace_auth": {},
-"destination_auth": {}
- },
- "replication_clusters": [],
- "bundles_activated": true,
- "bundles": {
-"boundaries": [
- "0x",
- "0x"
-],
-"numBundles": 1
- },
- "backlog_quota_map": {},
- "persistence": null,
- "latency_stats_sample_rate": {},
- "message_ttl_in_seconds": 0,
- "retention_policies": null,
- "deleted": false
-}
-```
-
- REST API
-
-{% endpoint GET /admin/namespaces/:property/:cluster/:namespace %}
-
-[More
info](../../reference/RestApi#/admin/namespaces/:property/:cluster/:namespace)
-
- Java
-
-```java
-admin.namespaces().getPolicies(namespace);
-```
-
-### List namespaces within a property
-
-You can list all namespaces within a given Pulsar {% popover property %}.
-
- pulsar-admin
-
-Use the [`list`](../../reference/CliTools#pulsar-admin-namespaces-list)
subcommand and specify the property:
-
-```shell
-$ pulsar-admin namespaces list test-property
-test-property/cl1/ns1
-test-property/cl2/ns2
-```
-
- REST API
-
-{% endpoint GET /admin/namespaces/:property %}
-
-[More info](../../reference/RestApi#/admin/namespaces/:property)
-
- Java
-
-```java
-admin.namespaces().getNamespaces(property);
-```
-
-### List namespaces within a cluster
-
-You can list all namespaces within a given Pulsar {% popover cluster %}.
-
- pulsar-admin
-
-Use the
[`list-cluster`](../../reference/CliTools#pulsar-admin-namespaces-list-cluster)
subcommand and specify the cluster:
-
-```shell
-$ pulsar-admin namespaces list-cluster test-property/cl1
-test-property/cl1/ns1
-test-property/cl1/ns1
-```
-
- REST API
-
-{% endpoint GET /admin/namespaces/:property/:cluster %}
-
-[More info](../../reference/RestApi#/admin/namespaces/:property/:cluster)
-
- Java
-
-```java
-admin.namespaces().getNamespaces(property, cluster);
-```
-
-### Delete
-
-You can delete existing namespaces from a property/cluster.
-
- pulsar-admin
-
-Use the [`delete`](../../reference/CliTools#pulsar-admin-namespaces-delete)
subcommand and specify the namespace:
-
-```shell
-$ pulsar-admin namespaces delete test-property/cl1/ns1
-```
-
- REST
-
-{% endpoint DELETE /admin/namespaces/:property/:cluster/:namespace %}
-
-[More
info](../../reference/RestApi#/admin/namespaces/:property/:cluster/:namespace)
-
- Java
-
-```java
-admin.namespaces().deleteNamespace(namespace);
-```
+---
+title: Managing permissions
+---
## Managing permissions
Review comment:
I'll change the title below (` ## Managing permissions`) to keep it
consistent with the other admin api pages
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] merlimat commented on a change in pull request #632: Splitted Admin API reference into multiple pages
merlimat commented on a change in pull request #632: Splitted Admin API reference into multiple pages URL: https://github.com/apache/incubator-pulsar/pull/632#discussion_r132001068 ## File path: site/docs/latest/admin-api/partitioned-topics.md ## @@ -1,9 +1,15 @@ +--- +title: Partitioned topics Review comment: Yes, fixed This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rdhabalia commented on a change in pull request #658: Use $UID instead of $USER in docker build
rdhabalia commented on a change in pull request #658: Use $UID instead of $USER in docker build URL: https://github.com/apache/incubator-pulsar/pull/658#discussion_r131995895 ## File path: site/scripts/docker-build.sh ## @@ -35,6 +35,9 @@ echo " Build Pulsar website using image $IMAGE" docker pull $IMAGE -DOCKER_CMD="docker run --user $USER -i -v $ROOT_DIR:/pulsar $IMAGE" +CI_USER=$(id -u) +CI_GROUP=$(id -g) -$DOCKER_CMD bash -l -c 'cd /pulsar/site && rvm use . && make setup && make protobuf_doc_gen && make build' +DOCKER_CMD="docker run -i -e CI_USER=$CI_USER -e CI_GROUP=$CI_GROUP -v $ROOT_DIR:/pulsar $IMAGE" + +$DOCKER_CMD bash -l -c 'cd /pulsar/site && rvm use . && make setup && make protobuf_doc_gen && make build && chown -R $CI_USER:$CI_GROUP /pulsar/generated-site' Review comment: did you forget `'` in the end of the line? ---- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #658: Use $UID instead of $USER in docker build
merlimat commented on issue #658: Use $UID instead of $USER in docker build URL: https://github.com/apache/incubator-pulsar/pull/658#issuecomment-321043000 @rdhabalia Please take a look. I've verified the website now build on Jenkins with this fix. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat opened a new pull request #658: Use $UID instead of $USER in docker build
merlimat opened a new pull request #658: Use $UID instead of $USER in docker build URL: https://github.com/apache/incubator-pulsar/pull/658 ### Motivation In #652, there was an attempt to fix the website build by using current user in Docker. It doesn't work in Jenkins and the (verified) solution here is to keep using "root" inside the container, but change the owner of these files after the build, so that Jenkins can read and write from outside the container. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat closed pull request #654: Do not delete inactive topics if they have retention policy
merlimat closed pull request #654: Do not delete inactive topics if they have retention policy URL: https://github.com/apache/incubator-pulsar/pull/654 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat closed pull request #641: Added missing copyright notice for Circe library
merlimat closed pull request #641: Added missing copyright notice for Circe library URL: https://github.com/apache/incubator-pulsar/pull/641 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat opened a new issue #657: Intermittent C++ test failure BatchMessageTest.testProducerConfig
merlimat opened a new issue #657: Intermittent C++ test failure BatchMessageTest.testProducerConfig URL: https://github.com/apache/incubator-pulsar/issues/657 As seen in https://builds.apache.org/job/pulsar-master/22/console ``` [17/75] BatchMessageTest.testProducerConfig (30 ms) log4cxx: Could not read configuration file [log4cxx.conf]. Note: Google Test filter = BatchMessageTest.testProducerConfig [==] Running 1 test from 1 test case. [--] Global test environment set-up. [--] 1 test from BatchMessageTest [ RUN ] BatchMessageTest.testProducerConfig [WARNING] /usr/src/gtest/src/gtest-death-test.cc:825:: Death tests use fork(), which is unsafe particularly in a threaded context. For this test, Google Test couldn't detect the number of threads. /pulsar/pulsar-client-cpp/tests/BatchMessageTest.cc:75: Failure Death test: conf.setBatchingMaxMessages(1) Result: failed to die. Error msg: [ DEATH ] [ FAILED ] BatchMessageTest.testProducerConfig (5 ms) [--] 1 test from BatchMessageTest (5 ms total) [--] Global test environment tear-down [==] 1 test from 1 test case ran. (6 ms total) [ PASSED ] 0 tests. [ FAILED ] 1 test, listed below: [ FAILED ] BatchMessageTest.testProducerConfig 1 FAILED TEST [17/75] BatchMessageTest.testProducerConfig returned/aborted with exit code 1 (30 ms) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rdhabalia commented on a change in pull request #632: Splitted Admin API reference into multiple pages
rdhabalia commented on a change in pull request #632: Splitted Admin API reference into multiple pages URL: https://github.com/apache/incubator-pulsar/pull/632#discussion_r131798565 ## File path: site/docs/latest/admin-api/partitioned-topics.md ## @@ -1,9 +1,15 @@ +--- +title: Partitioned topics Review comment: `Managing Partitioned topics`? in other doc, we have started with `Managing`. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rdhabalia commented on a change in pull request #632: Splitted Admin API reference into multiple pages
rdhabalia commented on a change in pull request #632: Splitted Admin API
reference into multiple pages
URL: https://github.com/apache/incubator-pulsar/pull/632#discussion_r131799120
##
File path: site/docs/latest/admin-api/permissions.md
##
@@ -1,155 +1,6 @@
-Pulsar {% popover namespaces %} are logical groupings of {% popover topics %}.
-
-Namespaces can be managed via:
-
-* The [`namespaces`](../../reference/CliTools#pulsar-admin-clusters) command
of the [`pulsar-admin`](../../reference/CliTools#pulsar-admin) tool
-* The `/admin/namespaces` endpoint of the admin [REST
API](../../reference/RestApi)
-* The `namespaces` method of the {% javadoc PulsarAdmin admin
org.apache.pulsar.client.admin.PulsarAdmin %} object in the [Java
API](../../applications/JavaClient)
-
-### Create
-
-You can create new namespaces under a given {% popover property %} and within
a Pulsar {% popover cluster %}.
-
- pulsar-admin
-
-Use the [`create`](../../reference/CliTools#pulsar-admin-namespaces-create)
subcommand and specify the namespace by name:
-
-```shell
-$ pulsar-admin namespaces create test-property/cl1/ns1
-```
-
- REST API
-
-{% endpoint PUT /admin/namespaces/:property/:cluster/:namespace %}
-
-[More
info](../../reference/RestApi#/admin/namespaces/:property/:cluster/:namespace)
-
- Java
-
-```java
-admin.namespaces().createNamespace(namespace);
-```
-
-### Get policies
-
-You can fetch the current policies associated with a namespace at any time.
-
- pulsar-admin
-
-Use the
[`policies`](../../reference/CliTools#pulsar-admin-namespaces-policies)
subcommand and specify the namespace:
-
-```shell
-$ pulsar-admin namespaces policies test-property/cl1/ns1
-{
- "auth_policies": {
-"namespace_auth": {},
-"destination_auth": {}
- },
- "replication_clusters": [],
- "bundles_activated": true,
- "bundles": {
-"boundaries": [
- "0x",
- "0x"
-],
-"numBundles": 1
- },
- "backlog_quota_map": {},
- "persistence": null,
- "latency_stats_sample_rate": {},
- "message_ttl_in_seconds": 0,
- "retention_policies": null,
- "deleted": false
-}
-```
-
- REST API
-
-{% endpoint GET /admin/namespaces/:property/:cluster/:namespace %}
-
-[More
info](../../reference/RestApi#/admin/namespaces/:property/:cluster/:namespace)
-
- Java
-
-```java
-admin.namespaces().getPolicies(namespace);
-```
-
-### List namespaces within a property
-
-You can list all namespaces within a given Pulsar {% popover property %}.
-
- pulsar-admin
-
-Use the [`list`](../../reference/CliTools#pulsar-admin-namespaces-list)
subcommand and specify the property:
-
-```shell
-$ pulsar-admin namespaces list test-property
-test-property/cl1/ns1
-test-property/cl2/ns2
-```
-
- REST API
-
-{% endpoint GET /admin/namespaces/:property %}
-
-[More info](../../reference/RestApi#/admin/namespaces/:property)
-
- Java
-
-```java
-admin.namespaces().getNamespaces(property);
-```
-
-### List namespaces within a cluster
-
-You can list all namespaces within a given Pulsar {% popover cluster %}.
-
- pulsar-admin
-
-Use the
[`list-cluster`](../../reference/CliTools#pulsar-admin-namespaces-list-cluster)
subcommand and specify the cluster:
-
-```shell
-$ pulsar-admin namespaces list-cluster test-property/cl1
-test-property/cl1/ns1
-test-property/cl1/ns1
-```
-
- REST API
-
-{% endpoint GET /admin/namespaces/:property/:cluster %}
-
-[More info](../../reference/RestApi#/admin/namespaces/:property/:cluster)
-
- Java
-
-```java
-admin.namespaces().getNamespaces(property, cluster);
-```
-
-### Delete
-
-You can delete existing namespaces from a property/cluster.
-
- pulsar-admin
-
-Use the [`delete`](../../reference/CliTools#pulsar-admin-namespaces-delete)
subcommand and specify the namespace:
-
-```shell
-$ pulsar-admin namespaces delete test-property/cl1/ns1
-```
-
- REST
-
-{% endpoint DELETE /admin/namespaces/:property/:cluster/:namespace %}
-
-[More
info](../../reference/RestApi#/admin/namespaces/:property/:cluster/:namespace)
-
- Java
-
-```java
-admin.namespaces().deleteNamespace(namespace);
-```
+---
+title: Managing permissions
+---
Review comment:
actually, if you see the content of this file: it has `set replication
cluster`, `split-bundle`, `unload-bundle`,.. it doesn't seem like a permissions
page? Do we really need this page, or we can keep this content into
namespaces.md only?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] rdhabalia commented on a change in pull request #632: Splitted Admin API reference into multiple pages
rdhabalia commented on a change in pull request #632: Splitted Admin API
reference into multiple pages
URL: https://github.com/apache/incubator-pulsar/pull/632#discussion_r131798719
##
File path: site/docs/latest/admin-api/permissions.md
##
@@ -1,155 +1,6 @@
-Pulsar {% popover namespaces %} are logical groupings of {% popover topics %}.
-
-Namespaces can be managed via:
-
-* The [`namespaces`](../../reference/CliTools#pulsar-admin-clusters) command
of the [`pulsar-admin`](../../reference/CliTools#pulsar-admin) tool
-* The `/admin/namespaces` endpoint of the admin [REST
API](../../reference/RestApi)
-* The `namespaces` method of the {% javadoc PulsarAdmin admin
org.apache.pulsar.client.admin.PulsarAdmin %} object in the [Java
API](../../applications/JavaClient)
-
-### Create
-
-You can create new namespaces under a given {% popover property %} and within
a Pulsar {% popover cluster %}.
-
- pulsar-admin
-
-Use the [`create`](../../reference/CliTools#pulsar-admin-namespaces-create)
subcommand and specify the namespace by name:
-
-```shell
-$ pulsar-admin namespaces create test-property/cl1/ns1
-```
-
- REST API
-
-{% endpoint PUT /admin/namespaces/:property/:cluster/:namespace %}
-
-[More
info](../../reference/RestApi#/admin/namespaces/:property/:cluster/:namespace)
-
- Java
-
-```java
-admin.namespaces().createNamespace(namespace);
-```
-
-### Get policies
-
-You can fetch the current policies associated with a namespace at any time.
-
- pulsar-admin
-
-Use the
[`policies`](../../reference/CliTools#pulsar-admin-namespaces-policies)
subcommand and specify the namespace:
-
-```shell
-$ pulsar-admin namespaces policies test-property/cl1/ns1
-{
- "auth_policies": {
-"namespace_auth": {},
-"destination_auth": {}
- },
- "replication_clusters": [],
- "bundles_activated": true,
- "bundles": {
-"boundaries": [
- "0x",
- "0x"
-],
-"numBundles": 1
- },
- "backlog_quota_map": {},
- "persistence": null,
- "latency_stats_sample_rate": {},
- "message_ttl_in_seconds": 0,
- "retention_policies": null,
- "deleted": false
-}
-```
-
- REST API
-
-{% endpoint GET /admin/namespaces/:property/:cluster/:namespace %}
-
-[More
info](../../reference/RestApi#/admin/namespaces/:property/:cluster/:namespace)
-
- Java
-
-```java
-admin.namespaces().getPolicies(namespace);
-```
-
-### List namespaces within a property
-
-You can list all namespaces within a given Pulsar {% popover property %}.
-
- pulsar-admin
-
-Use the [`list`](../../reference/CliTools#pulsar-admin-namespaces-list)
subcommand and specify the property:
-
-```shell
-$ pulsar-admin namespaces list test-property
-test-property/cl1/ns1
-test-property/cl2/ns2
-```
-
- REST API
-
-{% endpoint GET /admin/namespaces/:property %}
-
-[More info](../../reference/RestApi#/admin/namespaces/:property)
-
- Java
-
-```java
-admin.namespaces().getNamespaces(property);
-```
-
-### List namespaces within a cluster
-
-You can list all namespaces within a given Pulsar {% popover cluster %}.
-
- pulsar-admin
-
-Use the
[`list-cluster`](../../reference/CliTools#pulsar-admin-namespaces-list-cluster)
subcommand and specify the cluster:
-
-```shell
-$ pulsar-admin namespaces list-cluster test-property/cl1
-test-property/cl1/ns1
-test-property/cl1/ns1
-```
-
- REST API
-
-{% endpoint GET /admin/namespaces/:property/:cluster %}
-
-[More info](../../reference/RestApi#/admin/namespaces/:property/:cluster)
-
- Java
-
-```java
-admin.namespaces().getNamespaces(property, cluster);
-```
-
-### Delete
-
-You can delete existing namespaces from a property/cluster.
-
- pulsar-admin
-
-Use the [`delete`](../../reference/CliTools#pulsar-admin-namespaces-delete)
subcommand and specify the namespace:
-
-```shell
-$ pulsar-admin namespaces delete test-property/cl1/ns1
-```
-
- REST
-
-{% endpoint DELETE /admin/namespaces/:property/:cluster/:namespace %}
-
-[More
info](../../reference/RestApi#/admin/namespaces/:property/:cluster/:namespace)
-
- Java
-
-```java
-admin.namespaces().deleteNamespace(namespace);
-```
+---
+title: Managing permissions
+---
## Managing permissions
Review comment:
as title name is `Managing permissions`, should we just write `permissions`
here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] lucperkins opened a new pull request #656: Issue #655: fix misleading info on retention policies
lucperkins opened a new pull request #656: Issue #655: fix misleading info on retention policies URL: https://github.com/apache/incubator-pulsar/pull/656 ### Motivation This PR addresses issue #655. ### Modifications Misleading explanations and code samples have been removed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rdhabalia commented on issue #634: PIP-3 : Introduce message-dispatch rate limiting
rdhabalia commented on issue #634: PIP-3 : Introduce message-dispatch rate limiting URL: https://github.com/apache/incubator-pulsar/pull/634#issuecomment-320817486 @merlimat as per your suggestion in #402, I have added rate-limiting by msg-rate and byte-rate. I will update the doc once you get a chance to review it. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat closed pull request #652: Build website using the current user rather than root
merlimat closed pull request #652: Build website using the current user rather than root URL: https://github.com/apache/incubator-pulsar/pull/652 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat opened a new pull request #654: Do not delete inactive topics if they have retention policy
merlimat opened a new pull request #654: Do not delete inactive topics if they have retention policy URL: https://github.com/apache/incubator-pulsar/pull/654 ### Motivation When a retention policy is set, an "inactive" topic should not be garbage collected until the retention period has expired. Even if there are no subscriptions or active producers/consumers, the data that was published on the topic should be retained for the configured time. ### Modifications Check for the retention policy config during GC cycle. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat opened a new pull request #653: Removed duplicated ASF headers from BookKeeper
merlimat opened a new pull request #653: Removed duplicated ASF headers from BookKeeper URL: https://github.com/apache/incubator-pulsar/pull/653 ### Motivation As reported on the incubator mailing list, there are few files that were imported from BookKeeper that have double license headers (including a copyright line which shouldn't actually be there). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat opened a new pull request #652: Build website using the current user rather than root
merlimat opened a new pull request #652: Build website using the current user rather than root URL: https://github.com/apache/incubator-pulsar/pull/652 ### Motivation Trying to publish the website after the build on Jenkins, it appears that there are some permissions related issue regarding the generated files. This should be solved by forcing Docker to use the current user to run the container for the website build. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #632: Splitted Admin API reference into multiple pages
merlimat commented on issue #632: Splitted Admin API reference into multiple pages URL: https://github.com/apache/incubator-pulsar/pull/632#issuecomment-320798994 @rdhabalia Can you take a look at this one too? This is the change we had talked on the slack discussion This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on issue #641: Added missing copyright notice for Circe library
merlimat commented on issue #641: Added missing copyright notice for Circe library URL: https://github.com/apache/incubator-pulsar/pull/641#issuecomment-320729580 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat closed pull request #643: Ensure the Python client works with Python-3
merlimat closed pull request #643: Ensure the Python client works with Python-3 URL: https://github.com/apache/incubator-pulsar/pull/643 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat opened a new issue #651: Intermittent test failure PulsarClientToolTest.testInitialzation
merlimat opened a new issue #651: Intermittent test failure PulsarClientToolTest.testInitialzation URL: https://github.com/apache/incubator-pulsar/issues/651 As seen in https://builds.apache.org/job/pulsar-master/org.apache.pulsar$pulsar-client-tools/17/testReport/junit/org.apache.pulsar.client.cli/PulsarClientToolTest/testInitialzation/ ``` Error Message Method org.testng.internal.TestNGMethod.testInitialzation() didn't finish within the time-out 1 Stacktrace org.testng.internal.thread.ThreadTimeoutException: Method org.testng.internal.TestNGMethod.testInitialzation() didn't finish within the time-out 1 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] lucperkins commented on a change in pull request #647: Add link checker and fix broken links
lucperkins commented on a change in pull request #647: Add link checker and fix broken links URL: https://github.com/apache/incubator-pulsar/pull/647#discussion_r131753611 ## File path: site/Makefile ## @@ -34,11 +34,14 @@ python_setup: sudo pip install pdoc pygments setup: ruby_setup + scripts/html-setup.sh osx + +linkcheck_macos: + ~/bin/htmltest-osx Review comment: Good call. Fixing. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on a change in pull request #628: Docker image to build website in Jenkins
merlimat commented on a change in pull request #628: Docker image to build website in Jenkins URL: https://github.com/apache/incubator-pulsar/pull/628#discussion_r131746188 ## File path: build/docker/Dockerfile ## @@ -31,3 +32,22 @@ RUN cd /usr/src/gtest && cmake . && make && cp libgtest.a /usr/lib RUN git clone https://github.com/google/gtest-parallel.git ENV JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64 + +## Website build dependencies + +# Install Ruby-2.4.1 +RUN apt-get install -y +RUN gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB && \ Review comment: I followed the instruction on Ruby page on how to install on Ubuntu 16.04. I don't think this should be an issue since the release was signed with that key and that won't change. The key is verified automatically in the `rvm` tool. In any case this won't affect our build, since this only happens when re-creating the Pulsar build image, not when actually doing our CI job. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] merlimat commented on a change in pull request #628: Docker image to build website in Jenkins
merlimat commented on a change in pull request #628: Docker image to build
website in Jenkins
URL: https://github.com/apache/incubator-pulsar/pull/628#discussion_r131746296
##
File path: site/api/python/index.html
##
@@ -1125,653 +1058,649 @@ pulsar
module
Install from PyPI
Download Python wheel binary files for MacOS and Linux
directly from the PyPI archive.
-$ sudo pip install pulsar-client
-
-
-
+#!shell
+$ sudo pip install pulsar-client
+
Install from sources
Follow the instructions to compile the Pulsar C++ client library. This
method
will also build the Python binding for the library.
To install the Python bindings:
-$ cd
pulsar-client-cpp/python
+#!shell
+$ cd pulsar-client-cpp/python
$ sudo python setup.py install
-
-
-
+
Examples
Producer example
-import pulsar
-
-client = pulsar.Client(pulsar://localhost:6650)
+#!python
+import pulsar
-producer = client.create_producer(
-persistent://sample/standalone/ns/my-topic)
+client = pulsar.Client('pulsar://localhost:6650')
-for i in
range(10):
-producer.send(Hello-%d % i)
-
-client.close()
-
+producer = client.create_producer(
+'persistent://sample/standalone/ns/my-topic')
+for i in range(10):
+producer.send('Hello-%d' % i)
+client.close()
+
Consumer Example
-import pulsar
-
-client = pulsar.Client(pulsar://localhost:6650)
-consumer = client.subscribe(
-persistent://sample/standalone/ns/my-topic,
-my-sub)
-
-while True:
-msg = consumer.receive()
-print(Received message %s id=%s, msg.data(), msg.message_id())
-consumer.acknowledge(msg)
+#!python
+import pulsar
-client.close()
-
+client = pulsar.Client('pulsar://localhost:6650')
+consumer = client.subscribe(
+'persistent://sample/standalone/ns/my-topic',
+'my-sub')
+while True:
+msg = consumer.receive()
+print("Received message '%s' id='%s'", msg.data(), msg.message_id())
Review comment:
This `index.html` file needs to be removed. It's already in `.gitignore` and
it now gets generated during the build.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
