date:20190814

[GitHub] [pulsar] congbobo184 commented on issue #4838: NPE is thrown when a consumer consumes a partitioned topic with struct schema

2019-08-14 Thread GitBox

congbobo184 commented on issue #4838: NPE is thrown when a consumer consumes a 
partitioned topic with struct schema
URL: https://github.com/apache/pulsar/issues/4838#issuecomment-521135032
 
 
   what are the steps for the recurrence of this problem and 
   ```
   ./pulsar-admin topics create-partitioned-topic -p 3 
persistent://public/default/test-npe3
   
PulsarClient client = 
PulsarClient.builder().serviceUrl("pulsar://127.0.0.1:6650").build();
   Consumer consumer = client
   .newConsumer(Schema.AVRO(MessageProcess.class))
   .topic("persistent://public/default/test-npe3")
   .subscriptionName("test")
   .subscribe();
   while (true) {
   Message message = consumer.receive();
   consumer.acknowledge(message);
   
   }
   ```
   I did not reproduce it in this way.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar-translation] yangleimiao opened a new pull request #2: update personal address

2019-08-14 Thread GitBox

yangleimiao opened a new pull request #2: update personal address
URL: https://github.com/apache/pulsar-translation/pull/2

<--
### Contribution Checklist

- Name the pull request in the form "[Issue XYZ][component] Title of the
pull request", where *XYZ* should be replaced by the actual issue number.
Skip *Issue XYZ* if there is no associated github issue for this pull
request.
Skip *component* if you are unsure about which is the best component.
E.g. `[docs] Fix typo in produce method`.

- Fill out the template below to describe the changes contributed by the
pull request. That will give reviewers the context they need to do the review.

- Each pull request should address only one issue, not mix up code from
multiple issues.

- Each commit in the pull request has a meaningful commit message

- Once all items of the checklist are addressed, remove the above text and
this checklist, leaving only the filled out template below.

**(The sections below can be removed for hotfixes of typos)**
-->

*(If this PR fixes a github issue, please add `Fixes #`.)*

Fixes #

*(or if this PR is one task of a github issue, please add `Master Issue:
#` to link to the master issue.)*

Master Issue: #

### Motivation

*Explain here the context, and why you're making that change. What is the
problem you're trying to solve.*

### Modifications

*Describe the modifications you've done.*

### Verify this change

- [ ] Make sure that the change is correct.
- For how to check and verify, refer to [Translation and
localization](https://github.com/apache/pulsar/tree/master/site2#translation-and-localization).

### Documentation

- Is this pull request related to crowdin usage? (yes / no)
- Is this pull request related to translation quality? (yes / no)
- If yes, how to improve?
- Is this pull request related to translation guidelines?(yes / no)
- Is this pull request related to translation workflow?(yes / no)
- If a sth is not documented yet in this PR, please create a followup
issue for adding the documentation.

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 closed pull request #4949: Change rest api , post schema compatibility check always FULL

2019-08-14 Thread GitBox

congbobo184 closed pull request #4949: Change rest api ,post schema 
compatibility check always FULL
URL: https://github.com/apache/pulsar/pull/4949
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 opened a new pull request #4949: Change rest api , post schema compatibility check always FULL

2019-08-14 Thread GitBox

congbobo184 opened a new pull request #4949: Change rest api ,post schema 
compatibility check always FULL
URL: https://github.com/apache/pulsar/pull/4949
 
 
   ### Motivation
   To fix #4821 
   
   Does this pull request potentially affect one of the following parts:
   If yes was chosen, please highlight the changes
   
   Dependencies (does it add or upgrade a dependency): (no)
   The public API: (no)
   The schema: (yes)
   The default values of configurations: (no)
   The wire protocol: (no)
   The rest endpoints: (no)
   The admin cli options: (no)
   Anything that affects deployment: (no)
   
   ### Documentation
   Does this pull request introduce a new feature? (yes / no)
   If yes, how is the feature documented? (not applicable / docs / JavaDocs / 
not documented)
   If a feature is not applicable for documentation, explain why?
   If a feature is not documented yet in this PR, please create a followup 
issue for adding the documentation


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] LeisureChou commented on issue #4947: [function]Does the pulsar support connecting to the database in function?

2019-08-14 Thread GitBox

LeisureChou commented on issue #4947: [function]Does  the pulsar support 
connecting to the database in function?
URL: https://github.com/apache/pulsar/issues/4947#issuecomment-521129500
 
 
   or  can the state in function  save into db


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 opened a new pull request #4948: Rest api post schema use namespace level

2019-08-14 Thread GitBox

congbobo184 opened a new pull request #4948: Rest api post schema use namespace 
level
URL: https://github.com/apache/pulsar/pull/4948
 
 
   ### Motivation
   To fix #4821 
   
   Does this pull request potentially affect one of the following parts:
   If yes was chosen, please highlight the changes
   
   Dependencies (does it add or upgrade a dependency): (no)
   The public API: (no)
   The schema: (yes)
   The default values of configurations: (no)
   The wire protocol: (no)
   The rest endpoints: (no)
   The admin cli options: (no)
   Anything that affects deployment: (no)
   
   ### Documentation
   Does this pull request introduce a new feature? (yes / no)
   If yes, how is the feature documented? (not applicable / docs / JavaDocs / 
not documented)
   If a feature is not applicable for documentation, explain why?
   If a feature is not documented yet in this PR, please create a followup 
issue for adding the documentation


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 closed pull request #4948: Rest api post schema use namespace level

2019-08-14 Thread GitBox

congbobo184 closed pull request #4948: Rest api post schema use namespace level
URL: https://github.com/apache/pulsar/pull/4948
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch master updated: Do not log stack trace with warn log in websocket proxy (#4945)

2019-08-14 Thread nkurihar

This is an automated email from the ASF dual-hosted git repository.

nkurihar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 88e6b2b  Do not log stack trace with warn log in websocket proxy 
(#4945)
88e6b2b is described below

commit 88e6b2bc5707cd426bede75db43f96f405197375
Author: Masahiro Sakamoto 
AuthorDate: Wed Aug 14 16:30:18 2019 +0900

Do not log stack trace with warn log in websocket proxy (#4945)
---
 .../src/main/java/org/apache/pulsar/websocket/ProducerHandler.java  | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git 
a/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/ProducerHandler.java
 
b/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/ProducerHandler.java
index b5fdd8a..6ecf07e 100644
--- 
a/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/ProducerHandler.java
+++ 
b/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/ProducerHandler.java
@@ -260,7 +260,7 @@ public class ProducerHandler extends 
AbstractWebSocketHandler {
 getSession().getRemote().sendString(msg, new WriteCallback() {
 @Override
 public void writeFailed(Throwable th) {
-log.warn("[{}] Failed to send ack {}", 
producer.getTopic(), th.getMessage(), th);
+log.warn("[{}] Failed to send ack: {}", 
producer.getTopic(), th.getMessage());
 }
 
 @Override
@@ -272,9 +272,9 @@ public class ProducerHandler extends 
AbstractWebSocketHandler {
 }
 });
 } catch (JsonProcessingException e) {
-log.warn("[{}] Failed to generate ack json-response {}", 
producer.getTopic(), e.getMessage(), e);
+log.warn("[{}] Failed to generate ack json-response: {}", 
producer.getTopic(), e.getMessage());
 } catch (Exception e) {
-log.warn("[{}] Failed to send ack {}", producer.getTopic(), 
e.getMessage(), e);
+log.warn("[{}] Failed to send ack: {}", producer.getTopic(), 
e.getMessage());
 }
 }

[GitHub] [pulsar] nkurihar merged pull request #4945: [pulsar-websocket] Do not log stack trace with warn log in websocket proxy

2019-08-14 Thread GitBox

nkurihar merged pull request #4945: [pulsar-websocket] Do not log stack trace 
with warn log in websocket proxy
URL: https://github.com/apache/pulsar/pull/4945
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] kimcs commented on issue #4928: [java-client] ConsumerBuilder should have option for defaultSubscriptionInitialPosition

2019-08-14 Thread GitBox

kimcs commented on issue #4928: [java-client] ConsumerBuilder should have 
option for defaultSubscriptionInitialPosition
URL: https://github.com/apache/pulsar/issues/4928#issuecomment-521124404
 
 
   I have changed my design and now rely on the pulsar client application to 
track positions rather than using subscriptions and acknowledgements, so I no 
longer need this feature for what I'm working on. Still I think this option is 
useful, as it allows using Pulsar subscriptions to be master of stream progress 
with at-least-once semantics.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on issue #4838: NPE is thrown when a consumer consumes a partitioned topic with struct schema

2019-08-14 Thread GitBox

sijie commented on issue #4838: NPE is thrown when a consumer consumes a 
partitioned topic with struct schema
URL: https://github.com/apache/pulsar/issues/4838#issuecomment-521147568
 
 
   @congbobo184 I think I made a mistake in the description I provided. The 
exception was thrown when I use AUTO_CONSUME schema.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar-client-go] wolfstudy opened a new pull request #55: Check `golint`, `gofmt` and `go import` for project

2019-08-14 Thread GitBox

wolfstudy opened a new pull request #55: Check `golint`, `gofmt` and `go 
import` for project
URL: https://github.com/apache/pulsar-client-go/pull/55
 
 
   Signed-off-by: xiaolong.ran 
   
   ### Motivation
   
   Run `golangci-lint run` for formatting code.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] guillaume-braibant opened a new pull request #4951: Fix pull request 4700

2019-08-14 Thread GitBox

guillaume-braibant opened a new pull request #4951: Fix pull request 4700
URL: https://github.com/apache/pulsar/pull/4951
 
 
   ### Purpose
   
   Fixes pull request #4700 
   
   ### Motivation
   
   The updated API (using v1/apps for Workload API objects like DaemonSet, 
StatefulSet, and Deployment) introduced in #4700 requires a LabelSelector 
object for each Deployment, DaemonSet, and SatetfulSet object. The #4700 does 
not include those LabelSelector objects. This causes  the rejection of the K8S 
generic deployment scripts by the K8S API validator. 
   
   ### Changes
   
   _deployment/kubernetes/generic/k8s-1-9-and-above/bookie.yml :_
   
   - Add a LabelSelector object matching pods labels (required by API)
to the DaemonSet:spec object 
   
   _deployment/kubernetes/generic/k8s-1-9-and-above/broker.yml :_
   
   - Add a LabelSelector object matching pods labels (required by API)
to the Deployment:spec object .
   
   _deployment/kubernetes/generic/k8s-1-9-and-above/monitoring.yml :_
   
   - Add a LabelSelector object matching pods labels (required by API)
to the Deployment:spec object for Prometheus.
   
   - Add a LabelSelector object matching pods labels (required by API)
to the Deployment:spec object for Grafana.
   
   - Add a LabelSelector object matching pods labels (required by API)
to the Deployment:spec object for the Pulsar Dashboard.
   
   _deployment/kubernetes/generic/k8s-1-9-and-above/proxy.yml :_
   
   - Add a LabelSelector object matching pods labels (required by API)
to the Deployment:spec object.
   
   _deployment/kubernetes/generic/k8s-1-9-and-above/proxy.yml :_
   
   - Add a LabelSelector object matching pods labels (required by API)
to the StatefulSet:spec object.
   
   - Changing StatefulSet:spec:template:metadata:labels:cluster from 
'us-central' to 'local'. All other scripts assume cluster is 'local' and not 
'us-central'.
   
   ### Testing coverage
   
   - All scripts have been succefully applied to a fresh Minikube cluster 
without rejection
   - Cluster experiment (through admin pod) with a production rate of 100 has 
been done
   - Monitoring has NOT been tested (deployed but not functionally tested)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 commented on issue #4950: Modify rest api post schema compatibility strategy

2019-08-14 Thread GitBox

congbobo184 commented on issue #4950: Modify rest api post schema compatibility 
strategy
URL: https://github.com/apache/pulsar/pull/4950#issuecomment-521227167
 
 
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 removed a comment on issue #4950: Modify rest api post schema compatibility strategy

2019-08-14 Thread GitBox

congbobo184 removed a comment on issue #4950: Modify rest api post schema 
compatibility strategy
URL: https://github.com/apache/pulsar/pull/4950#issuecomment-521227167
 
 
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 commented on issue #4950: Modify rest api post schema compatibility strategy

2019-08-14 Thread GitBox

congbobo184 commented on issue #4950: Modify rest api post schema compatibility 
strategy
URL: https://github.com/apache/pulsar/pull/4950#issuecomment-521227153
 
 
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 commented on issue #4838: NPE is thrown when a consumer consumes a partitioned topic with struct schema

2019-08-14 Thread GitBox

congbobo184 commented on issue #4838: NPE is thrown when a consumer consumes a 
partitioned topic with struct schema
URL: https://github.com/apache/pulsar/issues/4838#issuecomment-521179091
 
 
   OK, I will reproduce this and fix the bug


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar-client-go] branch master updated: [Issue:53] Fix concurrent map write (#54)

2019-08-14 Thread rxl

This is an automated email from the ASF dual-hosted git repository.

rxl pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar-client-go.git


The following commit(s) were added to refs/heads/master by this push:
 new 8774688  [Issue:53] Fix concurrent map write (#54)
8774688 is described below

commit 87746885974efe881cedfef1db484f9f659d86b6
Author: 冉小龙 
AuthorDate: Wed Aug 14 19:11:07 2019 +0800

[Issue:53] Fix concurrent map write (#54)

* [Issue:53]Fix concurrent map write

Signed-off-by: xiaolong.ran 
---
 CONTRIBUTING.md   | 47 +-
 pulsar/consumer_test.go   | 82 ++-
 pulsar/impl_partition_consumer.go |  1 +
 pulsar/internal/connection.go |  2 +-
 4 files changed, 102 insertions(+), 30 deletions(-)

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 7a6e414..d05c687 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -21,23 +21,15 @@
 
 # How to contribute
 
-If you would like to contribute code to this project you can do so through 
GitHub by forking the repository and sending a pull request.
+If you would like to contribute code to this project, fork the repository and 
send a pull request.
 
-This document outlines some of the conventions on development workflow, commit 
message formatting, contact points and other resources to make it easier to get 
your contribution accepted.
+## Prerequisite
 
-## Steps to Contribute
+If you have not installed Go, install it according to the [installation 
instruction](http://golang.org/doc/install).
 
-Since the `go mod` package management tool is used in this project, your go 
version is required at **Go1.11+**.
+Since the `go mod` package management tool is used in this project, **Go 1.11 
or higher** version is required.
 
-### Fork
-
-Before you start contributing, you need to fork 
[pulsar-client-go](https://github.com/apache/pulsar) to your github repository.
-
-### Installation
-
-If you don't currently have a go environment installed，install Go according to 
the installation instructions here: http://golang.org/doc/install
-
-# mac os && linux
+### Install Go on Mac OS and Linux
 
 ```bash
 $ mkdir -p $HOME/github.com/apache/
@@ -47,19 +39,21 @@ $ cd pulsar-client-go
 $ go mod download
 ```
 
-When you execute `go mod download`, there may be some libs that cannot be 
downloaded. You can download them by referring to the proxy provided by 
[GOPROXY.io](https://goproxy.io/).
+If some libs cannot be downloaded when you enter the `go mod download` 
command, download them by referring to the proxy provided by 
[GOPROXY.io](https://goproxy.io/).
+
+## Fork
 
-### Contribution flow
+Before contributing, you need to fork 
[pulsar-client-go](https://github.com/apache/pulsar) to your github repository.
+
+## Contribution flow
 
 ```bash
 $ git remote add apache g...@github.com:apache/pulsar-client-go.git
-
 // sync with remote master
 $ git checkout master
 $ git fetch apache
 $ git rebase apache/master
 $ git push origin master
-
 // create PR branch
 $ git checkout -b your_branch   
 # do your work, and then
@@ -68,19 +62,16 @@ $ git commit -sm "xxx"
 $ git push origin your_branch
 ```
 
-Thanks for your contributions!
-
- Code style
-
-The coding style suggested by the Golang community is used in Apache 
pulsar-client-go. See the [style 
doc](https://github.com/golang/go/wiki/CodeReviewComments) for details.
+## Code style
 
-Please follow this style to make your pull request easy to review, maintain 
and develop.
+The coding style suggested by the Golang community is used in Apache 
pulsar-client-go. For details, refer to [style 
doc](https://github.com/golang/go/wiki/CodeReviewComments).
+Follow the style, make your pull request easy to review, maintain and develop.
 
- Create new file
+## Create new files
 
-The project uses the open source protocol of Apache License 2.0. When you need 
to create a new file when developing new features, 
-please add it at the beginning of the file. The location of the header file: 
[header file](.header).
+The project uses the open source protocol of Apache License 2.0. If you need 
to create a new file when developing new features, 
+add the license at the beginning of each file. The location of the header 
file: [header file](.header).
 
- Updating dependencies
+## Update dependencies
 
-Apache `pulsar-client-go` uses [Go 1.11 
module](https://github.com/golang/go/wiki/Modules) to manage dependencies. To 
add or update a dependency: use the `go mod edit` command to change the 
dependency.
+Apache `pulsar-client-go` uses [Go 1.11 
module](https://github.com/golang/go/wiki/Modules) to manage dependencies. To 
add or update a dependency, use the `go mod edit` command to change the 
dependency.
\ No newline at end of file
diff --git a/pulsar/consumer_test.go b/pulsar/consumer_test.go
index 6fe86cd..7cc32ed 100644
--- a/pulsar/consumer_test.go
+++

[GitHub] [pulsar] wolfstudy commented on issue #4939: [site2] Fix swagger of SourceBase and SinkBase display problem

2019-08-14 Thread GitBox

wolfstudy commented on issue #4939: [site2] Fix swagger of SourceBase and 
SinkBase display problem
URL: https://github.com/apache/pulsar/pull/4939#issuecomment-521206624
 
 
   run integration tests
   
   ```
   11:19:06.130 [kafka-producer-network-thread | 
b2a68b7b-3cf4-45ad-b69b-0682fe7127c2:org.apache.kafka.clients.NetworkClient@748]
 WARN  org.apache.kafka.clients.NetworkClient - [Producer 
clientId=b2a68b7b-3cf4-45ad-b69b-0682fe7127c2] Connection to node 1 
(localhost/127.0.0.1:35686) could not be established. Broker may not be 
available.
   11:19:06.240 
[ForkJoinPool.commonPool-worker-11:org.apache.pulsar.tests.integration.utils.DockerUtils@143]
 ERROR org.apache.pulsar.tests.integration.utils.DockerUtils - Error reading 
dir from container TestFileSystemOffload-wmear-pulsar-bookie-1
   java.io.IOException: gzip finished without exhausting source
at org.testcontainers.shaded.okio.GzipSource.read(GzipSource.java:100) 
~[testcontainers-1.11.2.jar:?]
at 
org.testcontainers.shaded.okio.RealBufferedSource$1.read(RealBufferedSource.java:439)
 ~[testcontainers-1.11.2.jar:?]
at 
org.apache.pulsar.tests.integration.utils.DockerUtils.dumpContainerDirToTargetCompressed(DockerUtils.java:136)
 ~[test-classes/:?]
at 
org.apache.pulsar.tests.integration.containers.PulsarContainer.beforeStop(PulsarContainer.java:101)
 ~[test-classes/:?]
at 
org.apache.pulsar.tests.integration.containers.ChaosContainer.stop(ChaosContainer.java:64)
 ~[test-classes/:?]
at 
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) 
[?:1.8.0_191]
at 
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) 
[?:1.8.0_191]
at 
java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) 
[?:1.8.0_191]
at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291) 
[?:1.8.0_191]
at 
java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731) 
[?:1.8.0_191]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) 
[?:1.8.0_191]
at 
java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) 
[?:1.8.0_191]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) 
[?:1.8.0_191]
at 
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) 
[?:1.8.0_191]
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar-client-go] wolfstudy commented on issue #30: Talk about `grabcnx` logic

2019-08-14 Thread GitBox

wolfstudy commented on issue #30: Talk about `grabcnx` logic
URL: https://github.com/apache/pulsar-client-go/issues/30#issuecomment-521500765
 
 
   ```
   11:27:31.826 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO  
org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:49977] 
[persistent://public/default/testSeek][sub-testSeek] Reset subscription to 
message id 0:4
   11:27:31.822 [pulsar-io-35-2] WARN  
org.apache.pulsar.broker.service.ServerCnx - 
[/127.0.0.1:49977][persistent://public/default/testSeek][sub-testSeek] Failed 
to create consumer: Subscription is fenced
   java.util.concurrent.CompletionException: 
org.apache.pulsar.broker.service.BrokerServiceException$SubscriptionFencedException:
 Subscription is fenced
at 
java.util.concurrent.CompletableFuture.encodeRelay(CompletableFuture.java:326) 
~[?:1.8.0_201]
at 
java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:984)
 ~[?:1.8.0_201]
at 
java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2124) 
~[?:1.8.0_201]
at 
org.apache.pulsar.broker.service.ServerCnx.lambda$null$13(ServerCnx.java:680) 
~[org.apache.pulsar-pulsar-broker-2.4.0.jar:2.4.0]
at 
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602) 
~[?:1.8.0_201]
at 
java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:614)
 ~[?:1.8.0_201]
at 
java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:1983) 
~[?:1.8.0_201]
at 
org.apache.pulsar.broker.service.ServerCnx.lambda$handleSubscribe$15(ServerCnx.java:633)
 ~[org.apache.pulsar-pulsar-broker-2.4.0.jar:2.4.0]
at 
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602) 
[?:1.8.0_201]
at 
java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:614)
 [?:1.8.0_201]
at 
java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:1983) 
[?:1.8.0_201]
at 
org.apache.pulsar.broker.service.ServerCnx.handleSubscribe(ServerCnx.java:622) 
[org.apache.pulsar-pulsar-broker-2.4.0.jar:2.4.0]
at 
org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:199)
 [org.apache.pulsar-pulsar-common-2.4.0.jar:2.4.0]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
 [io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
 [io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
 [io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]
at 
io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323)
 [io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]
at 
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297)
 [io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
 [io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] mingfang opened a new issue #4956: gen-yml-from-env.py is broken in 2.4.0

2019-08-14 Thread GitBox

mingfang opened a new issue #4956: gen-yml-from-env.py is broken in 2.4.0
URL: https://github.com/apache/pulsar/issues/4956
 
 
   ```sh
   docker run apachepulsar/pulsar-all:2.4.0 sh -c "\
   export configurationStoreServers=foo
   env|grep foo
   /pulsar/bin/gen-yml-from-env.py /pulsar/conf/functions_worker.yml
   grep configurationStoreServers /pulsar/conf/functions_worker.yml"
   ```
   configurationStoreServers=foo
   /pulsar/bin/gen-yml-from-env.py:49: YAMLLoadWarning: calling yaml.load() 
without Loader=... is deprecated, as the default Loader is unsafe. Please read 
https://msg.pyyaml.org/load for full details.
 conf = yaml.load(open(conf_filename))
   configurationStoreServers: localhost:2181


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] benxinglong opened a new issue #4954: 'msgBacklog' is wrong when use Producer.sendAsync() send message

2019-08-14 Thread GitBox

benxinglong opened a new issue #4954: 'msgBacklog' is wrong when use 
Producer.sendAsync() send message
URL: https://github.com/apache/pulsar/issues/4954
 
 
   **Describe the bug**
   When use Producer.sendAsync() method send message to some topic( already has 
subscription),the value of 'msgBacklog' is wrong.
   
   **To Reproduce**
   Steps to reproduce the behavior:
   1. use command "pulsar-client consume -s test -n 0 test" to create topic and 
subscription ,then exit
   2. send 10 messages to Topic "test"  by  Producer.sendAsync()  
   3. use command "pulsar-admin persistent stats test",check the value of 
'msgBacklog'
   4. the value of 'msgBacklog' is wrong
   
   **Expected behavior**
   The value of 'msgBacklog' should be 10
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie merged pull request #4938: [Doc] Add Notes for Create Topic

2019-08-14 Thread GitBox

sijie merged pull request #4938: [Doc] Add Notes for *Create Topic*
URL: https://github.com/apache/pulsar/pull/4938
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch master updated: [Doc] Add Notes for Create Topic

2019-08-14 Thread sijie

This is an automated email from the ASF dual-hosted git repository.

sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 900d81a  [Doc] Add Notes for *Create Topic*
900d81a is described below

commit 900d81ae701b3199a86b3a026eb120747da23987
Author: Anonymitaet <50226895+anonymit...@users.noreply.github.com>
AuthorDate: Thu Aug 15 09:33:25 2019 +0800

[Doc] Add Notes for *Create Topic*

Fix #4897
---
 site2/docs/admin-api-non-persistent-topics.md | 10 ++
 site2/docs/admin-api-partitioned-topics.md| 20 
 site2/docs/reference-pulsar-admin.md  | 20 
 3 files changed, 50 insertions(+)

diff --git a/site2/docs/admin-api-non-persistent-topics.md 
b/site2/docs/admin-api-non-persistent-topics.md
index 4427b21..2fd5240 100644
--- a/site2/docs/admin-api-non-persistent-topics.md
+++ b/site2/docs/admin-api-non-persistent-topics.md
@@ -178,6 +178,16 @@ admin.nonPersistentTopics().getInternalStats(topic);
 
 Partitioned topics in Pulsar must be explicitly created. When creating a new 
partitioned topic you need to provide a name for the topic as well as the 
desired number of partitions.
 
+>  Note
+>
+> By default, after 60 seconds of creation, topics are considered inactive and 
deleted automatically to prevent from generating trash data.
+>
+> To disable this feature, set `brokerDeleteInactiveTopicsEnabled` to `false`.
+>
+> To change the frequency of checking inactive topics, set 
`brokerDeleteInactiveTopicsFrequencySeconds` to your desired value.
+>
+> For more information about these two parameters, see 
[here](reference-configuration.md#broker).
+
  pulsar-admin
 
 ```shell
diff --git a/site2/docs/admin-api-partitioned-topics.md 
b/site2/docs/admin-api-partitioned-topics.md
index 0f68523..bab4eb4 100644
--- a/site2/docs/admin-api-partitioned-topics.md
+++ b/site2/docs/admin-api-partitioned-topics.md
@@ -20,6 +20,16 @@ persistent://tenant/namespace/topic
 Partitioned topics in Pulsar must be explicitly created. When creating a new 
partitioned topic you
 need to provide a name for the topic as well as the desired number of 
partitions.
 
+>  Note
+>
+> By default, after 60 seconds of creation, topics are considered inactive and 
deleted automatically to prevent from generating trash data.
+>
+> To disable this feature, set `brokerDeleteInactiveTopicsEnabled`  to `false`.
+>
+> To change the frequency of checking inactive topics, set 
`brokerDeleteInactiveTopicsFrequencySeconds` to your desired value.
+>
+> For more information about these two parameters, see 
[here](reference-configuration.md#broker).
+
  pulsar-admin
 
 You can create partitioned topics using the 
[`create-partitioned-topic`](reference-pulsar-admin.md#create-partitioned-topic)
@@ -51,6 +61,16 @@ admin.persistentTopics().createPartitionedTopic(topicName, 
numPartitions);
 Nonpartitioned topics in Pulsar must be explicitly created if 
allowAutoTopicCreation or createIfMissing is disabled.
 When creating a non-partitioned topic, you need to provide a topic name.
 
+>  Note
+>
+> By default, after 60 seconds of creation, topics are considered inactive and 
deleted automatically to prevent from generating trash data.
+>
+> To disable this feature, set `brokerDeleteInactiveTopicsEnabled` to `false`.
+>
+> To change the frequency of checking inactive topics, set 
`brokerDeleteInactiveTopicsFrequencySeconds` to your desired value.
+>
+> For more information about these two parameters, see 
[here](reference-configuration.md#broker).
+
  pulsar-admin
 
 You can create non-partitioned topics using the 
[`create`](reference-pulsar-admin.md#create)
diff --git a/site2/docs/reference-pulsar-admin.md 
b/site2/docs/reference-pulsar-admin.md
index e0cae4c..e88973f 100644
--- a/site2/docs/reference-pulsar-admin.md
+++ b/site2/docs/reference-pulsar-admin.md
@@ -2058,6 +2058,16 @@ Options
 ### `create-partitioned-topic`
 Create a partitioned topic. A partitioned topic must be created before 
producers can publish to it.
 
+>  Note
+>
+> By default, after 60 seconds of creation, topics are considered inactive and 
deleted automatically to prevent from generating trash data.
+>
+> To disable this feature, set `brokerDeleteInactiveTopicsEnabled` to `false`.
+>
+> To change the frequency of checking inactive topics, set 
`brokerDeleteInactiveTopicsFrequencySeconds` to your desired value.
+>
+> For more information about these two parameters, see 
[here](reference-configuration.md#broker).
+
 Usage
 ```bash
 $ pulsar-admin topics create-partitioned-topic 
{persistent|non-persistent}://tenant/namespace/topic options
@@ -2079,6 +2089,16 @@ $ pulsar-admin topics delete-partitioned-topic 
{persistent|non-persistent}
 ### `create`
 Creates a non-partitioned topic. A non-partitioned topic must explicitly be 
created by the user if allowAutoTopicCreation

[GitHub] [pulsar] codelipenghui opened a new pull request #4955: [WIP] [PIP-39] Introduce system topic

2019-08-14 Thread GitBox

codelipenghui opened a new pull request #4955: [WIP] [PIP-39] Introduce system 
topic
URL: https://github.com/apache/pulsar/pull/4955
 
 
   ### Motivation
   
   Introduce system topic and topic policies cache for 
[PIP-39](https://github.com/apache/pulsar/wiki/PIP-39%3A-Namespace-Change-Events)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on issue #4938: [Doc] Add Notes for Create Topic

2019-08-14 Thread GitBox

sijie commented on issue #4938: [Doc] Add Notes for *Create Topic*
URL: https://github.com/apache/pulsar/pull/4938#issuecomment-521468945
 
 
   run cpp tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar-translation] Jennifer88huang commented on issue #3: Create LICENSE

2019-08-14 Thread GitBox

Jennifer88huang commented on issue #3: Create LICENSE
URL: https://github.com/apache/pulsar-translation/pull/3#issuecomment-521478588
 
 
   @sijie Sure.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet commented on issue #4938: [Doc] Add Notes for Create Topic

2019-08-14 Thread GitBox

Anonymitaet commented on issue #4938: [Doc] Add Notes for *Create Topic*
URL: https://github.com/apache/pulsar/pull/4938#issuecomment-521478860
 
 
   @sijie could you please help merge this PR? Thank you


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet closed issue #4897: [Doc] Add notes for creating topic

2019-08-14 Thread GitBox

Anonymitaet closed issue #4897: [Doc] Add notes for creating topic 
URL: https://github.com/apache/pulsar/issues/4897
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet commented on issue #4897: [Doc] Add notes for creating topic

2019-08-14 Thread GitBox

Anonymitaet commented on issue #4897: [Doc] Add notes for creating topic 
URL: https://github.com/apache/pulsar/issues/4897#issuecomment-521479086
 
 
   This issue has been fixed, thanks for your suggestion @cumtwwei and thanks 
for your review @sijie.  


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] mingfang edited a comment on issue #572: Don't depend on ZooKeeper for coordination

2019-08-14 Thread GitBox

mingfang edited a comment on issue #572: Don't depend on ZooKeeper for 
coordination
URL: https://github.com/apache/pulsar/issues/572#issuecomment-517973547
 
 
   I'm now running without zookeeper by using zetcd in front of etcd


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on a change in pull request #4935: [Transaction][Buffer] handle command `EndTxnOnPartitiion`

2019-08-14 Thread GitBox

sijie commented on a change in pull request #4935: [Transaction][Buffer] handle 
command `EndTxnOnPartitiion`
URL: https://github.com/apache/pulsar/pull/4935#discussion_r313775652
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
 ##
 @@ -1358,6 +1360,57 @@ protected void handleGetSchema(CommandGetSchema 
commandGetSchema) {
 });
 }
 
+@Override
+protected void handleEndTxnOnPartition(PulsarApi.CommandEndTxnOnPartition 
commandEndTxnOnPartition) {
+final long requestId = commandEndTxnOnPartition.getRequestId();
+final String topicName = commandEndTxnOnPartition.getTopic();
+final long mostBits = commandEndTxnOnPartition.getTxnidMostBits();
+final long leastBits = commandEndTxnOnPartition.getTxnidLeastBits();
+
+TxnID txnID = new TxnID(mostBits, leastBits);
+log.info("Received CommandEndTxnOnPartition from {}, [{}] transaction 
{} on the topic {}", remoteAddress,
+ commandEndTxnOnPartition.getTxnAction(), txnID, topicName);
+service.getTopic(topicName, false).whenComplete((topic, throwable) -> {
+if (throwable != null) {
+ctx.writeAndFlush(
+Commands.newEndTxnOnPartitionResponse(requestId, 
ServerError.UnknownError, throwable.getMessage()));
+} else {
+if (topic.isPresent()) {
+final Topic partition = topic.get();
+// sequenceId ?
+TxnMarkerController markerController = new 
TxnMarkerController(topic.get(), txnID, -1);
+if 
(commandEndTxnOnPartition.getTxnAction().equals(PulsarApi.TxnAction.COMMIT)) {
+markerController.publishCommitMarker()
+.thenApply(position -> 
(PositionImpl)position)
+
.thenCombine(partition.getTxnBuffer(false), (position, transactionBuffer) ->
+transactionBuffer.commitTxn(txnID, 
position.getLedgerId(), position.getEntryId()))
+.thenAccept(ignore ->
+ctx.writeAndFlush(
+
Commands.newEndTxnOnPartitionResponse(requestId, txnID.getLeastSigBits(), 
txnID.getMostSigBits(
+.exceptionally(err -> {
+log.error("Commit txn error :", 
err);
+ctx.writeAndFlush(
+
Commands.newEndTxnOnPartitionResponse(requestId, ServerError.UnknownError, 
err.getMessage()));
+return null;
+});
+} else if 
(commandEndTxnOnPartition.getTxnAction().equals(PulsarApi.TxnAction.ABORT)) {
+markerController.publishAbortMarker()
+.thenCompose(ignore -> 
partition.getTxnBuffer(false))
+.thenCompose(buffer -> 
buffer.abortTxn(txnID))
+.exceptionally(err -> {
+log.error("Abort txn error : ", 
err);
 
 Review comment:
   same comment as above


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on a change in pull request #4935: [Transaction][Buffer] handle command `EndTxnOnPartitiion`

2019-08-14 Thread GitBox

sijie commented on a change in pull request #4935: [Transaction][Buffer] handle 
command `EndTxnOnPartitiion`
URL: https://github.com/apache/pulsar/pull/4935#discussion_r313768449
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
 ##
 @@ -1358,6 +1360,57 @@ protected void handleGetSchema(CommandGetSchema 
commandGetSchema) {
 });
 }
 
+@Override
+protected void handleEndTxnOnPartition(PulsarApi.CommandEndTxnOnPartition 
commandEndTxnOnPartition) {
+final long requestId = commandEndTxnOnPartition.getRequestId();
+final String topicName = commandEndTxnOnPartition.getTopic();
+final long mostBits = commandEndTxnOnPartition.getTxnidMostBits();
+final long leastBits = commandEndTxnOnPartition.getTxnidLeastBits();
+
+TxnID txnID = new TxnID(mostBits, leastBits);
+log.info("Received CommandEndTxnOnPartition from {}, [{}] transaction 
{} on the topic {}", remoteAddress,
 
 Review comment:
   this should be in debug level. otherwise the logging will be too noisy.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on a change in pull request #4935: [Transaction][Buffer] handle command `EndTxnOnPartitiion`

2019-08-14 Thread GitBox

sijie commented on a change in pull request #4935: [Transaction][Buffer] handle 
command `EndTxnOnPartitiion`
URL: https://github.com/apache/pulsar/pull/4935#discussion_r313769923
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
 ##
 @@ -1533,4 +1586,49 @@ public boolean isBatchMessageCompatibleVersion() {
 public String getClientVersion() {
 return clientVersion;
 }
+
+/**
+ * The transaction marker controller controls the commit marker or the 
abort marker publish to the specified topic.
+ * If it is a commit marker, we need to wait for the position it committed 
at the topic.
+ */
+final static class TxnMarkerController implements Topic.PublishContext {
 
 Review comment:
   don't you need to implement `getProducerName` to make sure exactly-once when 
publish markers?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on a change in pull request #4935: [Transaction][Buffer] handle command `EndTxnOnPartitiion`

2019-08-14 Thread GitBox

sijie commented on a change in pull request #4935: [Transaction][Buffer] handle 
command `EndTxnOnPartitiion`
URL: https://github.com/apache/pulsar/pull/4935#discussion_r313770650
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/nonpersistent/NonPersistentTopic.java
 ##
 @@ -345,6 +351,11 @@ public void removeProducer(Producer producer) {
 return delete(false, true, false);
 }
 
+@Override
+protected TransactionBufferProvider getProvider() throws IOException {
+return 
TransactionBufferProvider.newProvider(transactionBufferProvider);
 
 Review comment:
   you already know it is InMemTransactionBuffer. why can't you just construct 
an in-memory transaction buffer?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on a change in pull request #4935: [Transaction][Buffer] handle command `EndTxnOnPartitiion`

2019-08-14 Thread GitBox

sijie commented on a change in pull request #4935: [Transaction][Buffer] handle 
command `EndTxnOnPartitiion`
URL: https://github.com/apache/pulsar/pull/4935#discussion_r313770797
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java
 ##
 @@ -700,6 +708,10 @@ void removeSubscription(String subscriptionName) {
 return delete(false, true, false);
 }
 
+protected TransactionBufferProvider getProvider() throws IOException {
+return 
TransactionBufferProvider.newProvider(transactionBufferProvider);
 
 Review comment:
   same comment as above.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on a change in pull request #4935: [Transaction][Buffer] handle command `EndTxnOnPartitiion`

2019-08-14 Thread GitBox

sijie commented on a change in pull request #4935: [Transaction][Buffer] handle 
command `EndTxnOnPartitiion`
URL: https://github.com/apache/pulsar/pull/4935#discussion_r313770947
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java
 ##
 @@ -151,6 +156,9 @@
 private CompletableFuture currentCompaction = 
CompletableFuture.completedFuture(COMPACTION_NEVER_RUN);
 private final CompactedTopic compactedTopic;
 
+// TODO: set the provider by the config
 
 Review comment:
   Why not add the configuration setting here? Why another TODO item?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on a change in pull request #4935: [Transaction][Buffer] handle command `EndTxnOnPartitiion`

2019-08-14 Thread GitBox

sijie commented on a change in pull request #4935: [Transaction][Buffer] handle 
command `EndTxnOnPartitiion`
URL: https://github.com/apache/pulsar/pull/4935#discussion_r313771158
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
 ##
 @@ -1358,6 +1360,57 @@ protected void handleGetSchema(CommandGetSchema 
commandGetSchema) {
 });
 }
 
+@Override
+protected void handleEndTxnOnPartition(PulsarApi.CommandEndTxnOnPartition 
commandEndTxnOnPartition) {
+final long requestId = commandEndTxnOnPartition.getRequestId();
+final String topicName = commandEndTxnOnPartition.getTopic();
+final long mostBits = commandEndTxnOnPartition.getTxnidMostBits();
+final long leastBits = commandEndTxnOnPartition.getTxnidLeastBits();
+
+TxnID txnID = new TxnID(mostBits, leastBits);
+log.info("Received CommandEndTxnOnPartition from {}, [{}] transaction 
{} on the topic {}", remoteAddress,
+ commandEndTxnOnPartition.getTxnAction(), txnID, topicName);
+service.getTopic(topicName, false).whenComplete((topic, throwable) -> {
+if (throwable != null) {
+ctx.writeAndFlush(
+Commands.newEndTxnOnPartitionResponse(requestId, 
ServerError.UnknownError, throwable.getMessage()));
+} else {
+if (topic.isPresent()) {
+final Topic partition = topic.get();
+// sequenceId ?
+TxnMarkerController markerController = new 
TxnMarkerController(topic.get(), txnID, -1);
+if 
(commandEndTxnOnPartition.getTxnAction().equals(PulsarApi.TxnAction.COMMIT)) {
+markerController.publishCommitMarker()
+.thenApply(position -> 
(PositionImpl)position)
+
.thenCombine(partition.getTxnBuffer(false), (position, transactionBuffer) ->
+transactionBuffer.commitTxn(txnID, 
position.getLedgerId(), position.getEntryId()))
+.thenAccept(ignore ->
+ctx.writeAndFlush(
+
Commands.newEndTxnOnPartitionResponse(requestId, txnID.getLeastSigBits(), 
txnID.getMostSigBits(
+.exceptionally(err -> {
+log.error("Commit txn error :", 
err);
+ctx.writeAndFlush(
+
Commands.newEndTxnOnPartitionResponse(requestId, ServerError.UnknownError, 
err.getMessage()));
+return null;
+});
+} else if 
(commandEndTxnOnPartition.getTxnAction().equals(PulsarApi.TxnAction.ABORT)) {
+markerController.publishAbortMarker()
 
 Review comment:
   Why do you need to publish an abort marker?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on a change in pull request #4935: [Transaction][Buffer] handle command `EndTxnOnPartitiion`

2019-08-14 Thread GitBox

sijie commented on a change in pull request #4935: [Transaction][Buffer] handle 
command `EndTxnOnPartitiion`
URL: https://github.com/apache/pulsar/pull/4935#discussion_r313772027
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
 ##
 @@ -1358,6 +1360,57 @@ protected void handleGetSchema(CommandGetSchema 
commandGetSchema) {
 });
 }
 
+@Override
+protected void handleEndTxnOnPartition(PulsarApi.CommandEndTxnOnPartition 
commandEndTxnOnPartition) {
+final long requestId = commandEndTxnOnPartition.getRequestId();
+final String topicName = commandEndTxnOnPartition.getTopic();
+final long mostBits = commandEndTxnOnPartition.getTxnidMostBits();
+final long leastBits = commandEndTxnOnPartition.getTxnidLeastBits();
+
+TxnID txnID = new TxnID(mostBits, leastBits);
+log.info("Received CommandEndTxnOnPartition from {}, [{}] transaction 
{} on the topic {}", remoteAddress,
+ commandEndTxnOnPartition.getTxnAction(), txnID, topicName);
+service.getTopic(topicName, false).whenComplete((topic, throwable) -> {
+if (throwable != null) {
+ctx.writeAndFlush(
+Commands.newEndTxnOnPartitionResponse(requestId, 
ServerError.UnknownError, throwable.getMessage()));
+} else {
+if (topic.isPresent()) {
+final Topic partition = topic.get();
+// sequenceId ?
+TxnMarkerController markerController = new 
TxnMarkerController(topic.get(), txnID, -1);
+if 
(commandEndTxnOnPartition.getTxnAction().equals(PulsarApi.TxnAction.COMMIT)) {
+markerController.publishCommitMarker()
+.thenApply(position -> 
(PositionImpl)position)
+
.thenCombine(partition.getTxnBuffer(false), (position, transactionBuffer) ->
+transactionBuffer.commitTxn(txnID, 
position.getLedgerId(), position.getEntryId()))
+.thenAccept(ignore ->
+ctx.writeAndFlush(
+
Commands.newEndTxnOnPartitionResponse(requestId, txnID.getLeastSigBits(), 
txnID.getMostSigBits(
+.exceptionally(err -> {
+log.error("Commit txn error :", 
err);
 
 Review comment:
   add more meaningful information to the logging statement.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar.wiki] branch master updated: Updated Home (markdown)

2019-08-14 Thread sijie

This is an automated email from the ASF dual-hosted git repository.

sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git


The following commit(s) were added to refs/heads/master by this push:
 new 886169f  Updated Home (markdown)
886169f is described below

commit 886169ff069401c5c2758b49578f176b11ca7bd7
Author: Sijie Guo 
AuthorDate: Wed Aug 14 17:39:28 2019 +0800

Updated Home (markdown)
---
 Home.md | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/Home.md b/Home.md
index 1bfc8b3..224cc6d 100644
--- a/Home.md
+++ b/Home.md
@@ -10,20 +10,20 @@ We encourage to document any big change or feature or any 
addition to public use
 * [[PIP 39: Namespace Change Events]]
 * [[PIP 38: Batch Receiving Messages]]
 * [[PIP 37: Large message size handling in Pulsar]]
-* [[PIP 36: Max Message Size]]
 * [[PIP 35: Improve topic lookup for topics that have high number of 
partitions]]
 * [[PIP 33: Replicated subscriptions]]
-* [[PIP 32: Go Function API, Instance and LocalRun]]
 * [[PIP 31: Transaction Support]]
-* [[PIP 26: Delayed Message Delivery]]
 * [[PIP 21: Pulsar Edge Component]]
 
 ### Accepted
+* [[PIP 36: Max Message Size]]
 * [[PIP 34: Add new subscribe type Key_shared]]
+* [[PIP 32: Go Function API, Instance and LocalRun]]
 * [[PIP 30: change authentication provider API to support mutual 
authentication]]
 * [[PIP 29: One package for both pulsar-client and pulsar-admin]]
-* [[PIP 27: Add checklist in github pull request template]]
 * [[PIP 28: Pulsar Proxy Gateway Improvement]]
+* [[PIP 27: Add checklist in github pull request template]]
+* [[PIP 26: Delayed Message Delivery]]
 * [[PIP 25: Token based authentication]]
 * [[PIP 24: Simplify memory settings]]
 * [[PIP 23: Message Tracing By Interceptors]]

[GitHub] [pulsar] congbobo184 commented on issue #4950: Modify rest api post schema compatibility strategy

2019-08-14 Thread GitBox

congbobo184 commented on issue #4950: Modify rest api post schema compatibility 
strategy
URL: https://github.com/apache/pulsar/pull/4950#issuecomment-521191956
 
 
   run cpp tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch asf-site updated: Updated site at revision 88e6b2b

2019-08-14 Thread mmerli

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new eb6d3b0  Updated site at revision 88e6b2b
eb6d3b0 is described below

commit eb6d3b002caa675bc5a758e45bfd6f2ac15396a9
Author: jenkins 
AuthorDate: Wed Aug 14 08:47:05 2019 +

Updated site at revision 88e6b2b
---
 content/swagger/2.5.0-SNAPSHOT/swagger.json| 26 ++--
 .../swagger/2.5.0-SNAPSHOT/swaggerfunctions.json   | 46 +++---
 content/swagger/swagger.json   | 26 ++--
 content/swagger/swaggerfunctions.json  | 46 +++---
 4 files changed, 72 insertions(+), 72 deletions(-)

diff --git a/content/swagger/2.5.0-SNAPSHOT/swagger.json 
b/content/swagger/2.5.0-SNAPSHOT/swagger.json
index 5bc7863..f62fedb 100644
--- a/content/swagger/2.5.0-SNAPSHOT/swagger.json
+++ b/content/swagger/2.5.0-SNAPSHOT/swagger.json
@@ -8869,13 +8869,6 @@
 "directMemory" : {
   "$ref" : "#/definitions/ResourceUsage"
 },
-"msgThroughputOut" : {
-  "type" : "number",
-  "format" : "double"
-},
-"bandwidthOut" : {
-  "$ref" : "#/definitions/ResourceUsage"
-},
 "lastUpdate" : {
   "type" : "integer",
   "format" : "int64"
@@ -8884,23 +8877,30 @@
   "type" : "number",
   "format" : "double"
 },
-"bandwidthIn" : {
+"msgThroughputOut" : {
+  "type" : "number",
+  "format" : "double"
+},
+"cpu" : {
   "$ref" : "#/definitions/ResourceUsage"
 },
-"loadReportType" : {
-  "type" : "string"
+"underLoaded" : {
+  "type" : "boolean"
 },
 "overLoaded" : {
   "type" : "boolean"
 },
-"underLoaded" : {
-  "type" : "boolean"
+"loadReportType" : {
+  "type" : "string"
 },
-"cpu" : {
+"bandwidthIn" : {
   "$ref" : "#/definitions/ResourceUsage"
 },
 "memory" : {
   "$ref" : "#/definitions/ResourceUsage"
+},
+"bandwidthOut" : {
+  "$ref" : "#/definitions/ResourceUsage"
 }
   }
 },
diff --git a/content/swagger/2.5.0-SNAPSHOT/swaggerfunctions.json 
b/content/swagger/2.5.0-SNAPSHOT/swaggerfunctions.json
index 76da801..839f91f 100644
--- a/content/swagger/2.5.0-SNAPSHOT/swaggerfunctions.json
+++ b/content/swagger/2.5.0-SNAPSHOT/swaggerfunctions.json
@@ -1348,27 +1348,6 @@
 "Message" : {
   "type" : "object",
   "properties" : {
-"messageId" : {
-  "$ref" : "#/definitions/MessageId"
-},
-"sequenceId" : {
-  "type" : "integer",
-  "format" : "int64"
-},
-"redeliveryCount" : {
-  "type" : "integer",
-  "format" : "int32"
-},
-"schemaVersion" : {
-  "type" : "array",
-  "items" : {
-"type" : "string",
-"format" : "byte"
-  }
-},
-"replicated" : {
-  "type" : "boolean"
-},
 "publishTime" : {
   "type" : "integer",
   "format" : "int64"
@@ -1380,12 +1359,22 @@
 "topicName" : {
   "type" : "string"
 },
-"replicatedFrom" : {
-  "type" : "string"
+"replicated" : {
+  "type" : "boolean"
 },
 "producerName" : {
   "type" : "string"
 },
+"replicatedFrom" : {
+  "type" : "string"
+},
+"messageId" : {
+  "$ref" : "#/definitions/MessageId"
+},
+"sequenceId" : {
+  "type" : "integer",
+  "format" : "int64"
+},
 "keyBytes" : {
   "type" : "array",
   "items" : {
@@ -1403,6 +1392,17 @@
 "encryptionCtx" : {
   "$ref" : "#/definitions/EncryptionContext"
 },
+"redeliveryCount" : {
+  "type" : "integer",
+  "format" : "int32"
+},
+"schemaVersion" : {
+  "type" : "array",
+  "items" : {
+"type" : "string",
+"format" : "byte"
+  }
+},
 "data" : {
   "type" : "array",
   "items" : {
diff --git a/content/swagger/swagger.json b/content/swagger/swagger.json
index 5bc7863..f62fedb 100644
--- a/content/swagger/swagger.json
+++ b/content/swagger/swagger.json
@@ -8869,13 +8869,6 @@
 "directMemory" : {
   "$ref" : "#/definitions/ResourceUsage"
 },
-"msgThroughputOut" : {
-  "type" : "number",
-  "format" : "double"
-},
-"bandwidthOut" : {
-  "$ref" : "#/definitions/ResourceUsage"
-},
 "lastUpdate" : {
   "type" : "integer",
   "format"

[GitHub] [pulsar] sijie commented on issue #4934: [Doc] Update Schema Guide

2019-08-14 Thread GitBox

sijie commented on issue #4934: [Doc] Update Schema Guide
URL: https://github.com/apache/pulsar/pull/4934#issuecomment-521170948
 
 
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on issue #4939: [site2] Fix swagger of SourceBase and SinkBase display problem

2019-08-14 Thread GitBox

sijie commented on issue #4939: [site2] Fix swagger of SourceBase and SinkBase 
display problem
URL: https://github.com/apache/pulsar/pull/4939#issuecomment-521170663
 
 
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] zymap commented on a change in pull request #4935: [Transaction][Buffer] handle command `EndTxnOnPartitiion`

2019-08-14 Thread GitBox

zymap commented on a change in pull request #4935: [Transaction][Buffer] handle 
command `EndTxnOnPartitiion`
URL: https://github.com/apache/pulsar/pull/4935#discussion_r313788392
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
 ##
 @@ -1533,4 +1586,49 @@ public boolean isBatchMessageCompatibleVersion() {
 public String getClientVersion() {
 return clientVersion;
 }
+
+/**
+ * The transaction marker controller controls the commit marker or the 
abort marker publish to the specified topic.
+ * If it is a commit marker, we need to wait for the position it committed 
at the topic.
+ */
+final static class TxnMarkerController implements Topic.PublishContext {
 
 Review comment:
   I need. I will fix it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] zymap commented on a change in pull request #4935: [Transaction][Buffer] handle command `EndTxnOnPartitiion`

2019-08-14 Thread GitBox

zymap commented on a change in pull request #4935: [Transaction][Buffer] handle 
command `EndTxnOnPartitiion`
URL: https://github.com/apache/pulsar/pull/4935#discussion_r313788412
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
 ##
 @@ -1358,6 +1360,57 @@ protected void handleGetSchema(CommandGetSchema 
commandGetSchema) {
 });
 }
 
+@Override
+protected void handleEndTxnOnPartition(PulsarApi.CommandEndTxnOnPartition 
commandEndTxnOnPartition) {
+final long requestId = commandEndTxnOnPartition.getRequestId();
+final String topicName = commandEndTxnOnPartition.getTopic();
+final long mostBits = commandEndTxnOnPartition.getTxnidMostBits();
+final long leastBits = commandEndTxnOnPartition.getTxnidLeastBits();
+
+TxnID txnID = new TxnID(mostBits, leastBits);
+log.info("Received CommandEndTxnOnPartition from {}, [{}] transaction 
{} on the topic {}", remoteAddress,
+ commandEndTxnOnPartition.getTxnAction(), txnID, topicName);
+service.getTopic(topicName, false).whenComplete((topic, throwable) -> {
+if (throwable != null) {
+ctx.writeAndFlush(
+Commands.newEndTxnOnPartitionResponse(requestId, 
ServerError.UnknownError, throwable.getMessage()));
+} else {
+if (topic.isPresent()) {
+final Topic partition = topic.get();
+// sequenceId ?
+TxnMarkerController markerController = new 
TxnMarkerController(topic.get(), txnID, -1);
+if 
(commandEndTxnOnPartition.getTxnAction().equals(PulsarApi.TxnAction.COMMIT)) {
+markerController.publishCommitMarker()
+.thenApply(position -> 
(PositionImpl)position)
+
.thenCombine(partition.getTxnBuffer(false), (position, transactionBuffer) ->
+transactionBuffer.commitTxn(txnID, 
position.getLedgerId(), position.getEntryId()))
+.thenAccept(ignore ->
+ctx.writeAndFlush(
+
Commands.newEndTxnOnPartitionResponse(requestId, txnID.getLeastSigBits(), 
txnID.getMostSigBits(
+.exceptionally(err -> {
+log.error("Commit txn error :", 
err);
+ctx.writeAndFlush(
+
Commands.newEndTxnOnPartitionResponse(requestId, ServerError.UnknownError, 
err.getMessage()));
+return null;
+});
+} else if 
(commandEndTxnOnPartition.getTxnAction().equals(PulsarApi.TxnAction.ABORT)) {
+markerController.publishAbortMarker()
 
 Review comment:
   Sorry. It's my mistake.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 opened a new pull request #4950: Modify rest api post schema compatibility strategy

2019-08-14 Thread GitBox

congbobo184 opened a new pull request #4950: Modify rest api post schema 
compatibility strategy
URL: https://github.com/apache/pulsar/pull/4950
 
 
   ### Motivation
   To fix #4821 
   
   Does this pull request potentially affect one of the following parts:
   If yes was chosen, please highlight the changes
   
   Dependencies (does it add or upgrade a dependency): (no)
   The public API: (no)
   The schema: (yes)
   The default values of configurations: (no)
   The wire protocol: (no)
   The rest endpoints: (no)
   The admin cli options: (no)
   Anything that affects deployment: (no)
   
   ### Documentation
   Does this pull request introduce a new feature? (yes / no)
   If yes, how is the feature documented? (not applicable / docs / JavaDocs / 
not documented)
   If a feature is not applicable for documentation, explain why?
   If a feature is not documented yet in this PR, please create a followup 
issue for adding the documentation


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #4910: [doc] Add a documentation page for metrics reference

2019-08-14 Thread GitBox

Jennifer88huang commented on a change in pull request #4910: [doc] Add a 
documentation page for metrics reference
URL: https://github.com/apache/pulsar/pull/4910#discussion_r313792058
 
 

 ##
 File path: site2/docs/reference-metrics.md
 ##
 @@ -0,0 +1,243 @@
+---
+id: reference-metrics
+title: Pulsar Metrics
+sidebar_label: Pulsar Metrics
+---
+
+
+  table{
+font-size: 80%;
+  }
+
+
+Pulsar exposes metrics in Prometheus format that can be collected and used for 
monitoring the health of the cluster.
+
+* [ZooKeeper](#zookeeper)
+* [BookKeeper](#bookkeeper)
+* [Broker](#broker)
+
+## Overview
+
+The metrics exposed by Pulsar are in Prometheus format. The types of metrics 
are:
+
+- [Counter](https://prometheus.io/docs/concepts/metric_types/#counter): a 
cumulative metric that represents a single monotonically increasing counter 
whose value can only increase or be reset to zero on restart.
+- [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): a *gauge* 
is a metric that represents a single numerical value that can arbitrarily go up 
and down.
+- [Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram): a 
histogram samples observations (usually things like request durations or 
response sizes) and counts them in configurable buckets.
+- [Summary](https://prometheus.io/docs/concepts/metric_types/#summary): 
similar to a histogram, a summary samples observations (usually things like 
request durations and response sizes). While it also provides a total count of 
observations and a sum of all observed values, it calculates configurable 
quantiles over a sliding time window.
+
+## ZooKeeper
+
+The zookeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by configuring a system
+property `stats_server_port` to use a different port.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_znode_count | Gauge | Number of z-nodes stored. |
+| zookeeper_server_data_size_bytes | Gauge | The total size of all of z-nodes 
stored. |
+| zookeeper_server_connections | Gauge | Number of currently opened 
connections. |
+| zookeeper_server_watches_count | Gauge | Number of watchers registered. |
+| zookeeper_server_ephemerals_count | Gauge | Number of ephemeral z-nodes. |
+
+### Request metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_requests | Counter | The total number of requests received 
by a particular server. |
+| zookeeper_server_requests_latency_ms | Summary | The requests latency 
calculated in milliseconds.  Available labels: *type* (write, read).  
*write*: the requests that write data to zookeeper.*read*: the 
requests that read data from zookeeper.|
+
+## BookKeeper
+
+The bookkeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by updating `prometheusStatsHttpPort`
+in `bookkeeper.conf` configuration file.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_SERVER_STATUS | Gauge | The server status for bookie server. 
1: the bookie is running in writable mode.0: the bookie is 
running in readonly mode. |
+| bookkeeper_server_ADD_ENTRY_count | Counter | The total number of ADD_ENTRY 
requests received at the bookie. Label `success` used to distinguish successes 
and failures. |
+| bookkeeper_server_READ_ENTRY_count | Counter | The total number of 
READ_ENTRY requests received at the bookie. Label `success` used to distinguish 
successes and failures. |
+| bookie_WRITE_BYTES | Counter | The total number of bytes written to the 
bookie. |
+| bookie_READ_BYTES | Counter | The total number of bytes read from the 
bookie. |
+| bookkeeper_server_ADD_ENTRY_REQUEST | Histogram | The histogram of request 
latency of ADD_ENTRY requests at the bookie. Label `success` used to 
distinguish successes and failures. | 
+| bookkeeper_server_READ_ENTRY_REQUEST | Histogram | The histogram of request 
latency of READ_ENTRY requests at the bookie. Label `success` used to 
distinguish successes and failures. | 
+
+### Journal metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_journal_JOURNAL_SYNC_count | Counter | The total number of journal 
fsync operations happening at the bookie. Label `success` used to distinguish 
successes and failures. |
+| bookie_journal_JOURNAL_QUEUE_SIZE | Gauge | The total number of requests 
pending in the journal queue. |
+| bookie_journal_JOURNAL_FORCE_WRITE_QUEUE_SIZE | Gauge | The total number of 
force write (fsync) requests pending in the force-write queue. |
+| bookie_journal_JOURNAL_CB_QUEUE_SIZE | Gauge | The total number of callbacks 
pending in the callback queue. |
+| bookie_journal_JOURNAL_ADD_ENTRY | Histogram | The histogram of request 
latency of adding entries to the journal. |
+| bookie_journal_JOURNAL_SYNC | Histogram | The histogram of fsync latency of 
syncing data to the journal disk. |
+
+### Storage metrics
+
+| Name | Type | Description |
+|---|---|---|
+|

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #4910: [doc] Add a documentation page for metrics reference

2019-08-14 Thread GitBox

Jennifer88huang commented on a change in pull request #4910: [doc] Add a 
documentation page for metrics reference
URL: https://github.com/apache/pulsar/pull/4910#discussion_r313791211
 
 

 ##
 File path: site2/docs/reference-metrics.md
 ##
 @@ -0,0 +1,243 @@
+---
+id: reference-metrics
+title: Pulsar Metrics
+sidebar_label: Pulsar Metrics
+---
+
+
+  table{
+font-size: 80%;
+  }
+
+
+Pulsar exposes metrics in Prometheus format that can be collected and used for 
monitoring the health of the cluster.
+
+* [ZooKeeper](#zookeeper)
+* [BookKeeper](#bookkeeper)
+* [Broker](#broker)
+
+## Overview
+
+The metrics exposed by Pulsar are in Prometheus format. The types of metrics 
are:
+
+- [Counter](https://prometheus.io/docs/concepts/metric_types/#counter): a 
cumulative metric that represents a single monotonically increasing counter 
whose value can only increase or be reset to zero on restart.
+- [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): a *gauge* 
is a metric that represents a single numerical value that can arbitrarily go up 
and down.
+- [Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram): a 
histogram samples observations (usually things like request durations or 
response sizes) and counts them in configurable buckets.
+- [Summary](https://prometheus.io/docs/concepts/metric_types/#summary): 
similar to a histogram, a summary samples observations (usually things like 
request durations and response sizes). While it also provides a total count of 
observations and a sum of all observed values, it calculates configurable 
quantiles over a sliding time window.
+
+## ZooKeeper
+
+The zookeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by configuring a system
+property `stats_server_port` to use a different port.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_znode_count | Gauge | Number of z-nodes stored. |
+| zookeeper_server_data_size_bytes | Gauge | The total size of all of z-nodes 
stored. |
+| zookeeper_server_connections | Gauge | Number of currently opened 
connections. |
+| zookeeper_server_watches_count | Gauge | Number of watchers registered. |
+| zookeeper_server_ephemerals_count | Gauge | Number of ephemeral z-nodes. |
+
+### Request metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_requests | Counter | The total number of requests received 
by a particular server. |
+| zookeeper_server_requests_latency_ms | Summary | The requests latency 
calculated in milliseconds.  Available labels: *type* (write, read).  
*write*: the requests that write data to zookeeper.*read*: the 
requests that read data from zookeeper.|
+
+## BookKeeper
+
+The bookkeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by updating `prometheusStatsHttpPort`
+in `bookkeeper.conf` configuration file.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_SERVER_STATUS | Gauge | The server status for bookie server. 
1: the bookie is running in writable mode.0: the bookie is 
running in readonly mode. |
+| bookkeeper_server_ADD_ENTRY_count | Counter | The total number of ADD_ENTRY 
requests received at the bookie. Label `success` used to distinguish successes 
and failures. |
+| bookkeeper_server_READ_ENTRY_count | Counter | The total number of 
READ_ENTRY requests received at the bookie. Label `success` used to distinguish 
successes and failures. |
+| bookie_WRITE_BYTES | Counter | The total number of bytes written to the 
bookie. |
+| bookie_READ_BYTES | Counter | The total number of bytes read from the 
bookie. |
+| bookkeeper_server_ADD_ENTRY_REQUEST | Histogram | The histogram of request 
latency of ADD_ENTRY requests at the bookie. Label `success` used to 
distinguish successes and failures. | 
+| bookkeeper_server_READ_ENTRY_REQUEST | Histogram | The histogram of request 
latency of READ_ENTRY requests at the bookie. Label `success` used to 
distinguish successes and failures. | 
+
+### Journal metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_journal_JOURNAL_SYNC_count | Counter | The total number of journal 
fsync operations happening at the bookie. Label `success` used to distinguish 
successes and failures. |
+| bookie_journal_JOURNAL_QUEUE_SIZE | Gauge | The total number of requests 
pending in the journal queue. |
+| bookie_journal_JOURNAL_FORCE_WRITE_QUEUE_SIZE | Gauge | The total number of 
force write (fsync) requests pending in the force-write queue. |
+| bookie_journal_JOURNAL_CB_QUEUE_SIZE | Gauge | The total number of callbacks 
pending in the callback queue. |
+| bookie_journal_JOURNAL_ADD_ENTRY | Histogram | The histogram of request 
latency of adding entries to the journal. |
+| bookie_journal_JOURNAL_SYNC | Histogram | The histogram of fsync latency of 
syncing data to the journal disk. |
+
+### Storage metrics
+
+| Name | Type | Description |
+|---|---|---|
+|

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #4910: [doc] Add a documentation page for metrics reference

2019-08-14 Thread GitBox

Jennifer88huang commented on a change in pull request #4910: [doc] Add a 
documentation page for metrics reference
URL: https://github.com/apache/pulsar/pull/4910#discussion_r313785031
 
 

 ##
 File path: site2/docs/reference-metrics.md
 ##
 @@ -0,0 +1,243 @@
+---
+id: reference-metrics
+title: Pulsar Metrics
+sidebar_label: Pulsar Metrics
+---
+
+
+  table{
+font-size: 80%;
+  }
+
+
+Pulsar exposes metrics in Prometheus format that can be collected and used for 
monitoring the health of the cluster.
+
+* [ZooKeeper](#zookeeper)
+* [BookKeeper](#bookkeeper)
+* [Broker](#broker)
+
+## Overview
+
+The metrics exposed by Pulsar are in Prometheus format. The types of metrics 
are:
+
+- [Counter](https://prometheus.io/docs/concepts/metric_types/#counter): a 
cumulative metric that represents a single monotonically increasing counter 
whose value can only increase or be reset to zero on restart.
+- [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): a *gauge* 
is a metric that represents a single numerical value that can arbitrarily go up 
and down.
+- [Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram): a 
histogram samples observations (usually things like request durations or 
response sizes) and counts them in configurable buckets.
+- [Summary](https://prometheus.io/docs/concepts/metric_types/#summary): 
similar to a histogram, a summary samples observations (usually things like 
request durations and response sizes). While it also provides a total count of 
observations and a sum of all observed values, it calculates configurable 
quantiles over a sliding time window.
+
+## ZooKeeper
+
+The zookeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by configuring a system
+property `stats_server_port` to use a different port.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_znode_count | Gauge | Number of z-nodes stored. |
+| zookeeper_server_data_size_bytes | Gauge | The total size of all of z-nodes 
stored. |
+| zookeeper_server_connections | Gauge | Number of currently opened 
connections. |
+| zookeeper_server_watches_count | Gauge | Number of watchers registered. |
+| zookeeper_server_ephemerals_count | Gauge | Number of ephemeral z-nodes. |
+
+### Request metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_requests | Counter | The total number of requests received 
by a particular server. |
+| zookeeper_server_requests_latency_ms | Summary | The requests latency 
calculated in milliseconds.  Available labels: *type* (write, read).  
*write*: the requests that write data to zookeeper.*read*: the 
requests that read data from zookeeper.|
+
+## BookKeeper
+
+The bookkeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by updating `prometheusStatsHttpPort`
 
 Review comment:
   bookkeeper --> BookKeeper
   Check and refine all similar cases.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #4910: [doc] Add a documentation page for metrics reference

2019-08-14 Thread GitBox

Jennifer88huang commented on a change in pull request #4910: [doc] Add a 
documentation page for metrics reference
URL: https://github.com/apache/pulsar/pull/4910#discussion_r313783934
 
 

 ##
 File path: site2/docs/reference-metrics.md
 ##
 @@ -0,0 +1,243 @@
+---
+id: reference-metrics
+title: Pulsar Metrics
+sidebar_label: Pulsar Metrics
+---
+
+
+  table{
+font-size: 80%;
+  }
+
+
+Pulsar exposes metrics in Prometheus format that can be collected and used for 
monitoring the health of the cluster.
+
+* [ZooKeeper](#zookeeper)
+* [BookKeeper](#bookkeeper)
+* [Broker](#broker)
+
+## Overview
+
+The metrics exposed by Pulsar are in Prometheus format. The types of metrics 
are:
+
+- [Counter](https://prometheus.io/docs/concepts/metric_types/#counter): a 
cumulative metric that represents a single monotonically increasing counter 
whose value can only increase or be reset to zero on restart.
+- [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): a *gauge* 
is a metric that represents a single numerical value that can arbitrarily go up 
and down.
+- [Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram): a 
histogram samples observations (usually things like request durations or 
response sizes) and counts them in configurable buckets.
+- [Summary](https://prometheus.io/docs/concepts/metric_types/#summary): 
similar to a histogram, a summary samples observations (usually things like 
request durations and response sizes). While it also provides a total count of 
observations and a sum of all observed values, it calculates configurable 
quantiles over a sliding time window.
+
+## ZooKeeper
+
+The zookeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by configuring a system
 
 Review comment:
   You can use a different port by configuring the `stats_server_port` system 
property.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #4910: [doc] Add a documentation page for metrics reference

2019-08-14 Thread GitBox

Jennifer88huang commented on a change in pull request #4910: [doc] Add a 
documentation page for metrics reference
URL: https://github.com/apache/pulsar/pull/4910#discussion_r313790571
 
 

 ##
 File path: site2/docs/reference-metrics.md
 ##
 @@ -0,0 +1,243 @@
+---
+id: reference-metrics
+title: Pulsar Metrics
+sidebar_label: Pulsar Metrics
+---
+
+
+  table{
+font-size: 80%;
+  }
+
+
+Pulsar exposes metrics in Prometheus format that can be collected and used for 
monitoring the health of the cluster.
+
+* [ZooKeeper](#zookeeper)
+* [BookKeeper](#bookkeeper)
+* [Broker](#broker)
+
+## Overview
+
+The metrics exposed by Pulsar are in Prometheus format. The types of metrics 
are:
+
+- [Counter](https://prometheus.io/docs/concepts/metric_types/#counter): a 
cumulative metric that represents a single monotonically increasing counter 
whose value can only increase or be reset to zero on restart.
+- [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): a *gauge* 
is a metric that represents a single numerical value that can arbitrarily go up 
and down.
+- [Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram): a 
histogram samples observations (usually things like request durations or 
response sizes) and counts them in configurable buckets.
+- [Summary](https://prometheus.io/docs/concepts/metric_types/#summary): 
similar to a histogram, a summary samples observations (usually things like 
request durations and response sizes). While it also provides a total count of 
observations and a sum of all observed values, it calculates configurable 
quantiles over a sliding time window.
+
+## ZooKeeper
+
+The zookeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by configuring a system
+property `stats_server_port` to use a different port.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_znode_count | Gauge | Number of z-nodes stored. |
+| zookeeper_server_data_size_bytes | Gauge | The total size of all of z-nodes 
stored. |
+| zookeeper_server_connections | Gauge | Number of currently opened 
connections. |
+| zookeeper_server_watches_count | Gauge | Number of watchers registered. |
+| zookeeper_server_ephemerals_count | Gauge | Number of ephemeral z-nodes. |
+
+### Request metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_requests | Counter | The total number of requests received 
by a particular server. |
+| zookeeper_server_requests_latency_ms | Summary | The requests latency 
calculated in milliseconds.  Available labels: *type* (write, read).  
*write*: the requests that write data to zookeeper.*read*: the 
requests that read data from zookeeper.|
+
+## BookKeeper
+
+The bookkeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by updating `prometheusStatsHttpPort`
+in `bookkeeper.conf` configuration file.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_SERVER_STATUS | Gauge | The server status for bookie server. 
1: the bookie is running in writable mode.0: the bookie is 
running in readonly mode. |
+| bookkeeper_server_ADD_ENTRY_count | Counter | The total number of ADD_ENTRY 
requests received at the bookie. Label `success` used to distinguish successes 
and failures. |
+| bookkeeper_server_READ_ENTRY_count | Counter | The total number of 
READ_ENTRY requests received at the bookie. Label `success` used to distinguish 
successes and failures. |
+| bookie_WRITE_BYTES | Counter | The total number of bytes written to the 
bookie. |
+| bookie_READ_BYTES | Counter | The total number of bytes read from the 
bookie. |
+| bookkeeper_server_ADD_ENTRY_REQUEST | Histogram | The histogram of request 
latency of ADD_ENTRY requests at the bookie. Label `success` used to 
distinguish successes and failures. | 
+| bookkeeper_server_READ_ENTRY_REQUEST | Histogram | The histogram of request 
latency of READ_ENTRY requests at the bookie. Label `success` used to 
distinguish successes and failures. | 
+
+### Journal metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_journal_JOURNAL_SYNC_count | Counter | The total number of journal 
fsync operations happening at the bookie. Label `success` used to distinguish 
successes and failures. |
+| bookie_journal_JOURNAL_QUEUE_SIZE | Gauge | The total number of requests 
pending in the journal queue. |
+| bookie_journal_JOURNAL_FORCE_WRITE_QUEUE_SIZE | Gauge | The total number of 
force write (fsync) requests pending in the force-write queue. |
+| bookie_journal_JOURNAL_CB_QUEUE_SIZE | Gauge | The total number of callbacks 
pending in the callback queue. |
+| bookie_journal_JOURNAL_ADD_ENTRY | Histogram | The histogram of request 
latency of adding entries to the journal. |
+| bookie_journal_JOURNAL_SYNC | Histogram | The histogram of fsync latency of 
syncing data to the journal disk. |
+
+### Storage metrics
+
+| Name | Type | Description |
+|---|---|---|
+|

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #4910: [doc] Add a documentation page for metrics reference

2019-08-14 Thread GitBox

Jennifer88huang commented on a change in pull request #4910: [doc] Add a 
documentation page for metrics reference
URL: https://github.com/apache/pulsar/pull/4910#discussion_r313784310
 
 

 ##
 File path: site2/docs/reference-metrics.md
 ##
 @@ -0,0 +1,243 @@
+---
+id: reference-metrics
+title: Pulsar Metrics
+sidebar_label: Pulsar Metrics
+---
+
+
+  table{
+font-size: 80%;
+  }
+
+
+Pulsar exposes metrics in Prometheus format that can be collected and used for 
monitoring the health of the cluster.
+
+* [ZooKeeper](#zookeeper)
+* [BookKeeper](#bookkeeper)
+* [Broker](#broker)
+
+## Overview
+
+The metrics exposed by Pulsar are in Prometheus format. The types of metrics 
are:
+
+- [Counter](https://prometheus.io/docs/concepts/metric_types/#counter): a 
cumulative metric that represents a single monotonically increasing counter 
whose value can only increase or be reset to zero on restart.
+- [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): a *gauge* 
is a metric that represents a single numerical value that can arbitrarily go up 
and down.
+- [Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram): a 
histogram samples observations (usually things like request durations or 
response sizes) and counts them in configurable buckets.
+- [Summary](https://prometheus.io/docs/concepts/metric_types/#summary): 
similar to a histogram, a summary samples observations (usually things like 
request durations and response sizes). While it also provides a total count of 
observations and a sum of all observed values, it calculates configurable 
quantiles over a sliding time window.
+
+## ZooKeeper
+
+The zookeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by configuring a system
+property `stats_server_port` to use a different port.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_znode_count | Gauge | Number of z-nodes stored. |
 
 Review comment:
   You can check and refine all similar cases.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #4910: [doc] Add a documentation page for metrics reference

2019-08-14 Thread GitBox

Jennifer88huang commented on a change in pull request #4910: [doc] Add a 
documentation page for metrics reference
URL: https://github.com/apache/pulsar/pull/4910#discussion_r313784203
 
 

 ##
 File path: site2/docs/reference-metrics.md
 ##
 @@ -0,0 +1,243 @@
+---
+id: reference-metrics
+title: Pulsar Metrics
+sidebar_label: Pulsar Metrics
+---
+
+
+  table{
+font-size: 80%;
+  }
+
+
+Pulsar exposes metrics in Prometheus format that can be collected and used for 
monitoring the health of the cluster.
+
+* [ZooKeeper](#zookeeper)
+* [BookKeeper](#bookkeeper)
+* [Broker](#broker)
+
+## Overview
+
+The metrics exposed by Pulsar are in Prometheus format. The types of metrics 
are:
+
+- [Counter](https://prometheus.io/docs/concepts/metric_types/#counter): a 
cumulative metric that represents a single monotonically increasing counter 
whose value can only increase or be reset to zero on restart.
+- [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): a *gauge* 
is a metric that represents a single numerical value that can arbitrarily go up 
and down.
+- [Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram): a 
histogram samples observations (usually things like request durations or 
response sizes) and counts them in configurable buckets.
+- [Summary](https://prometheus.io/docs/concepts/metric_types/#summary): 
similar to a histogram, a summary samples observations (usually things like 
request durations and response sizes). While it also provides a total count of 
observations and a sum of all observed values, it calculates configurable 
quantiles over a sliding time window.
+
+## ZooKeeper
+
+The zookeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by configuring a system
+property `stats_server_port` to use a different port.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_znode_count | Gauge | Number of z-nodes stored. |
 
 Review comment:
   ```suggestion
   | zookeeper_server_znode_count | Gauge | The number of z-nodes stored. |
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #4910: [doc] Add a documentation page for metrics reference

2019-08-14 Thread GitBox

Jennifer88huang commented on a change in pull request #4910: [doc] Add a 
documentation page for metrics reference
URL: https://github.com/apache/pulsar/pull/4910#discussion_r313787705
 
 

 ##
 File path: site2/docs/reference-metrics.md
 ##
 @@ -0,0 +1,243 @@
+---
+id: reference-metrics
+title: Pulsar Metrics
+sidebar_label: Pulsar Metrics
+---
+
+
+  table{
+font-size: 80%;
+  }
+
+
+Pulsar exposes metrics in Prometheus format that can be collected and used for 
monitoring the health of the cluster.
+
+* [ZooKeeper](#zookeeper)
+* [BookKeeper](#bookkeeper)
+* [Broker](#broker)
+
+## Overview
+
+The metrics exposed by Pulsar are in Prometheus format. The types of metrics 
are:
+
+- [Counter](https://prometheus.io/docs/concepts/metric_types/#counter): a 
cumulative metric that represents a single monotonically increasing counter 
whose value can only increase or be reset to zero on restart.
+- [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): a *gauge* 
is a metric that represents a single numerical value that can arbitrarily go up 
and down.
+- [Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram): a 
histogram samples observations (usually things like request durations or 
response sizes) and counts them in configurable buckets.
+- [Summary](https://prometheus.io/docs/concepts/metric_types/#summary): 
similar to a histogram, a summary samples observations (usually things like 
request durations and response sizes). While it also provides a total count of 
observations and a sum of all observed values, it calculates configurable 
quantiles over a sliding time window.
+
+## ZooKeeper
+
+The zookeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by configuring a system
+property `stats_server_port` to use a different port.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_znode_count | Gauge | Number of z-nodes stored. |
+| zookeeper_server_data_size_bytes | Gauge | The total size of all of z-nodes 
stored. |
+| zookeeper_server_connections | Gauge | Number of currently opened 
connections. |
+| zookeeper_server_watches_count | Gauge | Number of watchers registered. |
+| zookeeper_server_ephemerals_count | Gauge | Number of ephemeral z-nodes. |
+
+### Request metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_requests | Counter | The total number of requests received 
by a particular server. |
+| zookeeper_server_requests_latency_ms | Summary | The requests latency 
calculated in milliseconds.  Available labels: *type* (write, read).  
*write*: the requests that write data to zookeeper.*read*: the 
requests that read data from zookeeper.|
+
+## BookKeeper
+
+The bookkeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by updating `prometheusStatsHttpPort`
+in `bookkeeper.conf` configuration file.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_SERVER_STATUS | Gauge | The server status for bookie server. 
1: the bookie is running in writable mode.0: the bookie is 
running in readonly mode. |
+| bookkeeper_server_ADD_ENTRY_count | Counter | The total number of ADD_ENTRY 
requests received at the bookie. Label `success` used to distinguish successes 
and failures. |
+| bookkeeper_server_READ_ENTRY_count | Counter | The total number of 
READ_ENTRY requests received at the bookie. Label `success` used to distinguish 
successes and failures. |
+| bookie_WRITE_BYTES | Counter | The total number of bytes written to the 
bookie. |
+| bookie_READ_BYTES | Counter | The total number of bytes read from the 
bookie. |
+| bookkeeper_server_ADD_ENTRY_REQUEST | Histogram | The histogram of request 
latency of ADD_ENTRY requests at the bookie. Label `success` used to 
distinguish successes and failures. | 
+| bookkeeper_server_READ_ENTRY_REQUEST | Histogram | The histogram of request 
latency of READ_ENTRY requests at the bookie. Label `success` used to 
distinguish successes and failures. | 
+
+### Journal metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_journal_JOURNAL_SYNC_count | Counter | The total number of journal 
fsync operations happening at the bookie. Label `success` used to distinguish 
successes and failures. |
 
 Review comment:
   ```suggestion
   | bookie_journal_JOURNAL_SYNC_count | Counter | The total number of journal 
fsync operations happening at the bookie. The `success` label is used to 
distinguish successes and failures. |
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #4910: [doc] Add a documentation page for metrics reference

2019-08-14 Thread GitBox

Jennifer88huang commented on a change in pull request #4910: [doc] Add a 
documentation page for metrics reference
URL: https://github.com/apache/pulsar/pull/4910#discussion_r313782133
 
 

 ##
 File path: site2/docs/reference-metrics.md
 ##
 @@ -0,0 +1,243 @@
+---
+id: reference-metrics
+title: Pulsar Metrics
+sidebar_label: Pulsar Metrics
+---
+

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #4910: [doc] Add a documentation page for metrics reference

2019-08-14 Thread GitBox

Jennifer88huang commented on a change in pull request #4910: [doc] Add a 
documentation page for metrics reference
URL: https://github.com/apache/pulsar/pull/4910#discussion_r313788304
 
 

 ##
 File path: site2/docs/reference-metrics.md
 ##
 @@ -0,0 +1,243 @@
+---
+id: reference-metrics
+title: Pulsar Metrics
+sidebar_label: Pulsar Metrics
+---
+
+
+  table{
+font-size: 80%;
+  }
+
+
+Pulsar exposes metrics in Prometheus format that can be collected and used for 
monitoring the health of the cluster.
+
+* [ZooKeeper](#zookeeper)
+* [BookKeeper](#bookkeeper)
+* [Broker](#broker)
+
+## Overview
+
+The metrics exposed by Pulsar are in Prometheus format. The types of metrics 
are:
+
+- [Counter](https://prometheus.io/docs/concepts/metric_types/#counter): a 
cumulative metric that represents a single monotonically increasing counter 
whose value can only increase or be reset to zero on restart.
+- [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): a *gauge* 
is a metric that represents a single numerical value that can arbitrarily go up 
and down.
+- [Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram): a 
histogram samples observations (usually things like request durations or 
response sizes) and counts them in configurable buckets.
+- [Summary](https://prometheus.io/docs/concepts/metric_types/#summary): 
similar to a histogram, a summary samples observations (usually things like 
request durations and response sizes). While it also provides a total count of 
observations and a sum of all observed values, it calculates configurable 
quantiles over a sliding time window.
+
+## ZooKeeper
+
+The zookeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by configuring a system
+property `stats_server_port` to use a different port.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_znode_count | Gauge | Number of z-nodes stored. |
+| zookeeper_server_data_size_bytes | Gauge | The total size of all of z-nodes 
stored. |
+| zookeeper_server_connections | Gauge | Number of currently opened 
connections. |
+| zookeeper_server_watches_count | Gauge | Number of watchers registered. |
+| zookeeper_server_ephemerals_count | Gauge | Number of ephemeral z-nodes. |
+
+### Request metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_requests | Counter | The total number of requests received 
by a particular server. |
+| zookeeper_server_requests_latency_ms | Summary | The requests latency 
calculated in milliseconds.  Available labels: *type* (write, read).  
*write*: the requests that write data to zookeeper.*read*: the 
requests that read data from zookeeper.|
+
+## BookKeeper
+
+The bookkeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by updating `prometheusStatsHttpPort`
+in `bookkeeper.conf` configuration file.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_SERVER_STATUS | Gauge | The server status for bookie server. 
1: the bookie is running in writable mode.0: the bookie is 
running in readonly mode. |
+| bookkeeper_server_ADD_ENTRY_count | Counter | The total number of ADD_ENTRY 
requests received at the bookie. Label `success` used to distinguish successes 
and failures. |
+| bookkeeper_server_READ_ENTRY_count | Counter | The total number of 
READ_ENTRY requests received at the bookie. Label `success` used to distinguish 
successes and failures. |
+| bookie_WRITE_BYTES | Counter | The total number of bytes written to the 
bookie. |
+| bookie_READ_BYTES | Counter | The total number of bytes read from the 
bookie. |
+| bookkeeper_server_ADD_ENTRY_REQUEST | Histogram | The histogram of request 
latency of ADD_ENTRY requests at the bookie. Label `success` used to 
distinguish successes and failures. | 
+| bookkeeper_server_READ_ENTRY_REQUEST | Histogram | The histogram of request 
latency of READ_ENTRY requests at the bookie. Label `success` used to 
distinguish successes and failures. | 
+
+### Journal metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_journal_JOURNAL_SYNC_count | Counter | The total number of journal 
fsync operations happening at the bookie. Label `success` used to distinguish 
successes and failures. |
+| bookie_journal_JOURNAL_QUEUE_SIZE | Gauge | The total number of requests 
pending in the journal queue. |
+| bookie_journal_JOURNAL_FORCE_WRITE_QUEUE_SIZE | Gauge | The total number of 
force write (fsync) requests pending in the force-write queue. |
+| bookie_journal_JOURNAL_CB_QUEUE_SIZE | Gauge | The total number of callbacks 
pending in the callback queue. |
+| bookie_journal_JOURNAL_ADD_ENTRY | Histogram | The histogram of request 
latency of adding entries to the journal. |
+| bookie_journal_JOURNAL_SYNC | Histogram | The histogram of fsync latency of 
syncing data to the journal disk. |
+
+### Storage metrics
+
+| Name | Type | Description |
+|---|---|---|
+|

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #4910: [doc] Add a documentation page for metrics reference

2019-08-14 Thread GitBox

Jennifer88huang commented on a change in pull request #4910: [doc] Add a 
documentation page for metrics reference
URL: https://github.com/apache/pulsar/pull/4910#discussion_r313790386
 
 

 ##
 File path: site2/docs/reference-metrics.md
 ##
 @@ -0,0 +1,243 @@
+---
+id: reference-metrics
+title: Pulsar Metrics
+sidebar_label: Pulsar Metrics
+---
+
+
+  table{
+font-size: 80%;
+  }
+
+
+Pulsar exposes metrics in Prometheus format that can be collected and used for 
monitoring the health of the cluster.
+
+* [ZooKeeper](#zookeeper)
+* [BookKeeper](#bookkeeper)
+* [Broker](#broker)
+
+## Overview
+
+The metrics exposed by Pulsar are in Prometheus format. The types of metrics 
are:
+
+- [Counter](https://prometheus.io/docs/concepts/metric_types/#counter): a 
cumulative metric that represents a single monotonically increasing counter 
whose value can only increase or be reset to zero on restart.
+- [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): a *gauge* 
is a metric that represents a single numerical value that can arbitrarily go up 
and down.
+- [Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram): a 
histogram samples observations (usually things like request durations or 
response sizes) and counts them in configurable buckets.
+- [Summary](https://prometheus.io/docs/concepts/metric_types/#summary): 
similar to a histogram, a summary samples observations (usually things like 
request durations and response sizes). While it also provides a total count of 
observations and a sum of all observed values, it calculates configurable 
quantiles over a sliding time window.
+
+## ZooKeeper
+
+The zookeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by configuring a system
+property `stats_server_port` to use a different port.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_znode_count | Gauge | Number of z-nodes stored. |
+| zookeeper_server_data_size_bytes | Gauge | The total size of all of z-nodes 
stored. |
+| zookeeper_server_connections | Gauge | Number of currently opened 
connections. |
+| zookeeper_server_watches_count | Gauge | Number of watchers registered. |
+| zookeeper_server_ephemerals_count | Gauge | Number of ephemeral z-nodes. |
+
+### Request metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_requests | Counter | The total number of requests received 
by a particular server. |
+| zookeeper_server_requests_latency_ms | Summary | The requests latency 
calculated in milliseconds.  Available labels: *type* (write, read).  
*write*: the requests that write data to zookeeper.*read*: the 
requests that read data from zookeeper.|
+
+## BookKeeper
+
+The bookkeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by updating `prometheusStatsHttpPort`
+in `bookkeeper.conf` configuration file.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_SERVER_STATUS | Gauge | The server status for bookie server. 
1: the bookie is running in writable mode.0: the bookie is 
running in readonly mode. |
+| bookkeeper_server_ADD_ENTRY_count | Counter | The total number of ADD_ENTRY 
requests received at the bookie. Label `success` used to distinguish successes 
and failures. |
+| bookkeeper_server_READ_ENTRY_count | Counter | The total number of 
READ_ENTRY requests received at the bookie. Label `success` used to distinguish 
successes and failures. |
+| bookie_WRITE_BYTES | Counter | The total number of bytes written to the 
bookie. |
+| bookie_READ_BYTES | Counter | The total number of bytes read from the 
bookie. |
+| bookkeeper_server_ADD_ENTRY_REQUEST | Histogram | The histogram of request 
latency of ADD_ENTRY requests at the bookie. Label `success` used to 
distinguish successes and failures. | 
+| bookkeeper_server_READ_ENTRY_REQUEST | Histogram | The histogram of request 
latency of READ_ENTRY requests at the bookie. Label `success` used to 
distinguish successes and failures. | 
+
+### Journal metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_journal_JOURNAL_SYNC_count | Counter | The total number of journal 
fsync operations happening at the bookie. Label `success` used to distinguish 
successes and failures. |
+| bookie_journal_JOURNAL_QUEUE_SIZE | Gauge | The total number of requests 
pending in the journal queue. |
+| bookie_journal_JOURNAL_FORCE_WRITE_QUEUE_SIZE | Gauge | The total number of 
force write (fsync) requests pending in the force-write queue. |
+| bookie_journal_JOURNAL_CB_QUEUE_SIZE | Gauge | The total number of callbacks 
pending in the callback queue. |
+| bookie_journal_JOURNAL_ADD_ENTRY | Histogram | The histogram of request 
latency of adding entries to the journal. |
+| bookie_journal_JOURNAL_SYNC | Histogram | The histogram of fsync latency of 
syncing data to the journal disk. |
+
+### Storage metrics
+
+| Name | Type | Description |
+|---|---|---|
+|

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #4910: [doc] Add a documentation page for metrics reference

2019-08-14 Thread GitBox

Jennifer88huang commented on a change in pull request #4910: [doc] Add a 
documentation page for metrics reference
URL: https://github.com/apache/pulsar/pull/4910#discussion_r313787952
 
 

 ##
 File path: site2/docs/reference-metrics.md
 ##
 @@ -0,0 +1,243 @@
+---
+id: reference-metrics
+title: Pulsar Metrics
+sidebar_label: Pulsar Metrics
+---
+
+
+  table{
+font-size: 80%;
+  }
+
+
+Pulsar exposes metrics in Prometheus format that can be collected and used for 
monitoring the health of the cluster.
+
+* [ZooKeeper](#zookeeper)
+* [BookKeeper](#bookkeeper)
+* [Broker](#broker)
+
+## Overview
+
+The metrics exposed by Pulsar are in Prometheus format. The types of metrics 
are:
+
+- [Counter](https://prometheus.io/docs/concepts/metric_types/#counter): a 
cumulative metric that represents a single monotonically increasing counter 
whose value can only increase or be reset to zero on restart.
+- [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): a *gauge* 
is a metric that represents a single numerical value that can arbitrarily go up 
and down.
+- [Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram): a 
histogram samples observations (usually things like request durations or 
response sizes) and counts them in configurable buckets.
+- [Summary](https://prometheus.io/docs/concepts/metric_types/#summary): 
similar to a histogram, a summary samples observations (usually things like 
request durations and response sizes). While it also provides a total count of 
observations and a sum of all observed values, it calculates configurable 
quantiles over a sliding time window.
+
+## ZooKeeper
+
+The zookeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by configuring a system
+property `stats_server_port` to use a different port.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_znode_count | Gauge | Number of z-nodes stored. |
+| zookeeper_server_data_size_bytes | Gauge | The total size of all of z-nodes 
stored. |
+| zookeeper_server_connections | Gauge | Number of currently opened 
connections. |
+| zookeeper_server_watches_count | Gauge | Number of watchers registered. |
+| zookeeper_server_ephemerals_count | Gauge | Number of ephemeral z-nodes. |
+
+### Request metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_requests | Counter | The total number of requests received 
by a particular server. |
+| zookeeper_server_requests_latency_ms | Summary | The requests latency 
calculated in milliseconds.  Available labels: *type* (write, read).  
*write*: the requests that write data to zookeeper.*read*: the 
requests that read data from zookeeper.|
+
+## BookKeeper
+
+The bookkeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by updating `prometheusStatsHttpPort`
+in `bookkeeper.conf` configuration file.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_SERVER_STATUS | Gauge | The server status for bookie server. 
1: the bookie is running in writable mode.0: the bookie is 
running in readonly mode. |
+| bookkeeper_server_ADD_ENTRY_count | Counter | The total number of ADD_ENTRY 
requests received at the bookie. Label `success` used to distinguish successes 
and failures. |
+| bookkeeper_server_READ_ENTRY_count | Counter | The total number of 
READ_ENTRY requests received at the bookie. Label `success` used to distinguish 
successes and failures. |
+| bookie_WRITE_BYTES | Counter | The total number of bytes written to the 
bookie. |
+| bookie_READ_BYTES | Counter | The total number of bytes read from the 
bookie. |
+| bookkeeper_server_ADD_ENTRY_REQUEST | Histogram | The histogram of request 
latency of ADD_ENTRY requests at the bookie. Label `success` used to 
distinguish successes and failures. | 
+| bookkeeper_server_READ_ENTRY_REQUEST | Histogram | The histogram of request 
latency of READ_ENTRY requests at the bookie. Label `success` used to 
distinguish successes and failures. | 
+
+### Journal metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_journal_JOURNAL_SYNC_count | Counter | The total number of journal 
fsync operations happening at the bookie. Label `success` used to distinguish 
successes and failures. |
+| bookie_journal_JOURNAL_QUEUE_SIZE | Gauge | The total number of requests 
pending in the journal queue. |
+| bookie_journal_JOURNAL_FORCE_WRITE_QUEUE_SIZE | Gauge | The total number of 
force write (fsync) requests pending in the force-write queue. |
+| bookie_journal_JOURNAL_CB_QUEUE_SIZE | Gauge | The total number of callbacks 
pending in the callback queue. |
+| bookie_journal_JOURNAL_ADD_ENTRY | Histogram | The histogram of request 
latency of adding entries to the journal. |
+| bookie_journal_JOURNAL_SYNC | Histogram | The histogram of fsync latency of 
syncing data to the journal disk. |
+
+### Storage metrics
+
+| Name | Type | Description |
+|---|---|---|
+|

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #4910: [doc] Add a documentation page for metrics reference

2019-08-14 Thread GitBox

Jennifer88huang commented on a change in pull request #4910: [doc] Add a 
documentation page for metrics reference
URL: https://github.com/apache/pulsar/pull/4910#discussion_r313789127
 
 

 ##
 File path: site2/docs/reference-metrics.md
 ##
 @@ -0,0 +1,243 @@
+---
+id: reference-metrics
+title: Pulsar Metrics
+sidebar_label: Pulsar Metrics
+---
+
+
+  table{
+font-size: 80%;
+  }
+
+
+Pulsar exposes metrics in Prometheus format that can be collected and used for 
monitoring the health of the cluster.
+
+* [ZooKeeper](#zookeeper)
+* [BookKeeper](#bookkeeper)
+* [Broker](#broker)
+
+## Overview
+
+The metrics exposed by Pulsar are in Prometheus format. The types of metrics 
are:
+
+- [Counter](https://prometheus.io/docs/concepts/metric_types/#counter): a 
cumulative metric that represents a single monotonically increasing counter 
whose value can only increase or be reset to zero on restart.
+- [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): a *gauge* 
is a metric that represents a single numerical value that can arbitrarily go up 
and down.
+- [Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram): a 
histogram samples observations (usually things like request durations or 
response sizes) and counts them in configurable buckets.
+- [Summary](https://prometheus.io/docs/concepts/metric_types/#summary): 
similar to a histogram, a summary samples observations (usually things like 
request durations and response sizes). While it also provides a total count of 
observations and a sum of all observed values, it calculates configurable 
quantiles over a sliding time window.
+
+## ZooKeeper
+
+The zookeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by configuring a system
+property `stats_server_port` to use a different port.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_znode_count | Gauge | Number of z-nodes stored. |
+| zookeeper_server_data_size_bytes | Gauge | The total size of all of z-nodes 
stored. |
+| zookeeper_server_connections | Gauge | Number of currently opened 
connections. |
+| zookeeper_server_watches_count | Gauge | Number of watchers registered. |
+| zookeeper_server_ephemerals_count | Gauge | Number of ephemeral z-nodes. |
+
+### Request metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_requests | Counter | The total number of requests received 
by a particular server. |
+| zookeeper_server_requests_latency_ms | Summary | The requests latency 
calculated in milliseconds.  Available labels: *type* (write, read).  
*write*: the requests that write data to zookeeper.*read*: the 
requests that read data from zookeeper.|
+
+## BookKeeper
+
+The bookkeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by updating `prometheusStatsHttpPort`
+in `bookkeeper.conf` configuration file.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_SERVER_STATUS | Gauge | The server status for bookie server. 
1: the bookie is running in writable mode.0: the bookie is 
running in readonly mode. |
+| bookkeeper_server_ADD_ENTRY_count | Counter | The total number of ADD_ENTRY 
requests received at the bookie. Label `success` used to distinguish successes 
and failures. |
+| bookkeeper_server_READ_ENTRY_count | Counter | The total number of 
READ_ENTRY requests received at the bookie. Label `success` used to distinguish 
successes and failures. |
+| bookie_WRITE_BYTES | Counter | The total number of bytes written to the 
bookie. |
+| bookie_READ_BYTES | Counter | The total number of bytes read from the 
bookie. |
+| bookkeeper_server_ADD_ENTRY_REQUEST | Histogram | The histogram of request 
latency of ADD_ENTRY requests at the bookie. Label `success` used to 
distinguish successes and failures. | 
+| bookkeeper_server_READ_ENTRY_REQUEST | Histogram | The histogram of request 
latency of READ_ENTRY requests at the bookie. Label `success` used to 
distinguish successes and failures. | 
+
+### Journal metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_journal_JOURNAL_SYNC_count | Counter | The total number of journal 
fsync operations happening at the bookie. Label `success` used to distinguish 
successes and failures. |
+| bookie_journal_JOURNAL_QUEUE_SIZE | Gauge | The total number of requests 
pending in the journal queue. |
+| bookie_journal_JOURNAL_FORCE_WRITE_QUEUE_SIZE | Gauge | The total number of 
force write (fsync) requests pending in the force-write queue. |
+| bookie_journal_JOURNAL_CB_QUEUE_SIZE | Gauge | The total number of callbacks 
pending in the callback queue. |
+| bookie_journal_JOURNAL_ADD_ENTRY | Histogram | The histogram of request 
latency of adding entries to the journal. |
+| bookie_journal_JOURNAL_SYNC | Histogram | The histogram of fsync latency of 
syncing data to the journal disk. |
+
+### Storage metrics
+
+| Name | Type | Description |
+|---|---|---|
+|

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #4910: [doc] Add a documentation page for metrics reference

2019-08-14 Thread GitBox

Jennifer88huang commented on a change in pull request #4910: [doc] Add a 
documentation page for metrics reference
URL: https://github.com/apache/pulsar/pull/4910#discussion_r313783263
 
 

 ##
 File path: site2/docs/reference-metrics.md
 ##
 @@ -0,0 +1,243 @@
+---
+id: reference-metrics
+title: Pulsar Metrics
+sidebar_label: Pulsar Metrics
+---
+
+
+  table{
+font-size: 80%;
+  }
+
+
+Pulsar exposes metrics in Prometheus format that can be collected and used for 
monitoring the health of the cluster.
+
+* [ZooKeeper](#zookeeper)
+* [BookKeeper](#bookkeeper)
+* [Broker](#broker)
+
+## Overview
+
+The metrics exposed by Pulsar are in Prometheus format. The types of metrics 
are:
+
+- [Counter](https://prometheus.io/docs/concepts/metric_types/#counter): a 
cumulative metric that represents a single monotonically increasing counter 
whose value can only increase or be reset to zero on restart.
+- [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): a *gauge* 
is a metric that represents a single numerical value that can arbitrarily go up 
and down.
+- [Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram): a 
histogram samples observations (usually things like request durations or 
response sizes) and counts them in configurable buckets.
+- [Summary](https://prometheus.io/docs/concepts/metric_types/#summary): 
similar to a histogram, a summary samples observations (usually things like 
request durations and response sizes). While it also provides a total count of 
observations and a sum of all observed values, it calculates configurable 
quantiles over a sliding time window.
+
+## ZooKeeper
+
+The zookeeper metrics are exposed under "/metrics" at port 8000. You can 
change the port by configuring a system
 
 Review comment:
   zookeeper --> ZooKeeper
   If there is similar cases, please check and refine throughout.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar-client-go] wolfstudy closed issue #53: Fatal error: concurrent map writes

2019-08-14 Thread GitBox

wolfstudy closed issue #53: Fatal error: concurrent map writes
URL: https://github.com/apache/pulsar-client-go/issues/53
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar-client-go] wolfstudy merged pull request #54: [Issue:53] Fix concurrent map write

2019-08-14 Thread GitBox

wolfstudy merged pull request #54: [Issue:53] Fix concurrent map write
URL: https://github.com/apache/pulsar-client-go/pull/54
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on a change in pull request #4910: [doc] Add a documentation page for metrics reference

2019-08-14 Thread GitBox

sijie commented on a change in pull request #4910: [doc] Add a documentation 
page for metrics reference
URL: https://github.com/apache/pulsar/pull/4910#discussion_r313886146
 
 

 ##
 File path: site2/docs/reference-metrics.md
 ##
 @@ -0,0 +1,243 @@
+---
+id: reference-metrics
+title: Pulsar Metrics
+sidebar_label: Pulsar Metrics
+---
+

[pulsar] branch master updated: [site2] Fix swagger of SourceBase and SinkBase display problem (#4939)

2019-08-14 Thread rxl

This is an automated email from the ASF dual-hosted git repository.

rxl pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 05a3cd8  [site2] Fix swagger of SourceBase and SinkBase display 
problem (#4939)
05a3cd8 is described below

commit 05a3cd8b81639348045f2ef5aca2e3e3a58ff5b0
Author: 冉小龙 
AuthorDate: Wed Aug 14 21:14:28 2019 +0800

[site2] Fix swagger of SourceBase and SinkBase display problem (#4939)

* [site2] Fix swagger of SourceBase and SinkBase display problem

Signed-off-by: xiaolong.ran 

* fix comments

Signed-off-by: xiaolong.ran 
---
 .../apache/pulsar/broker/admin/impl/SinksBase.java | 239 ++---
 .../pulsar/broker/admin/impl/SourcesBase.java  | 195 +
 2 files changed, 216 insertions(+), 218 deletions(-)

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/SinksBase.java
 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/SinksBase.java
index e21d861..ab2cb3d 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/SinksBase.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/SinksBase.java
@@ -61,7 +61,7 @@ public class SinksBase extends AdminResource implements 
Supplier
 @POST
 @ApiOperation(value = "Creates a new Pulsar Sink in cluster mode")
 @ApiResponses(value = {
-@ApiResponse(code = 400, message = "Invalid request (sink already 
exists, etc.)"),
+@ApiResponse(code = 400, message = "Invalid request (The Pulsar 
Sink already exists, etc.)"),
 @ApiResponse(code = 200, message = "Pulsar Sink successfully 
created"),
 @ApiResponse(code = 500, message = "Internal server error (failed 
to authorize, failed to get tenant data, failed to process package, etc.)"),
 @ApiResponse(code = 401, message = "Client is not authorized to 
perform operation"),
@@ -69,60 +69,60 @@ public class SinksBase extends AdminResource implements 
Supplier
 })
 @Path("/{tenant}/{namespace}/{sinkName}")
 @Consumes(MediaType.MULTIPART_FORM_DATA)
-public void registerSink(@ApiParam(value = "The sink's tenant")
+public void registerSink(@ApiParam(value = "The tenant of a Pulsar Sink")
  final @PathParam("tenant") String tenant,
- @ApiParam(value = "The sink's namespace")
+ @ApiParam(value = "The namespace of a Pulsar 
Sink")
  final @PathParam("namespace") String namespace,
- @ApiParam(value = "The sink's name")
+ @ApiParam(value = "The name of a Pulsar Sink")
  final @PathParam("sinkName") String sinkName,
  final @FormDataParam("data") InputStream 
uploadedInputStream,
  final @FormDataParam("data") 
FormDataContentDisposition fileDetail,
  final @FormDataParam("url") String sinkPkgUrl,
  @ApiParam(
  value =
- "A JSON value presenting a sink config 
playload. All available configuration options are:  \n" +
- "classname  \n" +
- "   The sink's class name if archive is 
file-url-path (file://)  \n" +
- "sourceSubscriptionName  \n" +
+ "A JSON value presenting config payload 
of a Pulsar Sink. All available configuration options are:  \n" +
+ "- **classname**  \n" +
+ "   The class name of a Pulsar Sink if 
archive is file-url-path (file://)  \n" +
+ "- **sourceSubscriptionName**  \n" +
  "   Pulsar source subscription name if 
user wants a specific  \n" +
  "   subscription-name for input-topic 
consumer  \n" +
- "inputs  \n" +
- "   The sink's input topic or topics 
(specified as a JSON array)  \n" +
- "topicsPattern  \n" +
+ "- **inputs**  \n" +
+ "   The input topic or topics of a Pulsar 
Sink (specified as a JSON array)  \n" +
+ "- **topicsPattern**  \n" +
  "   TopicsPattern to consume from list of 
topics under a namespace that " +
  "   match the pattern. [input] and 
[topicsPattern] are mutually " +
  "   exclusive. Add SerDe class name

[GitHub] [pulsar] wolfstudy merged pull request #4939: [site2] Fix swagger of SourceBase and SinkBase display problem

2019-08-14 Thread GitBox

wolfstudy merged pull request #4939: [site2] Fix swagger of SourceBase and 
SinkBase display problem
URL: https://github.com/apache/pulsar/pull/4939
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet commented on issue #4934: [Doc] Update Schema Guide

2019-08-14 Thread GitBox

Anonymitaet commented on issue #4934: [Doc] Update Schema Guide
URL: https://github.com/apache/pulsar/pull/4934#issuecomment-521249371
 
 
   @sijie could you please help merge this PR? Thank you


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch master updated: [Doc] Update Schema Guide (#4934)

2019-08-14 Thread sijie

This is an automated email from the ASF dual-hosted git repository.

sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 4a9f2f7  [Doc] Update Schema Guide (#4934)
4a9f2f7 is described below

commit 4a9f2f74d31f2eb3007a76ecb76d964f9e13c22f
Author: Anonymitaet <50226895+anonymit...@users.noreply.github.com>
AuthorDate: Wed Aug 14 21:55:40 2019 +0800

[Doc] Update Schema Guide (#4934)

1. Add descriptions.

2. Delete the following files because their contents are already in the new 
Schema Guide.


- concepts-schema-registry.md (contents are moved 
[here1](https://pulsar.apache.org/docs/en/next/schema-get-started/) and 
[here2](https://pulsar.apache.org/docs/en/next/schema-understand/#schema-version))
- developing-schema.md (contents are moved 
[here](https://pulsar.apache.org/docs/en/next/schema-manage/))
- admin-api-schemas.md (contents are moved 
[here](https://pulsar.apache.org/docs/en/next/schema-manage/))
---
 site2/docs/admin-api-schemas.md  | 98 
 site2/docs/concepts-schema-registry.md   | 81 ---
 site2/docs/developing-schema.md  | 57 
 site2/docs/schema-evolution-compatibility.md |  4 ++
 site2/docs/schema-get-started.md |  2 +
 site2/docs/schema-manage.md  |  2 +-
 site2/docs/schema-understand.md  | 11 ++--
 7 files changed, 13 insertions(+), 242 deletions(-)

diff --git a/site2/docs/admin-api-schemas.md b/site2/docs/admin-api-schemas.md
deleted file mode 100644
index f8cde5b..000
--- a/site2/docs/admin-api-schemas.md
+++ /dev/null
@@ -1,98 +0,0 @@

-id: admin-api-schemas
-title: Managing Schemas
-sidebar_label: Schemas

-
-Schemas, like other entities in Pulsar, can be managed using the [admin 
API](admin-api-overview.md). 
-
-## Schema resources
-
-A Pulsar schema is a fairly simple data structure stored in Pulsar for 
representing the structure of messages stored in a Pulsar topic. The schema 
structure consists of:
-
-- *Name*: A schema's name is the topic that the schema is associated to.
-- *Type*: A schema type represents the type of the schema. The predefined 
schema types can be found 
[here](concepts-schema-registry.md#supported-schema-formats). If it 
-  is a customized schema, it is left as an empty string.
-- *Payload*: It is a binary representation of the schema. How to interpret it 
is up to the implementation of the schema.
-- *Properties*: It is a user defined properties as a string/string map. 
Applications can use this bag for carrying any application specific logics. 
Possible properties
-  might be the Git hash associated with the schema, an environment string like 
`dev` or `prod`, etc.
-
-All the schemas are versioned with versions. So you can retrieve the schema 
definition of a given version if the version is not deleted.
-
-### Upload Schema
-
- pulsar-admin
-
-You can upload a new schema using the 
[`upload`](reference-pulsar-admin.md#get-5) subcommand:
-
-```shell
-$ pulsar-admin schemas upload  --filename 
/path/to/schema-definition-file 
-```
-
-The schema definition file should contain following json string on defining 
how the schema look like:
-
-```json
-{
-"type": "STRING",
-"schema": "",
-"properties": {
-"key1" : "value1"
-}
-}
-```
-
-An example of the schema definition file can be found at {@inject: 
github:SchemaExample:/conf/schema_example.conf}.
-
- REST
-
-{@inject: 
endpoint|POST|/admin/v2/schemas/:tenant/:namespace/:topic/schema|operation/uploadSchema}
-
-### Get Schema
-
- pulsar-admin
-
-You can get the latest version of Schema using the 
[`get`](reference-pulsar-admin.md#get-5) subcommand.
-
-```shell
-$ pulsar-admin schemas get 
-{
-"version": 0,
-"type": "String",
-"timestamp": 0,
-"data": "string",
-"properties": {
-"property1": "string",
-"property2": "string"
-}
-}
-```
-
-You can also retrieve the Schema of a given version by specifying `--version` 
option.
-
-```shell
-$ pulsar-admin schemas get  --version 
-```
-
- REST API
-
-Retrieve the latest version of the schema:
-
-{@inject: 
endpoint|GET|/admin/v2/schemas/:tenant/:namespace/:topic/schema|operation/getSchema}
-
-Retrieve the schema of a given version:
-
-{@inject: 
endpoint|GET|/admin/v2/schemas/:tenant/:namespace/:topic/schema/:version|operation/getSchema}
-
-### Delete Schema
-
- pulsar-admin
-
-You can delete a schema using the 
[`delete`](reference-pulsar-admin.md#delete-8) subcommand.
-
-```shell
-$ pulsar-admin schemas delete 
-```
-
- REST API
-
-{@inject: 
endpoint|DELETE|/admin/v2/schemas/:tenant/:namespace/:topic/schema|operation/deleteSchema}
diff --git a/site2/docs/concepts-schema-registry.md 
b/site2/docs/concepts-schema-registry.md
deleted file mode 100644
index 838ff4c..000

[GitHub] [pulsar] sijie merged pull request #4934: [Doc] Update Schema Guide

2019-08-14 Thread GitBox

sijie merged pull request #4934: [Doc] Update Schema Guide
URL: https://github.com/apache/pulsar/pull/4934
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on issue #4938: [Doc] Add Notes for Create Topic

2019-08-14 Thread GitBox

sijie commented on issue #4938: [Doc] Add Notes for *Create Topic*
URL: https://github.com/apache/pulsar/pull/4938#issuecomment-521257867
 
 
   run cpp tests
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet commented on issue #4938: [Doc] Add Notes for Create Topic

2019-08-14 Thread GitBox

Anonymitaet commented on issue #4938: [Doc] Add Notes for *Create Topic*
URL: https://github.com/apache/pulsar/pull/4938#issuecomment-521257897
 
 
   run c++ tests
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet removed a comment on issue #4938: [Doc] Add Notes for Create Topic

2019-08-14 Thread GitBox

Anonymitaet removed a comment on issue #4938: [Doc] Add Notes for *Create Topic*
URL: https://github.com/apache/pulsar/pull/4938#issuecomment-521257897
 
 
   run c++ tests
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie merged pull request #4818: [pulsar-broker] Fix expiry monitor to continue on non-recoverable error

2019-08-14 Thread GitBox

sijie merged pull request #4818: [pulsar-broker] Fix expiry monitor to continue 
on non-recoverable error
URL: https://github.com/apache/pulsar/pull/4818
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch master updated: [pulsar-broker] Fix expiry monitor to continue on non-recoverable error (#4818)

2019-08-14 Thread sijie

This is an automated email from the ASF dual-hosted git repository.

sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new c5ba529  [pulsar-broker] Fix expiry monitor to continue on 
non-recoverable error (#4818)
c5ba529 is described below

commit c5ba52983fee994de61984aae7d1757e9b738caf
Author: Rajan Dhabalia 
AuthorDate: Wed Aug 14 07:04:20 2019 -0700

[pulsar-broker] Fix expiry monitor to continue on non-recoverable error 
(#4818)

### Motivation

In #1046, we have added a flag (`autoSkipNonRecoverableData`) and mechanism 
to recover cursor if ledger data is deleted. However, expiery-monitor doesn't 
use that flag and it gets stuck when it finds non-recoverable ml-error while 
cleaning up expired message.

### Modification
Expiry-monitor can skip non-recoverable managed-ledger exception (eg: 
data/ledger doesn't exist anymore) when `autoSkipNonRecoverableData` flag is 
enabled.
---
 .../apache/bookkeeper/mledger/AsyncCallbacks.java  |  3 +-
 .../apache/bookkeeper/mledger/ManagedCursor.java   |  7 ++
 .../bookkeeper/mledger/impl/ManagedCursorImpl.java | 15 +++-
 .../bookkeeper/mledger/impl/ManagedLedgerImpl.java |  4 +
 .../bookkeeper/mledger/impl/OpFindNewest.java  |  5 +-
 .../mledger/impl/ManagedCursorContainerTest.java   |  7 ++
 .../bookkeeper/mledger/impl/ManagedCursorTest.java |  4 +-
 .../persistent/PersistentMessageExpiryMonitor.java | 14 +++-
 .../persistent/PersistentMessageFinder.java|  7 +-
 .../service/persistent/PersistentSubscription.java |  3 +-
 .../service/PersistentMessageFinderTest.java   | 89 +++---
 11 files changed, 136 insertions(+), 22 deletions(-)

diff --git 
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/AsyncCallbacks.java
 
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/AsyncCallbacks.java
index e861fad..8a21385 100644
--- 
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/AsyncCallbacks.java
+++ 
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/AsyncCallbacks.java
@@ -20,6 +20,7 @@ package org.apache.bookkeeper.mledger;
 
 import com.google.common.annotations.Beta;
 import java.util.List;
+import java.util.Optional;
 
 /**
  * Definition of all the callbacks used for the ManagedLedger asynchronous API.
@@ -116,7 +117,7 @@ public interface AsyncCallbacks {
 interface FindEntryCallback {
 void findEntryComplete(Position position, Object ctx);
 
-void findEntryFailed(ManagedLedgerException exception, Object ctx);
+void findEntryFailed(ManagedLedgerException exception, 
Optional failedReadPosition, Object ctx);
 }
 
 interface ResetCursorCallback {
diff --git 
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedCursor.java 
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedCursor.java
index fc3f6a4..03a68ba 100644
--- 
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedCursor.java
+++ 
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedCursor.java
@@ -587,4 +587,11 @@ public interface ManagedCursor {
  */
 void setThrottleMarkDelete(double throttleMarkDelete);
 
+/**
+ * Get {@link ManagedLedger} attached with cursor
+ * 
+ * @return ManagedLedger
+ */
+ManagedLedger getManagedLedger();
+
 }
diff --git 
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java
 
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java
index d185274..5292688 100644
--- 
a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java
+++ 
b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java
@@ -45,6 +45,7 @@ import java.util.ArrayDeque;
 import java.util.Collections;
 import java.util.List;
 import java.util.Map;
+import java.util.Optional;
 import java.util.Set;
 import java.util.concurrent.CountDownLatch;
 import java.util.concurrent.TimeUnit;
@@ -71,6 +72,7 @@ import 
org.apache.bookkeeper.mledger.AsyncCallbacks.ReadEntryCallback;
 import org.apache.bookkeeper.mledger.AsyncCallbacks.SkipEntriesCallback;
 import org.apache.bookkeeper.mledger.Entry;
 import org.apache.bookkeeper.mledger.ManagedCursor;
+import org.apache.bookkeeper.mledger.ManagedLedger;
 import org.apache.bookkeeper.mledger.ManagedLedgerConfig;
 import org.apache.bookkeeper.mledger.ManagedLedgerException;
 import 
org.apache.bookkeeper.mledger.ManagedLedgerException.CursorAlreadyClosedException;
@@ -766,7 +768,8 @@ public class ManagedCursorImpl implements ManagedCursor {
 }
 
 @Override
-public void findEntryFailed(ManagedLedgerException exception, 
Object ctx) {
+public void findEntryFailed(ManagedLedgerException exception, 
Optional failedReadPosition,
+Object ctx)

[GitHub] [pulsar] sijie commented on issue #4946: [pulsar-functions][python-examples] Fixing python function example custom_object_function.py

2019-08-14 Thread GitBox

sijie commented on issue #4946: [pulsar-functions][python-examples] Fixing 
python function example custom_object_function.py
URL: https://github.com/apache/pulsar/pull/4946#issuecomment-521314329
 
 
   @marcusalmeida cool


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar-translation] branch master updated (a08f6aa -> ae5064b)

2019-08-14 Thread sijie

This is an automated email from the ASF dual-hosted git repository.

sijie pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar-translation.git.


from a08f6aa  Merge pull request #2 from yangleimiao/master
 add 57bbd89  Create LICENSE
 new ae5064b  Merge pull request #3 from apache/add-license-1

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 LICENSE | 201 
 1 file changed, 201 insertions(+)
 create mode 100644 LICENSE

[pulsar-translation] 01/01: Merge pull request #3 from apache/add-license-1

2019-08-14 Thread sijie

This is an automated email from the ASF dual-hosted git repository.

sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar-translation.git

commit ae5064bda9745d452da27ffbdd3194258d2c3005
Merge: a08f6aa 57bbd89
Author: Sijie Guo 
AuthorDate: Thu Aug 15 00:16:47 2019 +0800

Merge pull request #3 from apache/add-license-1

### Motivation
Add a license to a repository.

### Modifications
Add Apache License 2.0.

 LICENSE | 201 
 1 file changed, 201 insertions(+)

[GitHub] [pulsar-translation] sijie merged pull request #3: Create LICENSE

2019-08-14 Thread GitBox

sijie merged pull request #3: Create LICENSE
URL: https://github.com/apache/pulsar-translation/pull/3
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar-translation] branch add-license-1 deleted (was 57bbd89)

2019-08-14 Thread sijie

This is an automated email from the ASF dual-hosted git repository.

sijie pushed a change to branch add-license-1
in repository https://gitbox.apache.org/repos/asf/pulsar-translation.git.


 was 57bbd89  Create LICENSE

The revisions that were on this branch are still contained in
other references; therefore, this change does not discard any commits
from the repository.

[GitHub] [pulsar-translation] sijie commented on issue #3: Create LICENSE

2019-08-14 Thread GitBox

sijie commented on issue #3: Create LICENSE
URL: https://github.com/apache/pulsar-translation/pull/3#issuecomment-521315240
 
 
   @Jennifer88huang For future pull requests, can you try to fork the repo and 
send the pull requests from your fork?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar-translation] branch add-license-1 created (now 57bbd89)

2019-08-14 Thread hjf

This is an automated email from the ASF dual-hosted git repository.

hjf pushed a change to branch add-license-1
in repository https://gitbox.apache.org/repos/asf/pulsar-translation.git.


  at 57bbd89  Create LICENSE

This branch includes the following new commits:

 new 57bbd89  Create LICENSE

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.

[pulsar-translation] 01/01: Create LICENSE

2019-08-14 Thread hjf

This is an automated email from the ASF dual-hosted git repository.

hjf pushed a commit to branch add-license-1
in repository https://gitbox.apache.org/repos/asf/pulsar-translation.git

commit 57bbd89bc5fbe619170d791ea3a944995b72fba5
Author: Jennifer Huang <47805623+jennifer88hu...@users.noreply.github.com>
AuthorDate: Wed Aug 14 22:25:13 2019 +0800

Create LICENSE
---
 LICENSE | 201 
 1 file changed, 201 insertions(+)

diff --git a/LICENSE b/LICENSE
new file mode 100644
index 000..261eeb9
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,201 @@
+ Apache License
+   Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+  "License" shall mean the terms and conditions for use, reproduction,
+  and distribution as defined by Sections 1 through 9 of this document.
+
+  "Licensor" shall mean the copyright owner or entity authorized by
+  the copyright owner that is granting the License.
+
+  "Legal Entity" shall mean the union of the acting entity and all
+  other entities that control, are controlled by, or are under common
+  control with that entity. For the purposes of this definition,
+  "control" means (i) the power, direct or indirect, to cause the
+  direction or management of such entity, whether by contract or
+  otherwise, or (ii) ownership of fifty percent (50%) or more of the
+  outstanding shares, or (iii) beneficial ownership of such entity.
+
+  "You" (or "Your") shall mean an individual or Legal Entity
+  exercising permissions granted by this License.
+
+  "Source" form shall mean the preferred form for making modifications,
+  including but not limited to software source code, documentation
+  source, and configuration files.
+
+  "Object" form shall mean any form resulting from mechanical
+  transformation or translation of a Source form, including but
+  not limited to compiled object code, generated documentation,
+  and conversions to other media types.
+
+  "Work" shall mean the work of authorship, whether in Source or
+  Object form, made available under the License, as indicated by a
+  copyright notice that is included in or attached to the work
+  (an example is provided in the Appendix below).
+
+  "Derivative Works" shall mean any work, whether in Source or Object
+  form, that is based on (or derived from) the Work and for which the
+  editorial revisions, annotations, elaborations, or other modifications
+  represent, as a whole, an original work of authorship. For the purposes
+  of this License, Derivative Works shall not include works that remain
+  separable from, or merely link (or bind by name) to the interfaces of,
+  the Work and Derivative Works thereof.
+
+  "Contribution" shall mean any work of authorship, including
+  the original version of the Work and any modifications or additions
+  to that Work or Derivative Works thereof, that is intentionally
+  submitted to Licensor for inclusion in the Work by the copyright owner
+  or by an individual or Legal Entity authorized to submit on behalf of
+  the copyright owner. For the purposes of this definition, "submitted"
+  means any form of electronic, verbal, or written communication sent
+  to the Licensor or its representatives, including but not limited to
+  communication on electronic mailing lists, source code control systems,
+  and issue tracking systems that are managed by, or on behalf of, the
+  Licensor for the purpose of discussing and improving the Work, but
+  excluding communication that is conspicuously marked or otherwise
+  designated in writing by the copyright owner as "Not a Contribution."
+
+  "Contributor" shall mean Licensor and any individual or Legal Entity
+  on behalf of whom a Contribution has been received by Licensor and
+  subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+  this License, each Contributor hereby grants to You a perpetual,
+  worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+  copyright license to reproduce, prepare Derivative Works of,
+  publicly display, publicly perform, sublicense, and distribute the
+  Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+  this License, each Contributor hereby grants to You a perpetual,
+  worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+  (except as stated in this section) patent license to make, have made,
+  use, offer to sell, sell, import, and otherwise transfer the Work,
+  where such license

[GitHub] [pulsar-translation] Jennifer88huang opened a new pull request #3: Create LICENSE

2019-08-14 Thread GitBox

Jennifer88huang opened a new pull request #3: Create LICENSE
URL: https://github.com/apache/pulsar-translation/pull/3
 
 
   ### Motivation
   Add a license to a repository.
   
   ### Modifications
   Add Apache License 2.0.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar-translation] branch master updated: update personal address

2019-08-14 Thread hjf

This is an automated email from the ASF dual-hosted git repository.

hjf pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar-translation.git


The following commit(s) were added to refs/heads/master by this push:
 new f904073  update personal address
 new a08f6aa  Merge pull request #2 from yangleimiao/master
f904073 is described below

commit f904073400c4820b3a79f3e45c769fe5c44e246b
Author: 杨磊 
AuthorDate: Wed Aug 14 14:27:09 2019 +0800

update personal address
---
 CONTRIBUTORS.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md
index 25800ea..e4cd728 100644
--- a/CONTRIBUTORS.md
+++ b/CONTRIBUTORS.md
@@ -4,7 +4,7 @@
 - Francis Du (https://github.com/francis-du)
 - Jennifer Huang (https://github.com/Jennifer88huang)
 - Jia Zhai (https://github.com/jiazhai)
-- Lei Yang ()
+- Lei Yang (https://github.com/yangleimiao)
 - Sijie Guo (https://github.com/sijie)
 - Susan (https://github.com/Ssusan)
 - Tianci Gao (https://github.com/bilahepan)

[GitHub] [pulsar-translation] Jennifer88huang merged pull request #2: update personal address

2019-08-14 Thread GitBox

Jennifer88huang merged pull request #2: update personal address
URL: https://github.com/apache/pulsar-translation/pull/2
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] marcusalmeida commented on issue #4946: [pulsar-functions][python-examples] Fixing python function example custom_object_function.py

2019-08-14 Thread GitBox

marcusalmeida commented on issue #4946: [pulsar-functions][python-examples] 
Fixing python function example custom_object_function.py
URL: https://github.com/apache/pulsar/pull/4946#issuecomment-521285889
 
 
   @sijie I tested using different Python versions (2.7.15+ and 3.6.8).
   
   I created two virtualenvs one for each version and ran the Pulsar in 
standalone mode in each virtualenv. And this code works.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie opened a new pull request #4952: [Transaction] Introduce Transaction Client API

2019-08-14 Thread GitBox

sijie opened a new pull request #4952: [Transaction] Introduce Transaction 
Client API
URL: https://github.com/apache/pulsar/pull/4952
 
 
   
   
   
   
   *Motivation*
   
   Introduce the transaction API to setup the skeleton on implementing
   the transactional client.
   
   Also add an example to show how to use the transaction API.
   
   *Modifications*
   
   - Add TransactionBuilder and Transaction
   - Add `newTransaction` to client
   - Add `newMessage(Transaction)` method to producer
   - Add `acknowledgeAsyc(MessageId, Transaction)` method to consumer
   
   NOTE: these methods are only exposed to the implementation classes 
(ProducerImpl, ProducerImpl, and ConsumerImpl).
   Will only add them to the interfaces when the features are completed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie opened a new pull request #4953: [transaction] Add transaction coordinator client

2019-08-14 Thread GitBox

sijie opened a new pull request #4953: [transaction] Add transaction 
coordinator client
URL: https://github.com/apache/pulsar/pull/4953
 
 
   
   
   
   
   *Motivation*
   
   Add the interface of transaction coordinator client. So that it can be used
   by coorindator for committing transactions on topics and subscriptions.
   
   NOTE: this PR is only introducing the interface. The implementation will come
   in subsequent pull requests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

88 matches

Mail list logo