[GitHub] [pulsar] tuteng commented on issue #5168: [Doc] Add Built-in Connector

[GitBox]

tuteng commented on issue #5168: [Doc] Add Built-in Connector
URL: https://github.com/apache/pulsar/pull/5168#issuecomment-530231100
 
 
   LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Monica-zy commented on issue #5164: [doc] Improve Pulsar Deployment monitoring

[GitBox]

Monica-zy commented on issue #5164: [doc] Improve Pulsar Deployment monitoring
URL: https://github.com/apache/pulsar/pull/5164#issuecomment-530226470
 
 
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jiazhai commented on issue #5148: Validate topic name before creating partition/non partition topic via admin cli.

[GitBox]

jiazhai commented on issue #5148: Validate topic name before creating 
partition/non partition topic via admin cli.
URL: https://github.com/apache/pulsar/pull/5148#issuecomment-530217754
 
 
   Good point for the checking. yes. we need the support for create 
"my-topic-name-partition-123"


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] liketic opened a new pull request #5174: Fix exception type check order bug

[GitBox]

liketic opened a new pull request #5174: Fix exception type check order bug
URL: https://github.com/apache/pulsar/pull/5174
 
 
   If ```!(t instanceof ExecutionException)``` is false, which means t is an 
instance of ``` ExecutionException```. If ```!(t instanceof 
ExecutionException)``` is true, then an instance of  ```PulsarClientException 
``` will be returned. So the last ```if``` block will never be executed. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on issue #5143: [Issue 5142][docs] Remove non-partitioned topic info from partitioned topic docs

[GitBox]

Jennifer88huang commented on issue #5143: [Issue 5142][docs] Remove 
non-partitioned topic info from partitioned topic docs
URL: https://github.com/apache/pulsar/pull/5143#issuecomment-530213686
 
 
   @rocketraman When we add a new topic, remember to add it also in the 
sidebars.json file.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on issue #5143: [Issue 5142][docs] Remove non-partitioned topic info from partitioned topic docs

[GitBox]

Jennifer88huang commented on issue #5143: [Issue 5142][docs] Remove 
non-partitioned topic info from partitioned topic docs
URL: https://github.com/apache/pulsar/pull/5143#issuecomment-530212561
 
 
   @zymap Could you please help check whether the commands and usage for the 
non-partitioned topics are right? Thank you very much.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] tuteng commented on issue #5153: [doc]Add brokerClient authentication configuration

[GitBox]

tuteng commented on issue #5153: [doc]Add brokerClient authentication 
configuration
URL: https://github.com/apache/pulsar/pull/5153#issuecomment-530212168
 
 
   @jiazhai PTAL


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang edited a comment on issue #5167: [doc] Improve Pulsar multi-cluster deployment

[GitBox]

Jennifer88huang edited a comment on issue #5167: [doc] Improve Pulsar 
multi-cluster deployment
URL: https://github.com/apache/pulsar/pull/5167#issuecomment-530183141
 
 
   @Monica-zy There are some conflicts in this PR with yours in #5135. Maybe we 
can deal with this one first, and then you can continue with your updates in 
#5135.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Monica-zy commented on issue #5050: [Doc] 'JSON Web Token Authentication' is not documented in the Security chapter of Pulsar document

[GitBox]

Monica-zy commented on issue #5050: [Doc] 'JSON Web Token Authentication' is 
not documented in the Security chapter of Pulsar document
URL: https://github.com/apache/pulsar/issues/5050#issuecomment-530208607
 
 
   Here is my suggested solution: Merge and rewrite the original two file 
security-token-client.md and security-token-admin.md into a new security-jwt.md 
which links to the list item "JSON Web Token Authentication" in the file 
security-overview.md; adjust the related content in security-overview and unify 
the sidebar title into a similar format as much as possible.
   Changes in security-overview.md:
   ![屏幕快照 2019-09-11 上午11 42 
00](https://user-images.githubusercontent.com/44013755/6413-7892b780-d489-11e9-9e63-5d6848aaec78.png)
   Changes in sidebar:
   ![屏幕快照 2019-09-11 上午11 46 
21](https://user-images.githubusercontent.com/44013755/64666764-ffe02b00-d489-11e9-8131-0079b2b6f58d.png)
   and welcome better suggestions and opinions to discuss here~
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet removed a comment on issue #5168: [Doc] Add Built-in Connector

[GitBox]

Anonymitaet removed a comment on issue #5168: [Doc] Add Built-in Connector
URL: https://github.com/apache/pulsar/pull/5168#issuecomment-529852717
 
 
   Fix https://github.com/apache/pulsar/issues/5015


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet edited a comment on issue #5015: [Doc] Update Pulsar Connector Guide

[GitBox]

Anonymitaet edited a comment on issue #5015: [Doc] Update Pulsar Connector Guide
URL: https://github.com/apache/pulsar/issues/5015#issuecomment-524149120
 
 
   PRs related to this issue:
   
   ## Use 
   * https://github.com/apache/pulsar/pull/5071
   
   ## Built-in connector 
   * https://github.com/apache/pulsar/pull/5168
   * https://github.com/apache/pulsar/pull/5173
   
   ## Debug 
   * https://github.com/apache/pulsar/pull/5041
   
   ## Connector Admin CLI
   * https://github.com/apache/pulsar/pull/5028


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet commented on issue #5168: [Doc] Add Built-in Connector

[GitBox]

Anonymitaet commented on issue #5168: [Doc] Add Built-in Connector
URL: https://github.com/apache/pulsar/pull/5168#issuecomment-530208324
 
 
   @murong00 
   
   as we discussed previously, could you please **provide examples for kafka, 
rabbitmq, redis, influxdb, and solr sink connectors**? 
   
   And your partners **provide examples for hadoop related sink connectors.**
   
   Thank you


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5173: [Doc] Update Canal source connector guide

[GitBox]

Anonymitaet commented on a change in pull request #5173: [Doc] Update Canal 
source connector guide
URL: https://github.com/apache/pulsar/pull/5173#discussion_r323047036
 
 

 ##
 File path: site2/docs/io-cdc-canal.md
 ##
 @@ -1,174 +1,200 @@
 ---
 id: io-cdc-canal
-title: CDC Canal Connector
-sidebar_label: CDC Canal Connector
+title: Canal source connector
+sidebar_label: Canal source connector
 ---
 
-### Source Configuration Options
+This guide explains the configuration and usage of Canal source connector.
 
-The Configuration is mostly related to Canal task config.
+## Configuration
+
+The configuration of Canal source connector has the following parameters.
+
+### Parameter
 
 | Name | Required | Default | Description |
 |--|--|-|-|
-| `zkServers` | `false` | `127.0.0.1:2181` | `The address and port of the 
zookeeper . if canal server configured to cluster mode` |
-| `batchSize` | `true` | `5120` | `Take 5120 records from the canal server in 
batches` |
-| `username` | `false` | `` | `Canal server account, not MySQL` |
-| `password` | `false` | `` | `Canal server password, not MySQL` |
-| `cluster` | `false` | `false` | `Decide whether to open cluster mode based 
on canal server configuration, true: cluster mode, false: standalone mode` |
-| `singleHostname` | `false` | `127.0.0.1` | `The address of canal server` |
-| `singlePort` | `false` | `1` | `The port of canal server` |
-
-
-### Configuration Example
-
-Here is a configuration Json example:
-
-```$json
-{
-"zkServers": "127.0.0.1:2181",
-"batchSize": "5120",
-"destination": "example",
-"username": "",
-"password": "",
-"cluster": false,
-"singleHostname": "127.0.0.1",
-"singlePort": "1",
-}
-```
-You could also find the yaml example in this 
[file](https://github.com/apache/pulsar/blob/master/pulsar-io/canal/src/main/resources/canal-mysql-source-config.yaml),
 which has similar content below:
-
-```$yaml
-configs:
-zkServers: "127.0.0.1:2181"
-batchSize: "5120"
-destination: "example"
-username: ""
-password: ""
-cluster: false
-singleHostname: "127.0.0.1"
-singlePort: "1"
-```
-
-### Usage example
-
-Here is a simple example to store MySQL change data using above example config.
-
-- Start a MySQL server
-
-```$bash
-docker pull mysql:5.7
-docker run -d -it --rm --name pulsar-mysql -p 3306:3306 -e 
MYSQL_ROOT_PASSWORD=canal -e MYSQL_USER=mysqluser -e MYSQL_PASSWORD=mysqlpw 
mysql:5.7
-```
-- Modify configuration files mysqld.cnf
-
-```
-[mysqld]
-pid-file= /var/run/mysqld/mysqld.pid
-socket  = /var/run/mysqld/mysqld.sock
-datadir = /var/lib/mysql
-#log-error  = /var/log/mysql/error.log
-# By default we only accept connections from localhost
-#bind-address   = 127.0.0.1
-# Disabling symbolic-links is recommended to prevent assorted security risks
-symbolic-links=0
-log-bin=mysql-bin
-binlog-format=ROW
-server_id=1
-```
-
-- Copy file to mysql server from local and restart mysql server
-```$bash
-docker cp mysqld.cnf pulsar-mysql:/etc/mysql/mysql.conf.d/
-docker restart pulsar-mysql
-```
-
-- Create test database in mysql server
-```$bash
-docker exec -it pulsar-mysql /bin/bash
-mysql -h 127.0.0.1 -uroot -pcanal -e 'create database test;'
-```
-
-- Start canal server and connect mysql server
-
-```
-docker pull canal/canal-server:v1.1.2
-docker run -d -it --link pulsar-mysql -e canal.auto.scan=false -e 
canal.destinations=test -e canal.instance.master.address=pulsar-mysql:3306 -e 
canal.instance.dbUsername=root -e canal.instance.dbPassword=canal -e 
canal.instance.connectionCharset=UTF-8 -e canal.instance.tsdb.enable=true -e 
canal.instance.gtidon=false --name=pulsar-canal-server -p 8000:8000 -p 
: -p 1:1 -p 2:2 -m 4096m canal/canal-server:v1.1.2
-```
-
-- Start pulsar standalone
-
-```$bash
-docker pull apachepulsar/pulsar:2.3.0
-docker run -d -it --link pulsar-canal-server -p 6650:6650 -p 8080:8080 -v 
$PWD/data:/pulsar/data --name pulsar-standalone apachepulsar/pulsar:2.3.0 
bin/pulsar standalone
-```
-
-- Start pulsar-io in standalone
-
-- Config file canal-mysql-source-config.yaml
-
-```$yaml
-configs:
-zkServers: ""
-batchSize: "5120"
-destination: "test"
-username: ""
-password: ""
-cluster: false
-singleHostname: "pulsar-canal-server"
-singlePort: "1"
-```
-- Consumer file pulsar-client.py for test
-```
-import pulsar
-
-client = pulsar.Client('pulsar://localhost:6650')
-consumer = client.subscribe('my-topic',
-subscription_name='my-sub')
-
-while True:
-msg = consumer.receive()
-print("Received message: '%s'" % msg.data())
-consumer.acknowledge(msg)
-
-client.close()
-```
-
-- Copy config file and test file to pulsar server
-
-```$bash
-docker cp canal-mysql-source-config.yaml pulsar-standalone:/pulsar/conf/
-docker cp pulsar-client.py pulsar-standalone:/pulsar/
-```
-
-- Download canal connector and 

[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5173: [Doc] Update Canal source connector guide

[GitBox]

Anonymitaet commented on a change in pull request #5173: [Doc] Update Canal 
source connector guide
URL: https://github.com/apache/pulsar/pull/5173#discussion_r323046232
 
 

 ##
 File path: site2/docs/io-cdc-canal.md
 ##
 @@ -1,174 +1,200 @@
 ---
 id: io-cdc-canal
-title: CDC Canal Connector
-sidebar_label: CDC Canal Connector
+title: Canal source connector
+sidebar_label: Canal source connector
 ---
 
-### Source Configuration Options
+This guide explains the configuration and usage of Canal source connector.
 
-The Configuration is mostly related to Canal task config.
+## Configuration
+
+The configuration of Canal source connector has the following parameters.
+
+### Parameter
 
 | Name | Required | Default | Description |
 |--|--|-|-|
-| `zkServers` | `false` | `127.0.0.1:2181` | `The address and port of the 
zookeeper . if canal server configured to cluster mode` |
-| `batchSize` | `true` | `5120` | `Take 5120 records from the canal server in 
batches` |
-| `username` | `false` | `` | `Canal server account, not MySQL` |
-| `password` | `false` | `` | `Canal server password, not MySQL` |
-| `cluster` | `false` | `false` | `Decide whether to open cluster mode based 
on canal server configuration, true: cluster mode, false: standalone mode` |
-| `singleHostname` | `false` | `127.0.0.1` | `The address of canal server` |
-| `singlePort` | `false` | `1` | `The port of canal server` |
-
-
-### Configuration Example
-
-Here is a configuration Json example:
-
-```$json
-{
-"zkServers": "127.0.0.1:2181",
-"batchSize": "5120",
-"destination": "example",
-"username": "",
-"password": "",
-"cluster": false,
-"singleHostname": "127.0.0.1",
-"singlePort": "1",
-}
-```
-You could also find the yaml example in this 
[file](https://github.com/apache/pulsar/blob/master/pulsar-io/canal/src/main/resources/canal-mysql-source-config.yaml),
 which has similar content below:
-
-```$yaml
-configs:
-zkServers: "127.0.0.1:2181"
-batchSize: "5120"
-destination: "example"
-username: ""
-password: ""
-cluster: false
-singleHostname: "127.0.0.1"
-singlePort: "1"
-```
-
-### Usage example
-
-Here is a simple example to store MySQL change data using above example config.
-
-- Start a MySQL server
-
-```$bash
-docker pull mysql:5.7
-docker run -d -it --rm --name pulsar-mysql -p 3306:3306 -e 
MYSQL_ROOT_PASSWORD=canal -e MYSQL_USER=mysqluser -e MYSQL_PASSWORD=mysqlpw 
mysql:5.7
-```
-- Modify configuration files mysqld.cnf
-
-```
-[mysqld]
-pid-file= /var/run/mysqld/mysqld.pid
-socket  = /var/run/mysqld/mysqld.sock
-datadir = /var/lib/mysql
-#log-error  = /var/log/mysql/error.log
-# By default we only accept connections from localhost
-#bind-address   = 127.0.0.1
-# Disabling symbolic-links is recommended to prevent assorted security risks
-symbolic-links=0
-log-bin=mysql-bin
-binlog-format=ROW
-server_id=1
-```
-
-- Copy file to mysql server from local and restart mysql server
-```$bash
-docker cp mysqld.cnf pulsar-mysql:/etc/mysql/mysql.conf.d/
-docker restart pulsar-mysql
-```
-
-- Create test database in mysql server
-```$bash
-docker exec -it pulsar-mysql /bin/bash
-mysql -h 127.0.0.1 -uroot -pcanal -e 'create database test;'
-```
-
-- Start canal server and connect mysql server
-
-```
-docker pull canal/canal-server:v1.1.2
-docker run -d -it --link pulsar-mysql -e canal.auto.scan=false -e 
canal.destinations=test -e canal.instance.master.address=pulsar-mysql:3306 -e 
canal.instance.dbUsername=root -e canal.instance.dbPassword=canal -e 
canal.instance.connectionCharset=UTF-8 -e canal.instance.tsdb.enable=true -e 
canal.instance.gtidon=false --name=pulsar-canal-server -p 8000:8000 -p 
: -p 1:1 -p 2:2 -m 4096m canal/canal-server:v1.1.2
-```
-
-- Start pulsar standalone
-
-```$bash
-docker pull apachepulsar/pulsar:2.3.0
-docker run -d -it --link pulsar-canal-server -p 6650:6650 -p 8080:8080 -v 
$PWD/data:/pulsar/data --name pulsar-standalone apachepulsar/pulsar:2.3.0 
bin/pulsar standalone
-```
-
-- Start pulsar-io in standalone
-
-- Config file canal-mysql-source-config.yaml
-
-```$yaml
-configs:
-zkServers: ""
-batchSize: "5120"
-destination: "test"
-username: ""
-password: ""
-cluster: false
-singleHostname: "pulsar-canal-server"
-singlePort: "1"
-```
-- Consumer file pulsar-client.py for test
-```
-import pulsar
-
-client = pulsar.Client('pulsar://localhost:6650')
-consumer = client.subscribe('my-topic',
-subscription_name='my-sub')
-
-while True:
-msg = consumer.receive()
-print("Received message: '%s'" % msg.data())
-consumer.acknowledge(msg)
-
-client.close()
-```
-
-- Copy config file and test file to pulsar server
-
-```$bash
-docker cp canal-mysql-source-config.yaml pulsar-standalone:/pulsar/conf/
-docker cp pulsar-client.py pulsar-standalone:/pulsar/
-```
-
-- Download canal connector and 

[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5173: [Doc] Update Canal source connector guide

[GitBox]

Anonymitaet commented on a change in pull request #5173: [Doc] Update Canal 
source connector guide
URL: https://github.com/apache/pulsar/pull/5173#discussion_r323046401
 
 

 ##
 File path: site2/docs/io-cdc-canal.md
 ##
 @@ -1,174 +1,200 @@
 ---
 id: io-cdc-canal
-title: CDC Canal Connector
-sidebar_label: CDC Canal Connector
+title: Canal source connector
+sidebar_label: Canal source connector
 ---
 
-### Source Configuration Options
+This guide explains the configuration and usage of Canal source connector.
 
-The Configuration is mostly related to Canal task config.
+## Configuration
+
+The configuration of Canal source connector has the following parameters.
+
+### Parameter
 
 | Name | Required | Default | Description |
 |--|--|-|-|
-| `zkServers` | `false` | `127.0.0.1:2181` | `The address and port of the 
zookeeper . if canal server configured to cluster mode` |
-| `batchSize` | `true` | `5120` | `Take 5120 records from the canal server in 
batches` |
-| `username` | `false` | `` | `Canal server account, not MySQL` |
-| `password` | `false` | `` | `Canal server password, not MySQL` |
-| `cluster` | `false` | `false` | `Decide whether to open cluster mode based 
on canal server configuration, true: cluster mode, false: standalone mode` |
-| `singleHostname` | `false` | `127.0.0.1` | `The address of canal server` |
-| `singlePort` | `false` | `1` | `The port of canal server` |
-
-
-### Configuration Example
-
-Here is a configuration Json example:
-
-```$json
-{
-"zkServers": "127.0.0.1:2181",
-"batchSize": "5120",
-"destination": "example",
-"username": "",
-"password": "",
-"cluster": false,
-"singleHostname": "127.0.0.1",
-"singlePort": "1",
-}
-```
-You could also find the yaml example in this 
[file](https://github.com/apache/pulsar/blob/master/pulsar-io/canal/src/main/resources/canal-mysql-source-config.yaml),
 which has similar content below:
-
-```$yaml
-configs:
-zkServers: "127.0.0.1:2181"
-batchSize: "5120"
-destination: "example"
-username: ""
-password: ""
-cluster: false
-singleHostname: "127.0.0.1"
-singlePort: "1"
-```
-
-### Usage example
-
-Here is a simple example to store MySQL change data using above example config.
-
-- Start a MySQL server
-
-```$bash
-docker pull mysql:5.7
-docker run -d -it --rm --name pulsar-mysql -p 3306:3306 -e 
MYSQL_ROOT_PASSWORD=canal -e MYSQL_USER=mysqluser -e MYSQL_PASSWORD=mysqlpw 
mysql:5.7
-```
-- Modify configuration files mysqld.cnf
-
-```
-[mysqld]
-pid-file= /var/run/mysqld/mysqld.pid
-socket  = /var/run/mysqld/mysqld.sock
-datadir = /var/lib/mysql
-#log-error  = /var/log/mysql/error.log
-# By default we only accept connections from localhost
-#bind-address   = 127.0.0.1
-# Disabling symbolic-links is recommended to prevent assorted security risks
-symbolic-links=0
-log-bin=mysql-bin
-binlog-format=ROW
-server_id=1
-```
-
-- Copy file to mysql server from local and restart mysql server
-```$bash
-docker cp mysqld.cnf pulsar-mysql:/etc/mysql/mysql.conf.d/
-docker restart pulsar-mysql
-```
-
-- Create test database in mysql server
-```$bash
-docker exec -it pulsar-mysql /bin/bash
-mysql -h 127.0.0.1 -uroot -pcanal -e 'create database test;'
-```
-
-- Start canal server and connect mysql server
-
-```
-docker pull canal/canal-server:v1.1.2
-docker run -d -it --link pulsar-mysql -e canal.auto.scan=false -e 
canal.destinations=test -e canal.instance.master.address=pulsar-mysql:3306 -e 
canal.instance.dbUsername=root -e canal.instance.dbPassword=canal -e 
canal.instance.connectionCharset=UTF-8 -e canal.instance.tsdb.enable=true -e 
canal.instance.gtidon=false --name=pulsar-canal-server -p 8000:8000 -p 
: -p 1:1 -p 2:2 -m 4096m canal/canal-server:v1.1.2
-```
-
-- Start pulsar standalone
-
-```$bash
-docker pull apachepulsar/pulsar:2.3.0
-docker run -d -it --link pulsar-canal-server -p 6650:6650 -p 8080:8080 -v 
$PWD/data:/pulsar/data --name pulsar-standalone apachepulsar/pulsar:2.3.0 
bin/pulsar standalone
-```
-
-- Start pulsar-io in standalone
-
-- Config file canal-mysql-source-config.yaml
-
-```$yaml
-configs:
-zkServers: ""
-batchSize: "5120"
-destination: "test"
-username: ""
-password: ""
-cluster: false
-singleHostname: "pulsar-canal-server"
-singlePort: "1"
-```
-- Consumer file pulsar-client.py for test
-```
-import pulsar
-
-client = pulsar.Client('pulsar://localhost:6650')
-consumer = client.subscribe('my-topic',
-subscription_name='my-sub')
-
-while True:
-msg = consumer.receive()
-print("Received message: '%s'" % msg.data())
-consumer.acknowledge(msg)
-
-client.close()
-```
-
-- Copy config file and test file to pulsar server
-
-```$bash
-docker cp canal-mysql-source-config.yaml pulsar-standalone:/pulsar/conf/
-docker cp pulsar-client.py pulsar-standalone:/pulsar/
-```
-
-- Download canal connector and 

[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5173: [Doc] Update Canal source connector guide

[GitBox]

Anonymitaet commented on a change in pull request #5173: [Doc] Update Canal 
source connector guide
URL: https://github.com/apache/pulsar/pull/5173#discussion_r323045408
 
 

 ##
 File path: site2/docs/io-cdc-canal.md
 ##
 @@ -1,174 +1,200 @@
 ---
 id: io-cdc-canal
-title: CDC Canal Connector
-sidebar_label: CDC Canal Connector
+title: Canal source connector
+sidebar_label: Canal source connector
 ---
 
-### Source Configuration Options
+This guide explains the configuration and usage of Canal source connector.
 
-The Configuration is mostly related to Canal task config.
+## Configuration
+
+The configuration of Canal source connector has the following parameters.
+
+### Parameter
 
 | Name | Required | Default | Description |
 |--|--|-|-|
-| `zkServers` | `false` | `127.0.0.1:2181` | `The address and port of the 
zookeeper . if canal server configured to cluster mode` |
-| `batchSize` | `true` | `5120` | `Take 5120 records from the canal server in 
batches` |
-| `username` | `false` | `` | `Canal server account, not MySQL` |
-| `password` | `false` | `` | `Canal server password, not MySQL` |
-| `cluster` | `false` | `false` | `Decide whether to open cluster mode based 
on canal server configuration, true: cluster mode, false: standalone mode` |
-| `singleHostname` | `false` | `127.0.0.1` | `The address of canal server` |
-| `singlePort` | `false` | `1` | `The port of canal server` |
-
-
-### Configuration Example
-
-Here is a configuration Json example:
-
-```$json
-{
-"zkServers": "127.0.0.1:2181",
-"batchSize": "5120",
-"destination": "example",
-"username": "",
-"password": "",
-"cluster": false,
-"singleHostname": "127.0.0.1",
-"singlePort": "1",
-}
-```
-You could also find the yaml example in this 
[file](https://github.com/apache/pulsar/blob/master/pulsar-io/canal/src/main/resources/canal-mysql-source-config.yaml),
 which has similar content below:
-
-```$yaml
-configs:
-zkServers: "127.0.0.1:2181"
-batchSize: "5120"
-destination: "example"
-username: ""
-password: ""
-cluster: false
-singleHostname: "127.0.0.1"
-singlePort: "1"
-```
-
-### Usage example
-
-Here is a simple example to store MySQL change data using above example config.
-
-- Start a MySQL server
-
-```$bash
-docker pull mysql:5.7
-docker run -d -it --rm --name pulsar-mysql -p 3306:3306 -e 
MYSQL_ROOT_PASSWORD=canal -e MYSQL_USER=mysqluser -e MYSQL_PASSWORD=mysqlpw 
mysql:5.7
-```
-- Modify configuration files mysqld.cnf
-
-```
-[mysqld]
-pid-file= /var/run/mysqld/mysqld.pid
-socket  = /var/run/mysqld/mysqld.sock
-datadir = /var/lib/mysql
-#log-error  = /var/log/mysql/error.log
-# By default we only accept connections from localhost
-#bind-address   = 127.0.0.1
-# Disabling symbolic-links is recommended to prevent assorted security risks
-symbolic-links=0
-log-bin=mysql-bin
-binlog-format=ROW
-server_id=1
-```
-
-- Copy file to mysql server from local and restart mysql server
-```$bash
-docker cp mysqld.cnf pulsar-mysql:/etc/mysql/mysql.conf.d/
-docker restart pulsar-mysql
-```
-
-- Create test database in mysql server
-```$bash
-docker exec -it pulsar-mysql /bin/bash
-mysql -h 127.0.0.1 -uroot -pcanal -e 'create database test;'
-```
-
-- Start canal server and connect mysql server
-
-```
-docker pull canal/canal-server:v1.1.2
-docker run -d -it --link pulsar-mysql -e canal.auto.scan=false -e 
canal.destinations=test -e canal.instance.master.address=pulsar-mysql:3306 -e 
canal.instance.dbUsername=root -e canal.instance.dbPassword=canal -e 
canal.instance.connectionCharset=UTF-8 -e canal.instance.tsdb.enable=true -e 
canal.instance.gtidon=false --name=pulsar-canal-server -p 8000:8000 -p 
: -p 1:1 -p 2:2 -m 4096m canal/canal-server:v1.1.2
-```
-
-- Start pulsar standalone
-
-```$bash
-docker pull apachepulsar/pulsar:2.3.0
-docker run -d -it --link pulsar-canal-server -p 6650:6650 -p 8080:8080 -v 
$PWD/data:/pulsar/data --name pulsar-standalone apachepulsar/pulsar:2.3.0 
bin/pulsar standalone
-```
-
-- Start pulsar-io in standalone
-
-- Config file canal-mysql-source-config.yaml
-
-```$yaml
-configs:
-zkServers: ""
-batchSize: "5120"
-destination: "test"
-username: ""
-password: ""
-cluster: false
-singleHostname: "pulsar-canal-server"
-singlePort: "1"
-```
-- Consumer file pulsar-client.py for test
-```
-import pulsar
-
-client = pulsar.Client('pulsar://localhost:6650')
-consumer = client.subscribe('my-topic',
-subscription_name='my-sub')
-
-while True:
-msg = consumer.receive()
-print("Received message: '%s'" % msg.data())
-consumer.acknowledge(msg)
-
-client.close()
-```
-
-- Copy config file and test file to pulsar server
-
-```$bash
-docker cp canal-mysql-source-config.yaml pulsar-standalone:/pulsar/conf/
-docker cp pulsar-client.py pulsar-standalone:/pulsar/
-```
-
-- Download canal connector and 

[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5173: [Doc] Update Canal source connector guide

[GitBox]

Anonymitaet commented on a change in pull request #5173: [Doc] Update Canal 
source connector guide
URL: https://github.com/apache/pulsar/pull/5173#discussion_r323045041
 
 

 ##
 File path: site2/docs/io-cdc-canal.md
 ##
 @@ -1,174 +1,200 @@
 ---
 id: io-cdc-canal
-title: CDC Canal Connector
-sidebar_label: CDC Canal Connector
+title: Canal source connector
+sidebar_label: Canal source connector
 ---
 
-### Source Configuration Options
+This guide explains the configuration and usage of Canal source connector.
 
-The Configuration is mostly related to Canal task config.
+## Configuration
+
+The configuration of Canal source connector has the following parameters.
+
+### Parameter
 
 | Name | Required | Default | Description |
 |--|--|-|-|
-| `zkServers` | `false` | `127.0.0.1:2181` | `The address and port of the 
zookeeper . if canal server configured to cluster mode` |
-| `batchSize` | `true` | `5120` | `Take 5120 records from the canal server in 
batches` |
-| `username` | `false` | `` | `Canal server account, not MySQL` |
-| `password` | `false` | `` | `Canal server password, not MySQL` |
-| `cluster` | `false` | `false` | `Decide whether to open cluster mode based 
on canal server configuration, true: cluster mode, false: standalone mode` |
-| `singleHostname` | `false` | `127.0.0.1` | `The address of canal server` |
-| `singlePort` | `false` | `1` | `The port of canal server` |
-
-
-### Configuration Example
-
-Here is a configuration Json example:
-
-```$json
-{
-"zkServers": "127.0.0.1:2181",
-"batchSize": "5120",
-"destination": "example",
-"username": "",
-"password": "",
-"cluster": false,
-"singleHostname": "127.0.0.1",
-"singlePort": "1",
-}
-```
-You could also find the yaml example in this 
[file](https://github.com/apache/pulsar/blob/master/pulsar-io/canal/src/main/resources/canal-mysql-source-config.yaml),
 which has similar content below:
-
-```$yaml
-configs:
-zkServers: "127.0.0.1:2181"
-batchSize: "5120"
-destination: "example"
-username: ""
-password: ""
-cluster: false
-singleHostname: "127.0.0.1"
-singlePort: "1"
-```
-
-### Usage example
-
-Here is a simple example to store MySQL change data using above example config.
-
-- Start a MySQL server
-
-```$bash
-docker pull mysql:5.7
-docker run -d -it --rm --name pulsar-mysql -p 3306:3306 -e 
MYSQL_ROOT_PASSWORD=canal -e MYSQL_USER=mysqluser -e MYSQL_PASSWORD=mysqlpw 
mysql:5.7
-```
-- Modify configuration files mysqld.cnf
-
-```
-[mysqld]
-pid-file= /var/run/mysqld/mysqld.pid
-socket  = /var/run/mysqld/mysqld.sock
-datadir = /var/lib/mysql
-#log-error  = /var/log/mysql/error.log
-# By default we only accept connections from localhost
-#bind-address   = 127.0.0.1
-# Disabling symbolic-links is recommended to prevent assorted security risks
-symbolic-links=0
-log-bin=mysql-bin
-binlog-format=ROW
-server_id=1
-```
-
-- Copy file to mysql server from local and restart mysql server
-```$bash
-docker cp mysqld.cnf pulsar-mysql:/etc/mysql/mysql.conf.d/
-docker restart pulsar-mysql
-```
-
-- Create test database in mysql server
-```$bash
-docker exec -it pulsar-mysql /bin/bash
-mysql -h 127.0.0.1 -uroot -pcanal -e 'create database test;'
-```
-
-- Start canal server and connect mysql server
-
-```
-docker pull canal/canal-server:v1.1.2
-docker run -d -it --link pulsar-mysql -e canal.auto.scan=false -e 
canal.destinations=test -e canal.instance.master.address=pulsar-mysql:3306 -e 
canal.instance.dbUsername=root -e canal.instance.dbPassword=canal -e 
canal.instance.connectionCharset=UTF-8 -e canal.instance.tsdb.enable=true -e 
canal.instance.gtidon=false --name=pulsar-canal-server -p 8000:8000 -p 
: -p 1:1 -p 2:2 -m 4096m canal/canal-server:v1.1.2
-```
-
-- Start pulsar standalone
-
-```$bash
-docker pull apachepulsar/pulsar:2.3.0
-docker run -d -it --link pulsar-canal-server -p 6650:6650 -p 8080:8080 -v 
$PWD/data:/pulsar/data --name pulsar-standalone apachepulsar/pulsar:2.3.0 
bin/pulsar standalone
-```
-
-- Start pulsar-io in standalone
-
-- Config file canal-mysql-source-config.yaml
-
-```$yaml
-configs:
-zkServers: ""
-batchSize: "5120"
-destination: "test"
-username: ""
-password: ""
-cluster: false
-singleHostname: "pulsar-canal-server"
-singlePort: "1"
-```
-- Consumer file pulsar-client.py for test
-```
-import pulsar
-
-client = pulsar.Client('pulsar://localhost:6650')
-consumer = client.subscribe('my-topic',
-subscription_name='my-sub')
-
-while True:
-msg = consumer.receive()
-print("Received message: '%s'" % msg.data())
-consumer.acknowledge(msg)
-
-client.close()
-```
-
-- Copy config file and test file to pulsar server
-
-```$bash
-docker cp canal-mysql-source-config.yaml pulsar-standalone:/pulsar/conf/
-docker cp pulsar-client.py pulsar-standalone:/pulsar/
-```
-
-- Download canal connector and 

[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5173: [Doc] Update Canal source connector guide

[GitBox]

Anonymitaet commented on a change in pull request #5173: [Doc] Update Canal 
source connector guide
URL: https://github.com/apache/pulsar/pull/5173#discussion_r323045209
 
 

 ##
 File path: site2/docs/io-cdc-canal.md
 ##
 @@ -1,174 +1,200 @@
 ---
 id: io-cdc-canal
-title: CDC Canal Connector
-sidebar_label: CDC Canal Connector
+title: Canal source connector
+sidebar_label: Canal source connector
 ---
 
-### Source Configuration Options
+This guide explains the configuration and usage of Canal source connector.
 
-The Configuration is mostly related to Canal task config.
+## Configuration
+
+The configuration of Canal source connector has the following parameters.
+
+### Parameter
 
 | Name | Required | Default | Description |
 |--|--|-|-|
-| `zkServers` | `false` | `127.0.0.1:2181` | `The address and port of the 
zookeeper . if canal server configured to cluster mode` |
-| `batchSize` | `true` | `5120` | `Take 5120 records from the canal server in 
batches` |
-| `username` | `false` | `` | `Canal server account, not MySQL` |
-| `password` | `false` | `` | `Canal server password, not MySQL` |
-| `cluster` | `false` | `false` | `Decide whether to open cluster mode based 
on canal server configuration, true: cluster mode, false: standalone mode` |
-| `singleHostname` | `false` | `127.0.0.1` | `The address of canal server` |
-| `singlePort` | `false` | `1` | `The port of canal server` |
-
-
-### Configuration Example
-
-Here is a configuration Json example:
-
-```$json
-{
-"zkServers": "127.0.0.1:2181",
-"batchSize": "5120",
-"destination": "example",
-"username": "",
-"password": "",
-"cluster": false,
-"singleHostname": "127.0.0.1",
-"singlePort": "1",
-}
-```
-You could also find the yaml example in this 
[file](https://github.com/apache/pulsar/blob/master/pulsar-io/canal/src/main/resources/canal-mysql-source-config.yaml),
 which has similar content below:
-
-```$yaml
-configs:
-zkServers: "127.0.0.1:2181"
-batchSize: "5120"
-destination: "example"
-username: ""
-password: ""
-cluster: false
-singleHostname: "127.0.0.1"
-singlePort: "1"
-```
-
-### Usage example
-
-Here is a simple example to store MySQL change data using above example config.
-
-- Start a MySQL server
-
-```$bash
-docker pull mysql:5.7
-docker run -d -it --rm --name pulsar-mysql -p 3306:3306 -e 
MYSQL_ROOT_PASSWORD=canal -e MYSQL_USER=mysqluser -e MYSQL_PASSWORD=mysqlpw 
mysql:5.7
-```
-- Modify configuration files mysqld.cnf
-
-```
-[mysqld]
-pid-file= /var/run/mysqld/mysqld.pid
-socket  = /var/run/mysqld/mysqld.sock
-datadir = /var/lib/mysql
-#log-error  = /var/log/mysql/error.log
-# By default we only accept connections from localhost
-#bind-address   = 127.0.0.1
-# Disabling symbolic-links is recommended to prevent assorted security risks
-symbolic-links=0
-log-bin=mysql-bin
-binlog-format=ROW
-server_id=1
-```
-
-- Copy file to mysql server from local and restart mysql server
-```$bash
-docker cp mysqld.cnf pulsar-mysql:/etc/mysql/mysql.conf.d/
-docker restart pulsar-mysql
-```
-
-- Create test database in mysql server
-```$bash
-docker exec -it pulsar-mysql /bin/bash
-mysql -h 127.0.0.1 -uroot -pcanal -e 'create database test;'
-```
-
-- Start canal server and connect mysql server
-
-```
-docker pull canal/canal-server:v1.1.2
-docker run -d -it --link pulsar-mysql -e canal.auto.scan=false -e 
canal.destinations=test -e canal.instance.master.address=pulsar-mysql:3306 -e 
canal.instance.dbUsername=root -e canal.instance.dbPassword=canal -e 
canal.instance.connectionCharset=UTF-8 -e canal.instance.tsdb.enable=true -e 
canal.instance.gtidon=false --name=pulsar-canal-server -p 8000:8000 -p 
: -p 1:1 -p 2:2 -m 4096m canal/canal-server:v1.1.2
-```
-
-- Start pulsar standalone
-
-```$bash
-docker pull apachepulsar/pulsar:2.3.0
-docker run -d -it --link pulsar-canal-server -p 6650:6650 -p 8080:8080 -v 
$PWD/data:/pulsar/data --name pulsar-standalone apachepulsar/pulsar:2.3.0 
bin/pulsar standalone
-```
-
-- Start pulsar-io in standalone
-
-- Config file canal-mysql-source-config.yaml
-
-```$yaml
-configs:
-zkServers: ""
-batchSize: "5120"
-destination: "test"
-username: ""
-password: ""
-cluster: false
-singleHostname: "pulsar-canal-server"
-singlePort: "1"
-```
-- Consumer file pulsar-client.py for test
-```
-import pulsar
-
-client = pulsar.Client('pulsar://localhost:6650')
-consumer = client.subscribe('my-topic',
-subscription_name='my-sub')
-
-while True:
-msg = consumer.receive()
-print("Received message: '%s'" % msg.data())
-consumer.acknowledge(msg)
-
-client.close()
-```
-
-- Copy config file and test file to pulsar server
-
-```$bash
-docker cp canal-mysql-source-config.yaml pulsar-standalone:/pulsar/conf/
-docker cp pulsar-client.py pulsar-standalone:/pulsar/
-```
-
-- Download canal connector and 

[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5173: [Doc] Update Canal source connector guide

[GitBox]

Anonymitaet commented on a change in pull request #5173: [Doc] Update Canal 
source connector guide
URL: https://github.com/apache/pulsar/pull/5173#discussion_r323045041
 
 

 ##
 File path: site2/docs/io-cdc-canal.md
 ##
 @@ -1,174 +1,200 @@
 ---
 id: io-cdc-canal
-title: CDC Canal Connector
-sidebar_label: CDC Canal Connector
+title: Canal source connector
+sidebar_label: Canal source connector
 ---
 
-### Source Configuration Options
+This guide explains the configuration and usage of Canal source connector.
 
-The Configuration is mostly related to Canal task config.
+## Configuration
+
+The configuration of Canal source connector has the following parameters.
+
+### Parameter
 
 | Name | Required | Default | Description |
 |--|--|-|-|
-| `zkServers` | `false` | `127.0.0.1:2181` | `The address and port of the 
zookeeper . if canal server configured to cluster mode` |
-| `batchSize` | `true` | `5120` | `Take 5120 records from the canal server in 
batches` |
-| `username` | `false` | `` | `Canal server account, not MySQL` |
-| `password` | `false` | `` | `Canal server password, not MySQL` |
-| `cluster` | `false` | `false` | `Decide whether to open cluster mode based 
on canal server configuration, true: cluster mode, false: standalone mode` |
-| `singleHostname` | `false` | `127.0.0.1` | `The address of canal server` |
-| `singlePort` | `false` | `1` | `The port of canal server` |
-
-
-### Configuration Example
-
-Here is a configuration Json example:
-
-```$json
-{
-"zkServers": "127.0.0.1:2181",
-"batchSize": "5120",
-"destination": "example",
-"username": "",
-"password": "",
-"cluster": false,
-"singleHostname": "127.0.0.1",
-"singlePort": "1",
-}
-```
-You could also find the yaml example in this 
[file](https://github.com/apache/pulsar/blob/master/pulsar-io/canal/src/main/resources/canal-mysql-source-config.yaml),
 which has similar content below:
-
-```$yaml
-configs:
-zkServers: "127.0.0.1:2181"
-batchSize: "5120"
-destination: "example"
-username: ""
-password: ""
-cluster: false
-singleHostname: "127.0.0.1"
-singlePort: "1"
-```
-
-### Usage example
-
-Here is a simple example to store MySQL change data using above example config.
-
-- Start a MySQL server
-
-```$bash
-docker pull mysql:5.7
-docker run -d -it --rm --name pulsar-mysql -p 3306:3306 -e 
MYSQL_ROOT_PASSWORD=canal -e MYSQL_USER=mysqluser -e MYSQL_PASSWORD=mysqlpw 
mysql:5.7
-```
-- Modify configuration files mysqld.cnf
-
-```
-[mysqld]
-pid-file= /var/run/mysqld/mysqld.pid
-socket  = /var/run/mysqld/mysqld.sock
-datadir = /var/lib/mysql
-#log-error  = /var/log/mysql/error.log
-# By default we only accept connections from localhost
-#bind-address   = 127.0.0.1
-# Disabling symbolic-links is recommended to prevent assorted security risks
-symbolic-links=0
-log-bin=mysql-bin
-binlog-format=ROW
-server_id=1
-```
-
-- Copy file to mysql server from local and restart mysql server
-```$bash
-docker cp mysqld.cnf pulsar-mysql:/etc/mysql/mysql.conf.d/
-docker restart pulsar-mysql
-```
-
-- Create test database in mysql server
-```$bash
-docker exec -it pulsar-mysql /bin/bash
-mysql -h 127.0.0.1 -uroot -pcanal -e 'create database test;'
-```
-
-- Start canal server and connect mysql server
-
-```
-docker pull canal/canal-server:v1.1.2
-docker run -d -it --link pulsar-mysql -e canal.auto.scan=false -e 
canal.destinations=test -e canal.instance.master.address=pulsar-mysql:3306 -e 
canal.instance.dbUsername=root -e canal.instance.dbPassword=canal -e 
canal.instance.connectionCharset=UTF-8 -e canal.instance.tsdb.enable=true -e 
canal.instance.gtidon=false --name=pulsar-canal-server -p 8000:8000 -p 
: -p 1:1 -p 2:2 -m 4096m canal/canal-server:v1.1.2
-```
-
-- Start pulsar standalone
-
-```$bash
-docker pull apachepulsar/pulsar:2.3.0
-docker run -d -it --link pulsar-canal-server -p 6650:6650 -p 8080:8080 -v 
$PWD/data:/pulsar/data --name pulsar-standalone apachepulsar/pulsar:2.3.0 
bin/pulsar standalone
-```
-
-- Start pulsar-io in standalone
-
-- Config file canal-mysql-source-config.yaml
-
-```$yaml
-configs:
-zkServers: ""
-batchSize: "5120"
-destination: "test"
-username: ""
-password: ""
-cluster: false
-singleHostname: "pulsar-canal-server"
-singlePort: "1"
-```
-- Consumer file pulsar-client.py for test
-```
-import pulsar
-
-client = pulsar.Client('pulsar://localhost:6650')
-consumer = client.subscribe('my-topic',
-subscription_name='my-sub')
-
-while True:
-msg = consumer.receive()
-print("Received message: '%s'" % msg.data())
-consumer.acknowledge(msg)
-
-client.close()
-```
-
-- Copy config file and test file to pulsar server
-
-```$bash
-docker cp canal-mysql-source-config.yaml pulsar-standalone:/pulsar/conf/
-docker cp pulsar-client.py pulsar-standalone:/pulsar/
-```
-
-- Download canal connector and 

[GitHub] [pulsar] Anonymitaet opened a new pull request #5173: [Doc] Update Canal source connector guide

[GitBox]

Anonymitaet opened a new pull request #5173: [Doc] Update Canal source 
connector guide
URL: https://github.com/apache/pulsar/pull/5173
 
 
   I've updated this guide based on `CanalSourceConfig.java` and 
`canal-mysql-source-config.yaml`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] aahmed-se commented on issue #5166: [Issue 5050][docs] Adjust the content structure of the security chapter

[GitBox]

aahmed-se commented on issue #5166: [Issue 5050][docs] Adjust the content 
structure of the security chapter
URL: https://github.com/apache/pulsar/pull/5166#issuecomment-530199047
 
 
   rebase this on master


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] aahmed-se commented on issue #5171: Add ping util to Docker file

[GitBox]

aahmed-se commented on issue #5171: Add ping util to Docker file
URL: https://github.com/apache/pulsar/pull/5171#issuecomment-530198976
 
 
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Monica-zy commented on issue #5164: [doc] Improve Pulsar Deployment monitoring

[GitBox]

Monica-zy commented on issue #5164: [doc] Improve Pulsar Deployment monitoring
URL: https://github.com/apache/pulsar/pull/5164#issuecomment-530198468
 
 
   run cpp tests
   run integration tests
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet edited a comment on issue #5015: [Doc] Update Pulsar Connector Guide

[GitBox]

Anonymitaet edited a comment on issue #5015: [Doc] Update Pulsar Connector Guide
URL: https://github.com/apache/pulsar/issues/5015#issuecomment-524149120
 
 
   PRs related to this issue:
   
   ## Use 
   * https://github.com/apache/pulsar/pull/5071
   
   ## Built-in connector 
   * https://github.com/apache/pulsar/pull/5168
   
   ## Debug 
   * https://github.com/apache/pulsar/pull/5041
   
   ## Connector Admin CLI
   * https://github.com/apache/pulsar/pull/5028


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Monica-zy commented on issue #5140: [doc] Improve Pulsar Deployment DCOS

[GitBox]

Monica-zy commented on issue #5140: [doc] Improve Pulsar Deployment DCOS
URL: https://github.com/apache/pulsar/pull/5140#issuecomment-530198123
 
 
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Monica-zy commented on issue #5092: [doc] Improve Pulsar Security Extending

[GitBox]

Monica-zy commented on issue #5092: [doc] Improve Pulsar Security Extending
URL: https://github.com/apache/pulsar/pull/5092#issuecomment-530198040
 
 
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] codelipenghui edited a comment on issue #5112: How to Date type -> AVRO -> Presto

[GitBox]

codelipenghui edited a comment on issue #5112: How to Date type -> AVRO -> 
Presto
URL: https://github.com/apache/pulsar/issues/5112#issuecomment-530193329
 
 
   @KannarFr I see something different. 
   
   > The output of show columns from topic
   
   Only show 2 columns? i can't see the internal columns and 
`timestamp.imillis` looks weird, i don't know why add `.imillis` in the end for 
column name `timestamp `.
   
   > The select * works and outputs
   
   Also no internal columns, do you trim them deliberately?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] codelipenghui commented on issue #5112: How to Date type -> AVRO -> Presto

[GitBox]

codelipenghui commented on issue #5112: How to Date type -> AVRO -> Presto
URL: https://github.com/apache/pulsar/issues/5112#issuecomment-530193329
 
 
   @KannarFr I see something different. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] aahmed-se commented on issue #5156: [Documentation] Add Rust Client Library

[GitBox]

aahmed-se commented on issue #5156: [Documentation] Add Rust Client Library
URL: https://github.com/apache/pulsar/pull/5156#issuecomment-530191337
 
 
   @AlJaMa You need to rebase this to get some test fixes from master


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] aahmed-se opened a new pull request #5172: Change logging for debezium key value pairs

[GitBox]

aahmed-se opened a new pull request #5172: Change logging for debezium key 
value pairs
URL: https://github.com/apache/pulsar/pull/5172
 
 
Make naming consistent for Debezium postgreSql


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jiazhai commented on issue #5151: Broker fails to start with function worker enabled and broker client using TLS

[GitBox]

jiazhai commented on issue #5151: Broker fails to start with function worker 
enabled and broker client using TLS
URL: https://github.com/apache/pulsar/pull/5151#issuecomment-530189510
 
 
   @cdbartholomew Thanks for the fix.  This may work for your test, but may not 
friendly for user setting. 
   From my view the right logic could like this:
   1. `workerConfig.isUseTls` is the main config. If user set  
`workerConfig.isUseTls=true`, then we should comply with it.
   2. once `workerConfig.isUseTls=true`, then it is expect that the broker has 
already has broker.conf configured with tls enabled and "webServicePortTls" 
with right value, 
   3. For the reason analysed in  #4560,  function worker will create internal 
admin/client inside worker, it should passively config these:
   ```
   brokerClientTlsEnabled=
   brokerClientAuthenticationPlugin=
   brokerClientAuthenticationParameters=
   ```
   
   As you find above `workerConfig.isUseTls` is the main config that user 
wanted. and configs in 2 and 3 above, are some passively configs or configs 
that should comply with user's requirements.
   
   @tuteng, who is work on fix for #4560.  
   @jerrypeng, who init this worker config. 
   @murong00, who also touched this part of code recently.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 commented on issue #5123: Modify the schema decode method can decode ByteBuf

[GitBox]

congbobo184 commented on issue #5123: Modify the schema decode method can 
decode ByteBuf
URL: https://github.com/apache/pulsar/pull/5123#issuecomment-530189511
 
 
   run Integration Tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] aahmed-se opened a new pull request #5171: Add ping util to Docker file

[GitBox]

aahmed-se opened a new pull request #5171: Add ping util to Docker file
URL: https://github.com/apache/pulsar/pull/5171
 
 
   This adds the ping command to the docker image


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] aahmed-se closed pull request #5160: Disable postgres debezium integration test

[GitBox]

aahmed-se closed pull request #5160: Disable postgres debezium integration test
URL: https://github.com/apache/pulsar/pull/5160
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch postgres2 deleted (was fceee7c)

[aahmed]

This is an automated email from the ASF dual-hosted git repository.

aahmed pushed a change to branch postgres2
in repository https://gitbox.apache.org/repos/asf/pulsar.git.


 was fceee7c  Change logging for debezium key value pairs

The revisions that were on this branch are still contained in
other references; therefore, this change does not discard any commits
from the repository.

[GitHub] [pulsar] aahmed-se closed pull request #5159: Change logging for debezium key value pairs

[GitBox]

aahmed-se closed pull request #5159: Change logging for debezium key value pairs
URL: https://github.com/apache/pulsar/pull/5159
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch postgres3 deleted (was c85c8ad)

[aahmed]

This is an automated email from the ASF dual-hosted git repository.

aahmed pushed a change to branch postgres3
in repository https://gitbox.apache.org/repos/asf/pulsar.git.


 was c85c8ad  Disable postgres debezium integration test

The revisions that were on this branch are still contained in
other references; therefore, this change does not discard any commits
from the repository.

[GitHub] [pulsar] aahmed-se closed pull request #5158: Make naming consistent for Debezium postgreSql

[GitBox]

aahmed-se closed pull request #5158: Make naming consistent for Debezium 
postgreSql
URL: https://github.com/apache/pulsar/pull/5158
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] aahmed-se closed pull request #5161: Add ping util to Docker file

[GitBox]

aahmed-se closed pull request #5161: Add ping util to Docker file
URL: https://github.com/apache/pulsar/pull/5161
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch postgres1 deleted (was 5da44c1)

[aahmed]

This is an automated email from the ASF dual-hosted git repository.

aahmed pushed a change to branch postgres1
in repository https://gitbox.apache.org/repos/asf/pulsar.git.


 was 5da44c1  Make naming consistent for Debezium postgreSql

The revisions that were on this branch are still contained in
other references; therefore, this change does not discard any commits
from the repository.

[pulsar] branch ping1 deleted (was 32a46c4)

[aahmed]

This is an automated email from the ASF dual-hosted git repository.

aahmed pushed a change to branch ping1
in repository https://gitbox.apache.org/repos/asf/pulsar.git.


 was 32a46c4  Add ping util to Docker file

The revisions that were on this branch are still contained in
other references; therefore, this change does not discard any commits
from the repository.

[GitHub] [pulsar] massakam commented on issue #5160: Disable postgres debezium integration test

[GitBox]

massakam commented on issue #5160: Disable postgres debezium integration test
URL: https://github.com/apache/pulsar/pull/5160#issuecomment-530188081
 
 
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 commented on issue #5123: Modify the schema decode method can decode ByteBuf

[GitBox]

congbobo184 commented on issue #5123: Modify the schema decode method can 
decode ByteBuf
URL: https://github.com/apache/pulsar/pull/5123#issuecomment-530186655
 
 
   run  Integration Tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on issue #5135: [doc] Improve Pulsar deployment bare metal multi cluster

[GitBox]

Jennifer88huang commented on issue #5135: [doc] Improve Pulsar deployment bare 
metal multi cluster
URL: https://github.com/apache/pulsar/pull/5135#issuecomment-530185475
 
 
   @Monica-zy You might need to double check this PR, and ask @Anonymitaet to 
help review.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #5135: [doc] Improve Pulsar deployment bare metal multi cluster

[GitBox]

Jennifer88huang commented on a change in pull request #5135: [doc] Improve 
Pulsar deployment bare metal multi cluster
URL: https://github.com/apache/pulsar/pull/5135#discussion_r323029461
 
 

 ##
 File path: site2/docs/deploy-bare-metal-multi-cluster.md
 ##
 @@ -62,45 +62,45 @@ The Pulsar binary package initially contains the following 
directories:
 
 Directory | Contains
 :-|:
-`bin` | Pulsar's [command-line tools](reference-cli-tools.md), such as 
[`pulsar`](reference-cli-tools.md#pulsar) and 
[`pulsar-admin`](reference-pulsar-admin.md)
+`bin` | [Command-line tools](reference-cli-tools.md) of Pulsar, such as 
[`pulsar`](reference-cli-tools.md#pulsar) and 
[`pulsar-admin`](reference-pulsar-admin.md)
 `conf` | Configuration files for Pulsar, including for [broker 
configuration](reference-configuration.md#broker), [ZooKeeper 
configuration](reference-configuration.md#zookeeper), and more
 `examples` | A Java JAR file containing example [Pulsar 
Functions](functions-overview.md)
-`lib` | The [JAR](https://en.wikipedia.org/wiki/JAR_(file_format)) files used 
by Pulsar
+`lib` | The [JAR](https://en.wikipedia.org/wiki/JAR_(file_format)) files that 
Pulsar uses 
 `licenses` | License files, in `.txt` form, for various components of the 
Pulsar codebase
 
-These directories will be created once you begin running Pulsar:
+The following directories are created once you begin running Pulsar:
 
 Directory | Contains
 :-|:
-`data` | The data storage directory used by ZooKeeper and BookKeeper
+`data` | The data storage directory that ZooKeeper and BookKeeper use
 `instances` | Artifacts created for [Pulsar Functions](functions-overview.md)
-`logs` | Logs created by the installation
+`logs` | Logs that the installation creates
 
 
-## Deploying ZooKeeper
+## Deploy ZooKeeper
 
 Each Pulsar instance relies on two separate ZooKeeper quorums.
 
-* [Local ZooKeeper](#deploying-local-zookeeper) operates at the cluster level 
and provides cluster-specific configuration management and coordination. Each 
Pulsar cluster needs to have a dedicated ZooKeeper cluster.
-* [Configuration Store](#deploying-configuration-store) operates at the 
instance level and provides configuration management for the entire system (and 
thus across clusters). The configuration store quorum can be provided by an 
independent cluster of machines or by the same machines used by local ZooKeeper.
+* [Local ZooKeeper](#deploying-local-zookeeper). It operates at the cluster 
level and provides cluster-specific configuration management and coordination. 
Each Pulsar cluster needs to have a dedicated ZooKeeper cluster.
 
 Review comment:
   Check the links when you update the titles.
   For example, since you have update the title from "Deploying local 
ZooKeeper" to "Deploy local ZooKeeper", the link should be update accordingly 
as "[Local ZooKeeper](#deploy-local-zookeeper)".
   Check all similar cases.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #5135: [doc] Improve Pulsar deployment bare metal multi cluster

[GitBox]

Jennifer88huang commented on a change in pull request #5135: [doc] Improve 
Pulsar deployment bare metal multi cluster
URL: https://github.com/apache/pulsar/pull/5135#discussion_r323028731
 
 

 ##
 File path: site2/docs/deploy-bare-metal-multi-cluster.md
 ##
 @@ -215,29 +211,29 @@ $ bin/pulsar initialize-cluster-metadata \
   --broker-service-url-tls pulsar+ssl://pulsar.us-west.example.com:6651/
 ```
 
-As you can see from the example above, the following needs to be specified:
+As you can see from the example above, you need to specify the following:
 
 * The name of the cluster
 * The local ZooKeeper connection string for the cluster
 * The configuration store connection string for the entire instance
 * The web service URL for the cluster
 * A broker service URL enabling interaction with the 
[brokers](reference-terminology.md#broker) in the cluster
 
-If you're using [TLS](security-tls-transport.md), you'll also need to specify 
a TLS web service URL for the cluster as well as a TLS broker service URL for 
the brokers in the cluster.
+If you use [TLS](security-tls-transport.md), you also need to specify a TLS 
web service URL for the cluster as well as a TLS broker service URL for the 
brokers in the cluster.
 
 Make sure to run `initialize-cluster-metadata` for each cluster in your 
instance.
 
-## Deploying BookKeeper
+## Deploy BookKeeper
 
 BookKeeper provides [persistent message 
storage](concepts-architecture-overview.md#persistent-storage) for Pulsar.
 
 Each Pulsar broker needs to have its own cluster of bookies. The BookKeeper 
cluster shares a local ZooKeeper quorum with the Pulsar cluster.
 
-### Configuring bookies
+### Configure bookies
 
-BookKeeper bookies can be configured using the 
[`conf/bookkeeper.conf`](reference-configuration.md#bookkeeper) configuration 
file. The most important aspect of configuring each bookie is ensuring that the 
[`zkServers`](reference-configuration.md#bookkeeper-zkServers) parameter is set 
to the connection string for the Pulsar cluster's local ZooKeeper.
+You can configure BookKeeper bookies using the 
[`conf/bookkeeper.conf`](reference-configuration.md#bookkeeper) configuration 
file. The most important aspect of configuring each bookie is ensuring that the 
[`zkServers`](reference-configuration.md#bookkeeper-zkServers) parameter is set 
to the connection string for the local ZooKeeper of Pulsar cluster.
 
-### Starting up bookies
+### Start up bookies
 
 Review comment:
   "start up" or "start"?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet edited a comment on issue #5168: [Doc] Add Built-in Connector

[GitBox]

Anonymitaet edited a comment on issue #5168: [Doc] Add Built-in Connector
URL: https://github.com/apache/pulsar/pull/5168#issuecomment-530184195
 
 
   @murong00 Great! That would be very helpful!
   
   The yellow highlighted contents in the image are to be added, could you 
please help add? (I'm adding as well) Thank you


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet commented on issue #5168: [Doc] Add Built-in Connector

[GitBox]

Anonymitaet commented on issue #5168: [Doc] Add Built-in Connector
URL: https://github.com/apache/pulsar/pull/5168#issuecomment-530184195
 
 
   @murong00 Great! That would be very helpful!
   
   The yellow highlighted contents in the image are to be added, could you 
please help? Thank you


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on issue #5135: [doc] Improve Pulsar deployment bare metal multi cluster

[GitBox]

Jennifer88huang commented on issue #5135: [doc] Improve Pulsar deployment bare 
metal multi cluster
URL: https://github.com/apache/pulsar/pull/5135#issuecomment-530183254
 
 
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] murong00 commented on issue #5168: [Doc] Add Built-in Connector

[GitBox]

murong00 commented on issue #5168: [Doc] Add Built-in Connector
URL: https://github.com/apache/pulsar/pull/5168#issuecomment-530183243
 
 
   run cpp tests
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet commented on issue #5163: [Doc] Add reader configuration description for Java client

[GitBox]

Anonymitaet commented on issue #5163: [Doc] Add reader configuration 
description for Java client
URL: https://github.com/apache/pulsar/pull/5163#issuecomment-530183103
 
 
   Fix https://github.com/apache/pulsar/issues/4509


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on issue #5167: [doc] Improve Pulsar multi-cluster deployment

[GitBox]

Jennifer88huang commented on issue #5167: [doc] Improve Pulsar multi-cluster 
deployment
URL: https://github.com/apache/pulsar/pull/5167#issuecomment-530183141
 
 
   @Monica-zy There are some conflicts in this PR with yours in #5135. Could 
you merge yours first and then try to review this one?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet edited a comment on issue #4509: [Documentation][Client Lib Configuration] A full list configuration for Producer/Consumer/Client on website

[GitBox]

Anonymitaet edited a comment on issue #4509: [Documentation][Client Lib 
Configuration] A full list configuration for Producer/Consumer/Client on website
URL: https://github.com/apache/pulsar/issues/4509#issuecomment-527814327
 
 
   Done in several PRs:
   
   * https://github.com/apache/pulsar/pull/5087
   * https://github.com/apache/pulsar/pull/5122
   * https://github.com/apache/pulsar/pull/5154
   * https://github.com/apache/pulsar/pull/5163


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] murong00 commented on issue #5168: [Doc] Add Built-in Connector

[GitBox]

murong00 commented on issue #5168: [Doc] Add Built-in Connector
URL: https://github.com/apache/pulsar/pull/5168#issuecomment-530183089
 
 
   @Anonymitaet I can help to add part of the sink examples when i'm available 
if you need.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jiazhai commented on issue #5153: [doc]Add brokerClient authentication configuration

[GitBox]

jiazhai commented on issue #5153: [doc]Add brokerClient authentication 
configuration
URL: https://github.com/apache/pulsar/pull/5153#issuecomment-530183039
 
 
   @tuteng Thanks for the fix. Please also help add these config for other auth 
method besides token.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet commented on issue #5154: [Doc] Add producer configuration description for Java client

[GitBox]

Anonymitaet commented on issue #5154: [Doc] Add producer configuration 
description for Java client
URL: https://github.com/apache/pulsar/pull/5154#issuecomment-530182863
 
 
   Fix https://github.com/apache/pulsar/pull/5154


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet edited a comment on issue #5154: [Doc] Add producer configuration description for Java client

[GitBox]

Anonymitaet edited a comment on issue #5154: [Doc] Add producer configuration 
description for Java client
URL: https://github.com/apache/pulsar/pull/5154#issuecomment-530182863
 
 
   Fix https://github.com/apache/pulsar/issues/4509


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet edited a comment on issue #4509: [Documentation][Client Lib Configuration] A full list configuration for Producer/Consumer/Client on website

[GitBox]

Anonymitaet edited a comment on issue #4509: [Documentation][Client Lib 
Configuration] A full list configuration for Producer/Consumer/Client on website
URL: https://github.com/apache/pulsar/issues/4509#issuecomment-527814327
 
 
   Done in several PRs:
   
   * https://github.com/apache/pulsar/pull/5087
   * https://github.com/apache/pulsar/pull/5122
   * https://github.com/apache/pulsar/pull/5154


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on issue #5169: [docs] Update the version-2.4.1-sidebars.json file for 2.4.1 release

[GitBox]

Jennifer88huang commented on issue #5169: [docs] Update the 
version-2.4.1-sidebars.json file for 2.4.1 release
URL: https://github.com/apache/pulsar/pull/5169#issuecomment-530181655
 
 
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on issue #5142: [docs] Managing partitioned topics page has odd "Nonpartitioned topics resources" section

[GitBox]

Jennifer88huang commented on issue #5142: [docs] Managing partitioned topics 
page has odd "Nonpartitioned topics resources" section
URL: https://github.com/apache/pulsar/issues/5142#issuecomment-530181265
 
 
   Great, I'll check the added content in #5143. Thanks for doing so.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jiazhai commented on a change in pull request #5151: Broker fails to start with function worker enabled and broker client using TLS

[GitBox]

jiazhai commented on a change in pull request #5151: Broker fails to start with 
function worker enabled and broker client using TLS
URL: https://github.com/apache/pulsar/pull/5151#discussion_r323024509
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/PulsarBrokerStarter.java
 ##
 @@ -187,7 +189,7 @@ private static boolean argsContains(String[] args, String 
arg) {
 
workerConfig.setZooKeeperSessionTimeoutMillis(brokerConfig.getZooKeeperSessionTimeoutMillis());
 
workerConfig.setZooKeeperOperationTimeoutSeconds(brokerConfig.getZooKeeperOperationTimeoutSeconds());
 
-workerConfig.setUseTls(brokerConfig.isTlsEnabled());
+workerConfig.setUseTls(useTls);
 
 Review comment:
   this changes is right. we should reuse var `useTls`. As commenets above, 
value for `useTls` need more confirm


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jiazhai commented on a change in pull request #5151: Broker fails to start with function worker enabled and broker client using TLS

[GitBox]

jiazhai commented on a change in pull request #5151: Broker fails to start with 
function worker enabled and broker client using TLS
URL: https://github.com/apache/pulsar/pull/5151#discussion_r323023781
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/PulsarBrokerStarter.java
 ##
 @@ -160,7 +160,9 @@ private static boolean argsContains(String[] args, String 
arg) {
 workerConfig = 
WorkerConfig.load(starterArguments.fnWorkerConfigFile);
 }
 // worker talks to local broker
-boolean useTls = workerConfig.isUseTls();
+// If the broker client is configured to use TLS, then we
+// configure the function worker to use TLS
+boolean useTls = brokerConfig.isBrokerClientTlsEnabled();
 
 Review comment:
   With this change `workerConfig.isUseTls()` will not take effect any more. 
   This seems not friendly for worker config.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jiazhai commented on a change in pull request #5151: Broker fails to start with function worker enabled and broker client using TLS

[GitBox]

jiazhai commented on a change in pull request #5151: Broker fails to start with 
function worker enabled and broker client using TLS
URL: https://github.com/apache/pulsar/pull/5151#discussion_r323023781
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/PulsarBrokerStarter.java
 ##
 @@ -160,7 +160,9 @@ private static boolean argsContains(String[] args, String 
arg) {
 workerConfig = 
WorkerConfig.load(starterArguments.fnWorkerConfigFile);
 }
 // worker talks to local broker
-boolean useTls = workerConfig.isUseTls();
+// If the broker client is configured to use TLS, then we
+// configure the function worker to use TLS
+boolean useTls = brokerConfig.isBrokerClientTlsEnabled();
 
 Review comment:
   With this change `workerConfig.isUseTls()` will not take effect any more. 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] liketic commented on issue #5146: Remove method testValidateFunctionsConfigs which was abandoned

[GitBox]

liketic commented on issue #5146: Remove method testValidateFunctionsConfigs 
which was abandoned
URL: https://github.com/apache/pulsar/pull/5146#issuecomment-530174974
 
 
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch postgres4 deleted (was a99ce46)

[sijie]

This is an automated email from the ASF dual-hosted git repository.

sijie pushed a change to branch postgres4
in repository https://gitbox.apache.org/repos/asf/pulsar.git.


 was a99ce46  Fix Postgres debezium test

The revisions that were on this branch are still contained in
other references; therefore, this change does not discard any commits
from the repository.

[GitHub] [pulsar] sijie merged pull request #5162: Fix Postgres debezium test

[GitBox]

sijie merged pull request #5162: Fix Postgres debezium test
URL: https://github.com/apache/pulsar/pull/5162
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on issue #5165: [PIP-43] Support producer to send msg with different schema

[GitBox]

sijie commented on issue #5165: [PIP-43] Support producer to send msg with 
different schema
URL: https://github.com/apache/pulsar/pull/5165#issuecomment-530127343
 
 
   /cc @congbobo184 for review as well.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on issue #5167: [doc] Improve Pulsar multi-cluster deployment

[GitBox]

sijie commented on issue #5167: [doc] Improve Pulsar multi-cluster deployment
URL: https://github.com/apache/pulsar/pull/5167#issuecomment-530126088
 
 
   @Monica-zy since you were also updating this documentation, can you please 
help review this change?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on issue #5160: Disable postgres debezium integration test

[GitBox]

sijie commented on issue #5160: Disable postgres debezium integration test
URL: https://github.com/apache/pulsar/pull/5160#issuecomment-530125294
 
 
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on issue #5168: [Doc] Add Built-in Connector

[GitBox]

sijie commented on issue #5168: [Doc] Add Built-in Connector
URL: https://github.com/apache/pulsar/pull/5168#issuecomment-530125017
 
 
   run  cpp tests
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on issue #5169: [docs] Update the version-2.4.1-sidebars.json file for 2.4.1 release

[GitBox]

sijie commented on issue #5169: [docs] Update the version-2.4.1-sidebars.json 
file for 2.4.1 release
URL: https://github.com/apache/pulsar/pull/5169#issuecomment-530124649
 
 
   run cpp tests
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] aahmed-se commented on issue #5162: Fix Postgres debezium test

[GitBox]

aahmed-se commented on issue #5162: Fix Postgres debezium test
URL: https://github.com/apache/pulsar/pull/5162#issuecomment-530112450
 
 
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on issue #271: Consider using gRPC as an externally exposed API

[GitBox]

sijie commented on issue #271: Consider using gRPC as an externally exposed API
URL: https://github.com/apache/pulsar/issues/271#issuecomment-530064381
 
 
   @mickdelaney Pulsar has a python client and there is an ongoing development 
of dotnet client. Does it meet your requirement? Or gRPC is your preferred 
option?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] aahmed-se commented on issue #5162: Fix Postgres debezium test

[GitBox]

aahmed-se commented on issue #5162: Fix Postgres debezium test
URL: https://github.com/apache/pulsar/pull/5162#issuecomment-530050729
 
 
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] feathj commented on issue #5170: Kinesis source stalls on startup

[GitBox]

feathj commented on issue #5170: Kinesis source stalls on startup
URL: https://github.com/apache/pulsar/issues/5170#issuecomment-530040890
 
 
   Hey @david-streamlio / @addisonj , here is the issue report for the problem 
I am seeing with the kinesis source.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] feathj opened a new issue #5170: Kinesis source stalls on startup

[GitBox]

feathj opened a new issue #5170: Kinesis source stalls on startup
URL: https://github.com/apache/pulsar/issues/5170
 
 
   **Describe the bug**
   Kinesis source function appears to just stall on startup.
   
   **To Reproduce**
   Steps to reproduce the behavior:
   1. Create kinesis source, tied to actual kinesis stream
   2. Write records to kinesis stream
   3. Notice kinesis source pulsar function attempts to read from source 
kinesis stream, and stalls indefinitely.
   
   **Expected behavior**
   Kinesis source pulsar function should process records from kinesis stream 
and write them to target topic.
   
   
   **Additional context**
   See #3784 where kinesis source functionality was introduced.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] mickdelaney commented on issue #271: Consider using gRPC as an externally exposed API

[GitBox]

mickdelaney commented on issue #271: Consider using gRPC as an externally 
exposed API
URL: https://github.com/apache/pulsar/issues/271#issuecomment-530040041
 
 
   Any progress likely on this ? 
   We’re Python, dotnet & so have no viable option to use Pulsar. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] fxbing commented on issue #4963: Add more config for auto-topic-creation

[GitBox]

fxbing commented on issue #4963: Add more config for auto-topic-creation
URL: https://github.com/apache/pulsar/pull/4963#issuecomment-530028710
 
 
   run java8 tests
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch master updated: Add producer configuration description for Java client (#5154)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 9c9d210  Add producer configuration description for Java client (#5154)
9c9d210 is described below

commit 9c9d2101d20fe86186b74ade461fcf45ed35f5ab
Author: Anonymitaet <50226895+anonymit...@users.noreply.github.com>
AuthorDate: Wed Sep 11 00:09:11 2019 +0800

Add producer configuration description for Java client (#5154)
---
 site2/docs/client-libraries-java.md | 38 -
 1 file changed, 29 insertions(+), 9 deletions(-)

diff --git a/site2/docs/client-libraries-java.md 
b/site2/docs/client-libraries-java.md
index ccee7f9..68029a8 100644
--- a/site2/docs/client-libraries-java.md
+++ b/site2/docs/client-libraries-java.md
@@ -74,7 +74,7 @@ If you're using [TLS](security-tls-authentication.md) 
authentication, the URL wi
 pulsar+ssl://pulsar.us-west.example.com:6651
 ```
 
-## Client configuration
+## Client 
 
 You can instantiate a {@inject: 
javadoc:PulsarClient:/client/org/apache/pulsar/client/api/PulsarClient} object 
using just a URL for the target Pulsar 
[cluster](reference-terminology.md#cluster), like this:
 
@@ -123,7 +123,7 @@ Check out the Javadoc for the {@inject: 
javadoc:PulsarClient:/client/org/apache/
 
 > In addition to client-level configuration, you can also apply 
 > [producer](#configuring-producers) and [consumer](#configuring-consumers) 
 > specific configuration, as you'll see in the sections below.
 
-## Producers
+## Producer
 
 In Pulsar, producers write messages to topics. Once you've instantiated a 
{@inject: 
javadoc:PulsarClient:/client/org/apache/pulsar/client/api/PulsarClient} object 
(as in the section [above](#client-configuration)), you can create a {@inject: 
javadoc:Producer:/client/org/apache/pulsar/client/api/Producer} for a specific 
Pulsar [topic](reference-terminology.md#topic).
 
@@ -162,9 +162,29 @@ stringProducer.send("My message");
 >});
 > ```
 
-### Configuring producers
+### Configure producer
 
-If you instantiate a `Producer` object specifying only a topic name, as in the 
example above, the producer will use the default configuration. To use a 
non-default configuration, there's a variety of configurable parameters that 
you can set. 
+If you instantiate a `Producer` object specifying only a topic name, as in the 
example above, the producer uses the default configuration. 
+
+If you create a producer, you may use the `loadConf` configuration. Below are 
the available parameters used in `loadConf`.
+
+Type | Name| Description|  Default
+|---|---|---|---
+String|`topicName`|Topic name| null|
+String|`producerName`|Producer name| null
+long|`sendTimeoutMs`|Message send timeout in ms.If a message is not 
acknowledged by a server before the `sendTimeout` expires, an error is 
triggered.|3
+boolean|`blockIfQueueFull`|If set to `true`, when the outgoing message queue 
is full, the `Send` and `SendAsync` methods of producer block rather than 
failing and throwing errors. If set to `false`, when the outgoing 
message queue is full, the `Send` and `SendAsync` methods of producer fail and 
throw `ProducerQueueIsFullError` exceptions.The size of the outgoing 
message queue is determined by the `MaxPendingMessages` parameter.|false
+int|`maxPendingMessages`|Maximum size of a queue holding pending 
messages.For example, a message waiting to receive an acknowledgment 
from a [broker](reference-terminology.md#broker). By default, when 
the queue is full, all calls to the `Send` and `SendAsync` methods fail 
**unless** `BlockIfQueueFull` is set to `true`.|1000
+int|`maxPendingMessagesAcrossPartitions`|Maximum number of pending messages 
across partitions. This setting is used to lower the max pending 
messages for each partition ({@link #setMaxPendingMessages(int)}) if the total 
exceeds the configured value.|5
+MessageRoutingMode|`messageRoutingMode`|Message routing logic for producers on 
[partitioned 
topics](concepts-architecture-overview.md#partitioned-topics). This 
logic is applied only when no key is set on messages. Below are the 
available options: `pulsar.RoundRobinDistribution`: round 
robin `pulsar.UseSinglePartition`: publish all messages to a 
single partition`pulsar.CustomPartition`: a custom partitioning 
scheme|`pulsar.Roun [...]
+HashingScheme|`hashingScheme`|Hashing function that determines the partition 
on which a particular message is published (**partitioned topics 
only**).Below are the available options: 
`pulsar.JavaStringHash`: the equivalent of `String.hashCode()` in 
Java `pulsar.Murmur3_32Hash`: applies the 
[Murmur3](https://en.wikipedia.org/wiki/MurmurHash) hashing 
function`pulsar.BoostHash`: applies the hashing function from 
C++'s [Boost](https://www.b [...]
+ProducerCryptoFailureAction|`cryptoFailureAction`|Producer should take 

[GitHub] [pulsar] merlimat merged pull request #5154: [Doc] Add producer configuration description for Java client

[GitBox]

merlimat merged pull request #5154: [Doc] Add producer configuration 
description for Java client
URL: https://github.com/apache/pulsar/pull/5154
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] merlimat commented on issue #5152: Fixed spark receiver to account for all the consumer config options

[GitBox]

merlimat commented on issue #5152: Fixed spark receiver to account for all the 
consumer config options
URL: https://github.com/apache/pulsar/pull/5152#issuecomment-530005543
 
 
   run integration tests
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] KannarFr edited a comment on issue #5112: How to Date type -> AVRO -> Presto

[GitBox]

KannarFr edited a comment on issue #5112: How to Date type -> AVRO -> Presto
URL: https://github.com/apache/pulsar/issues/5112#issuecomment-529988531
 
 
   @congbobo184 @codelipenghui 
   
   I used joda, with the logical schema type. like 
   
   ```java
   @AvroSchema("{\"type\":\"long\",\"logicalType\":\"timestamp-millis\"}")
   public DateTime timestamp;
   public String ownerid;
   ```
   
   The output of `show columns from topic`
   
   ```sql
timestamp.imillis| bigint|   |
ownerid  | varchar   |   |
   ```
   
   When I select this particular field I've got `Column 'timestamp.imillis' 
cannot be resolved` other fields (text, int) work well.
   
   The `select *` works and outputs 
   
   ```sql
   | timestamp.imillis |  ownerid  |
   +---+---+
   | 1568128483098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128480098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128481098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128482098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   ```
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] KannarFr edited a comment on issue #5112: How to Date type -> AVRO -> Presto

[GitBox]

KannarFr edited a comment on issue #5112: How to Date type -> AVRO -> Presto
URL: https://github.com/apache/pulsar/issues/5112#issuecomment-529988531
 
 
   @congbobo184 @codelipenghui 
   
   I used joda, with the logical schema type. like 
   
   ```java
   @AvroSchema("{\"type\":\"long\",\"logicalType\":\"timestamp-millis\"}")
   public DateTime timestamp;
   ```
   
   The output of `show columns from topic`
   
   ```sql
timestamp.imillis| bigint   |   |
ownerid  | varchar   |   |
   ```
   
   When I select this particular field I've got `Column 'timestamp.imillis' 
cannot be resolved` other fields (text, int) work well.
   
   The `select *` works and outputs 
   
   ```sql
   | timestamp.imillis |  ownerid  |
   +---+---+
   | 1568128483098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128480098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128481098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128482098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   ```
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] KannarFr edited a comment on issue #5112: How to Date type -> AVRO -> Presto

[GitBox]

KannarFr edited a comment on issue #5112: How to Date type -> AVRO -> Presto
URL: https://github.com/apache/pulsar/issues/5112#issuecomment-529988531
 
 
   @congbobo184 @codelipenghui 
   
   I used joda, with the logical schema type. like 
   
   ```java
   @AvroSchema("{\"type\":\"long\",\"logicalType\":\"timestamp-millis\"}")
   public DateTime timestamp;
   ```
   
   The output of `show columns from topic`
   
   ```sql
timestamp.imillis| bigint|   |
ownerid  | varchar   |   |
   ```
   
   When I select this particular field I've got `Column 'timestamp.imillis' 
cannot be resolved` other fields (text, int) work well.
   
   The `select *` works and outputs 
   
   ```sql
   | timestamp.imillis |  ownerid  |
   +---+---+
   | 1568128483098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128480098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128481098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128482098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   ```
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] KannarFr edited a comment on issue #5112: How to Date type -> AVRO -> Presto

[GitBox]

KannarFr edited a comment on issue #5112: How to Date type -> AVRO -> Presto
URL: https://github.com/apache/pulsar/issues/5112#issuecomment-529988531
 
 
   @congbobo184 @codelipenghui 
   
   I used joda, with the logical schema type. like 
   
   ```java
   @AvroSchema("{\"type\":\"long\",\"logicalType\":\"timestamp-millis\"}")
   public DateTime timestamp;
   ```
   
   The output of `show columns from topic`
   
   ```sql
timestamp.imillis| bigint  |   |
ownerid  | varchar   |   |
   ```
   
   When I select this particular field I've got `Column 'timestamp.imillis' 
cannot be resolved` other fields (text, int) work well.
   
   The `select *` works and outputs 
   
   ```sql
   | timestamp.imillis |  ownerid  |
   +---+---+
   | 1568128483098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128480098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128481098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128482098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   ```
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] KannarFr edited a comment on issue #5112: How to Date type -> AVRO -> Presto

[GitBox]

KannarFr edited a comment on issue #5112: How to Date type -> AVRO -> Presto
URL: https://github.com/apache/pulsar/issues/5112#issuecomment-529988531
 
 
   @congbobo184 @codelipenghui 
   
   I used joda, with the logical schema type. like 
   
   ```java
   @AvroSchema("{\"type\":\"long\",\"logicalType\":\"timestamp-millis\"}")
   public DateTime timestamp;
   ```
   
   The output of `show columns from topic`
   
   ```sql
timestamp.imillis| bigint  |   |
ownerid | varchar   |   |
   ```
   
   When I select this particular field I've got `Column 'timestamp.imillis' 
cannot be resolved` other fields (text, int) work well.
   
   The `select *` works and outputs 
   
   ```sql
   | timestamp.imillis |  ownerid  |
   +---+---+
   | 1568128483098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128480098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128481098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128482098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   ```
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] KannarFr commented on issue #5112: How to Date type -> AVRO -> Presto

[GitBox]

KannarFr commented on issue #5112: How to Date type -> AVRO -> Presto
URL: https://github.com/apache/pulsar/issues/5112#issuecomment-529988531
 
 
   @congbobo184 @codelipenghui 
   
   I used joda, with the logical schema type. like 
   
   ```java
   @AvroSchema("{\"type\":\"long\",\"logicalType\":\"timestamp-millis\"}")
   public DateTime timestamp;
   ```
   
   The output of `show columns from topic`
   
   ```sql
timestamp.imillis| bigint|   |
   ```
   
   When I select this particular field I've got `Column 'timestamp.imillis' 
cannot be resolved` other fields (text, int) work well.
   
   The `select *` works and outputs 
   
   ```sql
   | timestamp.imillis |  ownerid  |
   +---+---+
   | 1568128483098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128480098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128481098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   | 1568128482098 | orga_858600a8-74f4-4d75-a8a3-f5b368be093c |
   ```
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] aahmed-se commented on issue #5159: Change logging for debezium key value pairs

[GitBox]

aahmed-se commented on issue #5159: Change logging for debezium key value pairs
URL: https://github.com/apache/pulsar/pull/5159#issuecomment-529945302
 
 
   @vzhikserg no it needs to be changed the focus right now is getting this 
merged
   
   https://github.com/apache/pulsar/pull/5162


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] aahmed-se commented on issue #5162: Fix Postgres debezium test

[GitBox]

aahmed-se commented on issue #5162: Fix Postgres debezium test
URL: https://github.com/apache/pulsar/pull/5162#issuecomment-529941147
 
 
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] rocketraman commented on issue #5142: [docs] Managing partitioned topics page has odd "Nonpartitioned topics resources" section

[GitBox]

rocketraman commented on issue #5142: [docs] Managing partitioned topics page 
has odd "Nonpartitioned topics resources" section
URL: https://github.com/apache/pulsar/issues/5142#issuecomment-529939329
 
 
   I have added a new page "Managing non-partitioned topics", for review, to 
pull request #5143 .


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] rocketraman commented on a change in pull request #5143: [Issue 5142][docs] Remove non-partitioned topic info from partitioned topic docs

[GitBox]

rocketraman commented on a change in pull request #5143: [Issue 5142][docs] 
Remove non-partitioned topic info from partitioned topic docs
URL: https://github.com/apache/pulsar/pull/5143#discussion_r322747022
 
 

 ##
 File path: site2/docs/admin-api-non-partitioned-topics.md
 ##
 @@ -0,0 +1,98 @@
+---
+id: admin-api-non-partitioned-topics
+title: Managing non-partitioned topics
+sidebar_label: Non-Partitioned topics
+---
+
+
+You can use Pulsar's [admin API](admin-api-overview.md) to create and manage 
non-partitioned topics.
+
+In all of the instructions and commands below, the topic name structure is:
+
+```shell
+persistent://tenant/namespace/topic
+```
+
+## Non-Partitioned topics resources
+
+### Create
+
+Non-partitioned topics in Pulsar must be explicitly created. When creating a 
new non-partitioned topic you
+need to provide a name for the topic.
+
+>  Note
+>
+> By default, after 60 seconds of creation, topics are considered inactive and 
deleted automatically to prevent from generating trash data.
+>
+> To disable this feature, set `brokerDeleteInactiveTopicsEnabled`  to `false`.
+>
+> To change the frequency of checking inactive topics, set 
`brokerDeleteInactiveTopicsFrequencySeconds` to your desired value.
+>
+> For more information about these two parameters, see 
[here](reference-configuration.md#broker).
+
+ pulsar-admin
+
+You can create non-partitioned topics using the 
[`create`](reference-pulsar-admin.md#create-3)
+command and specifying the topic name as an argument.
+Here's an example:
+
+```shell
+$ bin/pulsar-admin topics create \
+  persistent://my-tenant/my-namespace/my-topic
+```
+
+ REST API
+
+{@inject: 
endpoint|PUT|/admin/v2/persistent/:tenant/:namespace/:topic|operation/createNonPartitionedTopic}
+
+ Java
+
+```java
+String topicName = "persistent://my-tenant/my-namespace/my-topic";
+// this does not seem to exist?
+admin.topics().create(topicName);
 
 Review comment:
   @Jennifer88huang @sijie Note this, I could not find this method in the 
actual API docs 
https://pulsar.apache.org/api/admin/org/apache/pulsar/client/admin/Topics.html. 
Not sure why.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] rocketraman commented on issue #5142: [docs] Managing partitioned topics page has odd "Nonpartitioned topics resources" section

[GitBox]

rocketraman commented on issue #5142: [docs] Managing partitioned topics page 
has odd "Nonpartitioned topics resources" section
URL: https://github.com/apache/pulsar/issues/5142#issuecomment-529926353
 
 
   @Jennifer88huang Note the title of the page is "Managing Partitioned 
Topics". I think a different page "Managing Non-Partitioned Topics" needs to be 
created, as such a page does not currently appear to exist. If so, I can add 
this to my pull request.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] vzhikserg commented on issue #5159: Change logging for debezium key value pairs

[GitBox]

vzhikserg commented on issue #5159: Change logging for debezium key value pairs
URL: https://github.com/apache/pulsar/pull/5159#issuecomment-529924194
 
 
   It seems that this pull request contains changes from another one 
([https://github.com/apache/pulsar/pull/5158/files](https://github.com/apache/pulsar/pull/5158/files)).
 Was it done on pupose?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] borlandor commented on issue #5136: Java Function cannot work with any JSON parser JAR

[GitBox]

borlandor commented on issue #5136: Java Function cannot work with any JSON 
parser JAR
URL: https://github.com/apache/pulsar/issues/5136#issuecomment-529921770
 
 
   Trying to use Localrun mode, the function works well, but always works 
abnormally in Create mode. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] codelipenghui commented on issue #5112: How to Date type -> AVRO -> Presto

[GitBox]

codelipenghui commented on issue #5112: How to Date type -> AVRO -> Presto
URL: https://github.com/apache/pulsar/issues/5112#issuecomment-529917236
 
 
   @KannarFr Currently can not work well with list, i think we need add support 
for presto ARRAY type in pulsar SQL.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Fernandoize removed a comment on issue #3211: pulsar producer create fail, get error :Received error from server: java.lang.NullPointerException

[GitBox]

Fernandoize removed a comment on issue #3211: pulsar producer create fail, get 
error :Received error from server: java.lang.NullPointerException
URL: https://github.com/apache/pulsar/issues/3211#issuecomment-529916520
 
 
   i get this issue in 2.4.1


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Fernandoize commented on issue #3211: pulsar producer create fail, get error :Received error from server: java.lang.NullPointerException

[GitBox]

Fernandoize commented on issue #3211: pulsar producer create fail, get error 
:Received error from server: java.lang.NullPointerException
URL: https://github.com/apache/pulsar/issues/3211#issuecomment-529916520
 
 
   i get this issue in 2.4.1


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] KannarFr edited a comment on issue #5112: How to Date type -> AVRO -> Presto

[GitBox]

KannarFr edited a comment on issue #5112: How to Date type -> AVRO -> Presto
URL: https://github.com/apache/pulsar/issues/5112#issuecomment-529914770
 
 
   @codelipenghui and about `List` to presto array, do you have an 
example?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] KannarFr edited a comment on issue #5112: How to Date type -> AVRO -> Presto

[GitBox]

KannarFr edited a comment on issue #5112: How to Date type -> AVRO -> Presto
URL: https://github.com/apache/pulsar/issues/5112#issuecomment-529914770
 
 
   @codelipenghui and about List to presto array, do you have an 
example?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] KannarFr commented on issue #5112: How to Date type -> AVRO -> Presto

[GitBox]

KannarFr commented on issue #5112: How to Date type -> AVRO -> Presto
URL: https://github.com/apache/pulsar/issues/5112#issuecomment-529914770
 
 
   @codelipenghui and about List[String] to presto array, do you have an 
example?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services