[GitHub] [pulsar] vzhikserg removed a comment on issue #5205: [doc] Fix broken links to the info about schemas and Schema Registry
vzhikserg removed a comment on issue #5205: [doc] Fix broken links to the info about schemas and Schema Registry URL: https://github.com/apache/pulsar/pull/5205#issuecomment-533315344 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] vzhikserg commented on issue #5205: [doc] Fix broken links to the info about schemas and Schema Registry
vzhikserg commented on issue #5205: [doc] Fix broken links to the info about schemas and Schema Registry URL: https://github.com/apache/pulsar/pull/5205#issuecomment-533434205 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Monica-zy commented on issue #5236: [doc] Improve Pulsar Administration Pulsar proxy
Monica-zy commented on issue #5236: [doc] Improve Pulsar Administration Pulsar proxy URL: https://github.com/apache/pulsar/pull/5236#issuecomment-533421045 @Anonymitaet Could you please help me review this, thanks~ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Monica-zy opened a new pull request #5236: [doc] Improve Pulsar Administration Pulsar proxy
Monica-zy opened a new pull request #5236: [doc] Improve Pulsar Administration Pulsar proxy URL: https://github.com/apache/pulsar/pull/5236 Motivation Improve the language and the overall descriptive style of the Pulsar Administration document (Administration-proxy section): http://pulsar.apache.org/docs/en/administration-proxy/ Modifications Adjust the tone, personal pronouns, voice also some typo errors of some sentences in the document. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet removed a comment on issue #5210: [Doc] Add *Kafka sink connector guide*
Anonymitaet removed a comment on issue #5210: [Doc] Add *Kafka sink connector guide* URL: https://github.com/apache/pulsar/pull/5210#issuecomment-533416683 run C++ tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5210: [Doc] Add *Kafka sink connector guide*
Anonymitaet commented on issue #5210: [Doc] Add *Kafka sink connector guide* URL: https://github.com/apache/pulsar/pull/5210#issuecomment-533416886 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet removed a comment on issue #5210: [Doc] Add *Kafka sink connector guide*
Anonymitaet removed a comment on issue #5210: [Doc] Add *Kafka sink connector guide* URL: https://github.com/apache/pulsar/pull/5210#issuecomment-533416462 run C++ tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5203: [Doc] Add *Kafka source connector guide*
Anonymitaet commented on issue #5203: [Doc] Add *Kafka source connector guide* URL: https://github.com/apache/pulsar/pull/5203#issuecomment-533416495 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5210: [Doc] Add *Kafka sink connector guide*
Anonymitaet commented on issue #5210: [Doc] Add *Kafka sink connector guide* URL: https://github.com/apache/pulsar/pull/5210#issuecomment-533416683 run C++ tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5211: [Doc] Add *Kinesis source connector guide*
Anonymitaet commented on issue #5211: [Doc] Add *Kinesis source connector guide* URL: https://github.com/apache/pulsar/pull/5211#issuecomment-533416374 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5210: [Doc] Add *Kafka sink connector guide*
Anonymitaet commented on issue #5210: [Doc] Add *Kafka sink connector guide* URL: https://github.com/apache/pulsar/pull/5210#issuecomment-533416462 run C++ tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5214: [Doc] Add *Kinesis sink connector guide*
Anonymitaet commented on issue #5214: [Doc] Add *Kinesis sink connector guide* URL: https://github.com/apache/pulsar/pull/5214#issuecomment-533416308 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5224: [Doc] Add *processing guarantee for connector*
Anonymitaet commented on a change in pull request #5224: [Doc] Add *processing guarantee for connector* URL: https://github.com/apache/pulsar/pull/5224#discussion_r326478609 ## File path: site2/docs/io-overview.md ## @@ -34,9 +34,106 @@ Common sinks include other messaging systems and SQL and NoSQL databases. For the complete list of Pulsar built-in sink connectors, see [sink connector](io-connectors.md#sink-connector). +## Processing guarantee + +Processing guarantees are used to: + +* handle errors when writing messages to Pulsar topics. + +* determine the subscription type used by Pulsar when reading messages from Pulsar topics. Review comment: @tuteng thank you. I've incorporated your comment, could you PTAL? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5226: [Doc] Add *HDFS2 sink connector guide*
Anonymitaet commented on a change in pull request #5226: [Doc] Add *HDFS2 sink connector guide* URL: https://github.com/apache/pulsar/pull/5226#discussion_r326477603 ## File path: site2/docs/io-hdfs2-sink.md ## @@ -0,0 +1,53 @@ +--- +id: io-hdfs2-sink +title: HDFS2 sink connector +sidebar_label: HDFS2 sink connector +--- + +The HDFS2 sink connector pulls the messages from Pulsar topics +and persists the messages to HDFS files. + +## Configuration + +The configuration of the HDFS2 sink connector has the following properties. + +### Property + +| Name | Type|Required | Default | Description +|--|--|--|-|-| +| `hdfsConfigResources` | String|true| | A file or a comma-separated list containing the Hadoop file system configuration.**Example**'core-site.xml''hdfs-site.xml' | +| `directory` | String | true | |The HDFS directory where files read from or written to. | +| `encoding` | String |false | |The character encoding for the files.**Example**UTF-8ASCII | +| `compression` | Compression |false | |The compression code used to compress or de-compress the files on HDFS. | +| `kerberosUserPrincipal` |String| false| |The principal account of Kerberos user used for authentication. | +| `keytab` | String|false|| The full pathname of the Kerberos keytab file used for authentication. | +| `filenamePrefix` |String| false | |The prefix of the files created inside the HDFS directory.**Example** The value of topicA result in files named topicA-. | +| `fileExtension` | String| false | | The extension added to the files written to HDFS.**Example**'.txt' '.seq' | +| `separator` | char|false | |The character used to separate records in a text file. If no value is provided, the contents from all records are concatenated together in one continuous byte array. | +| `syncInterval` | long| false || The interval between calls to flush data to HDFS disk in milliseconds. | +| `maxPendingRecords` |int| false|Integer.MAX_VALUE | The maximum number of records that hold in memory before acking. Setting this property to 1 makes every record send to disk before the record is acked.Setting this property to a higher value allows buffering records before flushing them to disk. + Review comment: Got, thank you, I've incorporated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] liketic commented on issue #5147: Deprecate getstatus subcommand for source command
liketic commented on issue #5147: Deprecate getstatus subcommand for source command URL: https://github.com/apache/pulsar/pull/5147#issuecomment-533408486 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] stevenwangnarvar opened a new issue #5235: lost message when not ack in partitioned topics in go client
stevenwangnarvar opened a new issue #5235: lost message when not ack in partitioned topics in go client URL: https://github.com/apache/pulsar/issues/5235 **Describe the bug** try to test this fix https://github.com/apache/pulsar/pull/4653, but found have message lost during re-process messages which are not acknowledgment. **To Reproduce** Steps to reproduce the behavior: 1. Create a partitioned topic 2. run a test go program, send several messages, and receive them but not ack them. 3. Check the received message, check if any messages are lost. **Expected behavior** No message should lost. **Screenshots** https://user-images.githubusercontent.com/49885838/65297627-b2b43580-dbbc-11e9-9739-531696a82580.png";> https://user-images.githubusercontent.com/49885838/65297632-b8aa1680-dbbc-11e9-8e31-eaf151596201.png";> **Desktop (please complete the following information):** - OS: Mac **Additional context** Queue info: ``` /pulsar-admin topics create-partitioned-topic --partitions 4 persistent://dev/genericfileproc/npulsar_test_output_topic ``` Test code ``` package main import ( "context" "fmt" "log" "time" "github.com/apache/pulsar/pulsar-client-go/pulsar" ) func main() { topic := "persistent://dev/genericfileproc/npulsar_test_output_topic" // Instantiate a Pulsar client client, err := pulsar.NewClient(pulsar.ClientOptions{ URL: "pulsar://localhost:6650", }) if err != nil { log.Fatal(err) } // Use the client to instantiate a producer producer, err := client.CreateProducer(pulsar.ProducerOptions{ Topic: topic, }) if err != nil { log.Fatal(err) } // Use the client object to instantiate a consumer consumer, err := client.Subscribe(pulsar.ConsumerOptions{ Topic:topic, SubscriptionName: "gfp-input-carrier-flat-file", Type: pulsar.Shared, AckTimeout: 15 * time.Second, }) ctx := context.Background() // Send 5 messages synchronously for i := 0; i < 5; i++ { // Create a message msg := pulsar.ProducerMessage{ Payload: []byte(fmt.Sprintf("message-%d", i)), } // Attempt to send the message if err := producer.Send(ctx, msg); err != nil { log.Fatal(err) } } // Listen indefinitely on the topic for { msg, err := consumer.Receive(ctx) if err != nil { log.Fatal(err) } // Do something with the message fmt.Println("" + time.Now().Format(time.RFC850) + " consume " + string(msg.Payload())) //consumer.Ack(msg) } } ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on a change in pull request #5226: [Doc] Add *HDFS2 sink connector guide*
tuteng commented on a change in pull request #5226: [Doc] Add *HDFS2 sink connector guide* URL: https://github.com/apache/pulsar/pull/5226#discussion_r326458977 ## File path: site2/docs/io-hdfs2-sink.md ## @@ -0,0 +1,53 @@ +--- +id: io-hdfs2-sink +title: HDFS2 sink connector +sidebar_label: HDFS2 sink connector +--- + +The HDFS2 sink connector pulls the messages from Pulsar topics +and persists the messages to HDFS files. + +## Configuration + +The configuration of the HDFS2 sink connector has the following properties. + +### Property + +| Name | Type|Required | Default | Description +|--|--|--|-|-| +| `hdfsConfigResources` | String|true| | A file or a comma-separated list containing the Hadoop file system configuration.**Example**'core-site.xml''hdfs-site.xml' | +| `directory` | String | true | |The HDFS directory where files read from or written to. | +| `encoding` | String |false | |The character encoding for the files.**Example**UTF-8ASCII | +| `compression` | Compression |false | |The compression code used to compress or de-compress the files on HDFS. | +| `kerberosUserPrincipal` |String| false| |The principal account of Kerberos user used for authentication. | +| `keytab` | String|false|| The full pathname of the Kerberos keytab file used for authentication. | +| `filenamePrefix` |String| false | |The prefix of the files created inside the HDFS directory.**Example** The value of topicA result in files named topicA-. | +| `fileExtension` | String| false | | The extension added to the files written to HDFS.**Example**'.txt' '.seq' | +| `separator` | char|false | |The character used to separate records in a text file. If no value is provided, the contents from all records are concatenated together in one continuous byte array. | +| `syncInterval` | long| false || The interval between calls to flush data to HDFS disk in milliseconds. | +| `maxPendingRecords` |int| false|Integer.MAX_VALUE | The maximum number of records that hold in memory before acking. Setting this property to 1 makes every record send to disk before the record is acked.Setting this property to a higher value allows buffering records before flushing them to disk. + Review comment: `syncInterval` default is 0 `separator` `fileExtension` `filenamePrefix` `keytab` `kerberosUserPrincipal` `compression` these fields do not need to be added with default values. `hdfsConfigResources` `directory` these fields are required, there should be no need to add default values This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5214: [Doc] Add *Kinesis sink connector guide*
tuteng commented on issue #5214: [Doc] Add *Kinesis sink connector guide* URL: https://github.com/apache/pulsar/pull/5214#issuecomment-533387036 run cpp tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5211: [Doc] Add *Kinesis source connector guide*
tuteng commented on issue #5211: [Doc] Add *Kinesis source connector guide* URL: https://github.com/apache/pulsar/pull/5211#issuecomment-533386807 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on a change in pull request #5224: [Doc] Add *processing guarantee for connector*
tuteng commented on a change in pull request #5224: [Doc] Add *processing guarantee for connector* URL: https://github.com/apache/pulsar/pull/5224#discussion_r326455385 ## File path: site2/docs/io-overview.md ## @@ -34,9 +34,106 @@ Common sinks include other messaging systems and SQL and NoSQL databases. For the complete list of Pulsar built-in sink connectors, see [sink connector](io-connectors.md#sink-connector). +## Processing guarantee + +Processing guarantees are used to: + +* handle errors when writing messages to Pulsar topics. + +* determine the subscription type used by Pulsar when reading messages from Pulsar topics. Review comment: I don't think this part of the content is necessary, and there is no necessary relationship between `Processing guarantees` and `subscription type`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] k2la opened a new issue #5234: Memory Leak in `CreateProducer` of pulsar-client-go
k2la opened a new issue #5234: Memory Leak in `CreateProducer` of pulsar-client-go URL: https://github.com/apache/pulsar/issues/5234 Memory usage increases every time we create a producer with `CreateProducer` of pulsar-client-go and close it. Memory leaks might be happening in`createProducer` of pulsar-client-cpp. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] vruc opened a new pull request #5233: Update full-width typo in schema-understand.md
vruc opened a new pull request #5233: Update full-width typo in schema-understand.md URL: https://github.com/apache/pulsar/pull/5233 fix full-width “” to half-width " *(If this PR fixes a github issue, please add `Fixes #`.)* Fixes # *(or if this PR is one task of a github issue, please add `Master Issue: #` to link to the master issue.)* Master Issue: # ### Motivation *Explain here the context, and why you're making that change. What is the problem you're trying to solve.* ### Modifications *Describe the modifications you've done.* ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API: (yes / no) - The schema: (yes / no / don't know) - The default values of configurations: (yes / no) - The wire protocol: (yes / no) - The rest endpoints: (yes / no) - The admin cli options: (yes / no) - Anything that affects deployment: (yes / no / don't know) ### Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5222: 2.4.1 pulsar-admin should have 8 schema compatibility check strategies
congbobo184 commented on issue #5222: 2.4.1 pulsar-admin should have 8 schema compatibility check strategies URL: https://github.com/apache/pulsar/issues/5222#issuecomment-533378233 @pouledodue now pulsar-admin only support four compatibility strategy, here will have an update. you can see [PIP-44](https://github.com/apache/pulsar/wiki/PIP-44%3A-Separate-schema-compatibility-checker-for-producer-and-consumer), auto update schema and compatibility strategy will be separated, #5227 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5210: [Doc] Add *Kafka sink connector guide*
tuteng commented on issue #5210: [Doc] Add *Kafka sink connector guide* URL: https://github.com/apache/pulsar/pull/5210#issuecomment-533367279 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] pouledodue closed issue #5231: schema scheme BACKWARD should permit removing fields
pouledodue closed issue #5231: schema scheme BACKWARD should permit removing fields URL: https://github.com/apache/pulsar/issues/5231 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5231: schema scheme BACKWARD should permit removing fields
sijie commented on issue #5231: schema scheme BACKWARD should permit removing fields URL: https://github.com/apache/pulsar/issues/5231#issuecomment-533360836 in 2.4.1 and before, the namespace level compatibility check policy is only applied for producers and consumers. the admin restful api is using FULL compatibility check. there is a change in 2.5.0 to change restful admin to respect to namespace setting. See #4821 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5203: [Doc] Add *Kafka source connector guide*
Anonymitaet commented on issue #5203: [Doc] Add *Kafka source connector guide* URL: https://github.com/apache/pulsar/pull/5203#issuecomment-533353921 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5210: [Doc] Add *Kafka sink connector guide*
Anonymitaet commented on issue #5210: [Doc] Add *Kafka sink connector guide* URL: https://github.com/apache/pulsar/pull/5210#issuecomment-533353696 run c++ tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat opened a new pull request #5232: Upgrade dependencies for security fixes
merlimat opened a new pull request #5232: Upgrade dependencies for security fixes URL: https://github.com/apache/pulsar/pull/5232 ### Motivation Upgrade the version of several dependencies to include fixes for a series of vulnerabilities, as reported in #4057. ### Modifications * Guava 21.0 -> 28.1 * Commons Compress 1.15 -> 1.19 * Jetty 9.4.12.v20180830 -> 9.4.20.v20190813 * Netty 3.10.1.Final -> netty-3.10.6.Final (Netty 3.x is used by ZooKeeper) Pulsar SQL dependencies: * async-http-client 1.6.5 -> 1.9.40 * Maven components 3.0.4 -> 3.6.2 Note: jackson-databind was already upgraded in #5011 Fix #4057 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] vzhikserg commented on issue #5205: [doc] Fix broken links to the info about schemas and Schema Registry
vzhikserg commented on issue #5205: [doc] Fix broken links to the info about schemas and Schema Registry URL: https://github.com/apache/pulsar/pull/5205#issuecomment-533315344 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] vzhikserg removed a comment on issue #5205: [doc] Fix broken links to the info about schemas and Schema Registry
vzhikserg removed a comment on issue #5205: [doc] Fix broken links to the info about schemas and Schema Registry URL: https://github.com/apache/pulsar/pull/5205#issuecomment-533281092 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] vzhikserg removed a comment on issue #5205: [doc] Fix broken links to the info about schemas and Schema Registry
vzhikserg removed a comment on issue #5205: [doc] Fix broken links to the info about schemas and Schema Registry URL: https://github.com/apache/pulsar/pull/5205#issuecomment-533091788 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] pouledodue commented on issue #5209: HTTP 500 on pulsar-admin schemas delete
pouledodue commented on issue #5209: HTTP 500 on pulsar-admin schemas delete URL: https://github.com/apache/pulsar/issues/5209#issuecomment-533309706 yes it's when the topic does not exist: ``` pulsar-admin tenants create r pulsar-admin namespaces create r/t pulsar-admin schemas delete persistent://r/t/e ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] pouledodue removed a comment on issue #5231: schema scheme BACKWARD should permit removing fields
pouledodue removed a comment on issue #5231: schema scheme BACKWARD should permit removing fields URL: https://github.com/apache/pulsar/issues/5231#issuecomment-533301565 oops other way around fixing it This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] pouledodue commented on issue #5231: schema scheme BACKWARD should permit new fields
pouledodue commented on issue #5231: schema scheme BACKWARD should permit new fields URL: https://github.com/apache/pulsar/issues/5231#issuecomment-533301565 oops other way around fixing it This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat opened a new pull request #5230: Allow for topic deletions with regex consumers
merlimat opened a new pull request #5230: Allow for topic deletions with regex consumers URL: https://github.com/apache/pulsar/pull/5230 ### Motivation Currently it's not possible to delete topics when there is a regex consumer attached to them. The reason is that the regex consumer will immediately reconnect and cause the topic to be re-created. ### Modifications * Added a flag in CommandSubscribe so that a regex consumer will never trigger the creation of a topic. * Subscribing to a non-existing topic, will give a specific error, that the consumer will interpret as a permanent failure and thus will stop retrying. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] pouledodue opened a new issue #5231: schema scheme BACKWARD should permit new fields
pouledodue opened a new issue #5231: schema scheme BACKWARD should permit new fields URL: https://github.com/apache/pulsar/issues/5231 **Describe the bug** With schema scheme set to `backward` I should be able to upload a new schema containing a new field. **To Reproduce** Steps to reproduce the behavior: I wrote a small script: [schema_test.sh.txt](https://github.com/apache/pulsar/files/3633093/schema_test.sh.txt) **Expected behavior** updated schema should contain `lakitu` **Screenshots** If applicable, add screenshots to help explain your problem. **Desktop (please complete the following information):** - macOS Catalina, - docker -> pulsar:2.4.1 in standalone mode ** Stack trace ``` 20:30:16.429 [ForkJoinPool.commonPool-worker-1] ERROR org.apache.pulsar.broker.service.schema.AvroSchemaBasedCompatibilityCheck - Error during schema compatibility check: Unable to read schema: { "type" : "record", "name" : "ZSchema", "fields" : [ { "name" : "goumba", "type" : "string" }, { "name" : "yoshi", "type" : "string" } ] } using schema: { "type" : "record", "name" : "ZSchema", "fields" : [ { "name" : "goumba", "type" : "string" }, { "name" : "yoshi", "type" : "string" }, { "name" : "lakitu", "type" : "string" } ] } org.apache.avro.SchemaValidationException: Unable to read schema: { "type" : "record", "name" : "ZSchema", "fields" : [ { "name" : "goumba", "type" : "string" }, { "name" : "yoshi", "type" : "string" } ] } using schema: { "type" : "record", "name" : "ZSchema", "fields" : [ { "name" : "goumba", "type" : "string" }, { "name" : "yoshi", "type" : "string" }, { "name" : "lakitu", "type" : "string" } ] } at org.apache.avro.ValidateMutualRead.canRead(ValidateMutualRead.java:70) ~[org.apache.avro-avro-1.8.2.jar:1.8.2] at org.apache.avro.ValidateMutualRead.validate(ValidateMutualRead.java:44) ~[org.apache.avro-avro-1.8.2.jar:1.8.2] at org.apache.avro.ValidateLatest.validate(ValidateLatest.java:51) ~[org.apache.avro-avro-1.8.2.jar:1.8.2] at org.apache.pulsar.broker.service.schema.AvroSchemaBasedCompatibilityCheck.isCompatible(AvroSchemaBasedCompatibilityCheck.java:56) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1] at org.apache.pulsar.broker.service.schema.AvroSchemaCompatibilityCheck.isCompatible(AvroSchemaCompatibilityCheck.java:26) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1] at org.apache.pulsar.broker.service.schema.AvroSchemaBasedCompatibilityCheck.isCompatible(AvroSchemaBasedCompatibilityCheck.java:42) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1] at org.apache.pulsar.broker.service.schema.AvroSchemaCompatibilityCheck.isCompatible(AvroSchemaCompatibilityCheck.java:26) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1] at org.apache.pulsar.broker.service.schema.SchemaRegistryServiceImpl.isCompatible(SchemaRegistryServiceImpl.java:220) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1] at org.apache.pulsar.broker.service.schema.SchemaRegistryServiceImpl.lambda$checkCompatibilityWithLatest$14(SchemaRegistryServiceImpl.java:229) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1] at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602) ~[?:1.8.0_212] at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577) ~[?:1.8.0_212] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_212] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[?:1.8.0_212] at org.apache.pulsar.broker.service.schema.BookkeeperSchemaStorage.lambda$null$5(BookkeeperSchemaStorage.java:188) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1] at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822) [?:1.8.0_212] at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:797) [?:1.8.0_212] at java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:443) [?:1.8.0_212] at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) [?:1.8.0_212] at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) [?:1.8.0_212] at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) [?:1.8.0_212] at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) [?:1.8.0_212] ``` This is an automated message from the Apache Git Service. To respond to the message, please log o
[pulsar] branch master updated: Fix error message when deleting topic failed (#5228)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new eefe061 Fix error message when deleting topic failed (#5228) eefe061 is described below commit eefe061c1ce4e378a4a1669a4c2a0027a4a1bf54 Author: Masahiro Sakamoto AuthorDate: Fri Sep 20 05:05:29 2019 +0900 Fix error message when deleting topic failed (#5228) --- .../java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java index 71c758b..d7cd44b 100644 --- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java +++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java @@ -588,7 +588,7 @@ public class PersistentTopicsBase extends AdminResource { log.info("[{}] Successfully removed topic {}", clientAppId(), topicName); } catch (Exception e) { Throwable t = e.getCause(); -log.error("[{}] Failed to get delete topic {}", clientAppId(), topicName, t); +log.error("[{}] Failed to delete topic {}", clientAppId(), topicName, t); if (t instanceof TopicBusyException) { throw new RestException(Status.PRECONDITION_FAILED, "Topic has active producers/subscriptions"); } else {
[GitHub] [pulsar] merlimat merged pull request #5228: [pulsar-broker] Fix error message when deleting topic failed
merlimat merged pull request #5228: [pulsar-broker] Fix error message when deleting topic failed URL: https://github.com/apache/pulsar/pull/5228 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: Fixed flaky tests in ReplicatorRateLimiterTest (#5207)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 0bc952d Fixed flaky tests in ReplicatorRateLimiterTest (#5207) 0bc952d is described below commit 0bc952d0c65304929bc81ba8ec0c42a0c91e04d5 Author: Matteo Merli AuthorDate: Thu Sep 19 13:05:44 2019 -0700 Fixed flaky tests in ReplicatorRateLimiterTest (#5207) --- .../pulsar/broker/service/ReplicatorRateLimiterTest.java | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ReplicatorRateLimiterTest.java b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ReplicatorRateLimiterTest.java index dba37ac..3c74ed8 100644 --- a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ReplicatorRateLimiterTest.java +++ b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ReplicatorRateLimiterTest.java @@ -87,7 +87,7 @@ public class ReplicatorRateLimiterTest extends ReplicatorTestBase { public void testReplicatorRateLimiterDynamicallyChange() throws Exception { log.info("--- Starting ReplicatorTest::{} --- ", methodName); -final String namespace = "pulsar/replicatorchange"; +final String namespace = "pulsar/replicatorchange-" + System.currentTimeMillis(); final String topicName = "persistent://" + namespace + "/ratechange"; admin1.namespaces().createNamespace(namespace); @@ -156,11 +156,11 @@ public class ReplicatorRateLimiterTest extends ReplicatorTestBase { * * @throws Exception */ -@Test(dataProvider = "dispatchRateType", timeOut = 5000) +@Test(dataProvider = "dispatchRateType") public void testReplicatorRateLimiterMessageNotReceivedAllMessages(DispatchRateType dispatchRateType) throws Exception { log.info("--- Starting ReplicatorTest::{} --- ", methodName); -final String namespace = "pulsar/replicatorbyteandmsg" + dispatchRateType.toString(); +final String namespace = "pulsar/replicatorbyteandmsg-" + dispatchRateType.toString() + "-" + System.currentTimeMillis(); final String topicName = "persistent://" + namespace + "/notReceivedAll"; admin1.namespaces().createNamespace(namespace); @@ -242,11 +242,11 @@ public class ReplicatorRateLimiterTest extends ReplicatorTestBase { * * @throws Exception */ -@Test(timeOut = 5000) +@Test public void testReplicatorRateLimiterMessageReceivedAllMessages() throws Exception { log.info("--- Starting ReplicatorTest::{} --- ", methodName); -final String namespace = "pulsar/replicatormsg"; +final String namespace = "pulsar/replicatormsg-" + System.currentTimeMillis(); final String topicName = "persistent://" + namespace + "/notReceivedAll"; admin1.namespaces().createNamespace(namespace);
[GitHub] [pulsar] merlimat merged pull request #5207: Fixed flaky tests in ReplicatorRateLimiterTest
merlimat merged pull request #5207: Fixed flaky tests in ReplicatorRateLimiterTest URL: https://github.com/apache/pulsar/pull/5207 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] vzhikserg commented on issue #5205: [doc] Fix broken links to the info about schemas and Schema Registry
vzhikserg commented on issue #5205: [doc] Fix broken links to the info about schemas and Schema Registry URL: https://github.com/apache/pulsar/pull/5205#issuecomment-533281092 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5207: Fixed flaky tests in ReplicatorRateLimiterTest
aahmed-se commented on issue #5207: Fixed flaky tests in ReplicatorRateLimiterTest URL: https://github.com/apache/pulsar/pull/5207#issuecomment-533243879 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #5207: Fixed flaky tests in ReplicatorRateLimiterTest
merlimat commented on issue #5207: Fixed flaky tests in ReplicatorRateLimiterTest URL: https://github.com/apache/pulsar/pull/5207#issuecomment-533234022 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat commented on issue #5228: [pulsar-broker] Fix error message when deleting topic failed
merlimat commented on issue #5228: [pulsar-broker] Fix error message when deleting topic failed URL: https://github.com/apache/pulsar/pull/5228#issuecomment-533233678 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat opened a new issue #5229: Intermittent test failure in PulsarFunctionE2ESecurityTest.testAuthorizationWithAnonymousUser
merlimat opened a new issue #5229: Intermittent test failure in PulsarFunctionE2ESecurityTest.testAuthorizationWithAnonymousUser URL: https://github.com/apache/pulsar/issues/5229 The test is reproducibly failing from time to time due to the subscription cleanup failure and the function api giving up deleing the subscription after N attempts. Full tests logs: https://gist.github.com/merlimat/3cd7102e39be08789d21d18bbbdbc255 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] 0x6e6562 commented on issue #4593: [Start Standalone Error] In the case of wifi connection, /bin/pulsar standalone is not working properly
0x6e6562 commented on issue #4593: [Start Standalone Error] In the case of wifi connection, /bin/pulsar standalone is not working properly URL: https://github.com/apache/pulsar/issues/4593#issuecomment-533208371 I can reproduce these symptoms on OSX by starting prometheus before starting pulsar. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4621: [PIP-38] Support batch receive in java client.
sijie commented on issue #4621: [PIP-38] Support batch receive in java client. URL: https://github.com/apache/pulsar/pull/4621#issuecomment-533187750 @codelipenghui can you rebase the PR to master? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5210: [Doc] Add *Kafka sink connector guide*
Anonymitaet commented on issue #5210: [Doc] Add *Kafka sink connector guide* URL: https://github.com/apache/pulsar/pull/5210#issuecomment-533124169 run c++ tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] MarvinCai commented on issue #5148: Validate topic name before creating partition/non partition topic via admin cli.
MarvinCai commented on issue #5148: Validate topic name before creating partition/non partition topic via admin cli. URL: https://github.com/apache/pulsar/pull/5148#issuecomment-533124037 @jiazhai addressed you comments, can you please take another look, thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5203: [Doc] Add *Kafka source connector guide*
tuteng commented on issue #5203: [Doc] Add *Kafka source connector guide* URL: https://github.com/apache/pulsar/pull/5203#issuecomment-533116662 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie merged pull request #5212: [Doc] Add Node.js Client Docs
sijie merged pull request #5212: [Doc] Add Node.js Client Docs URL: https://github.com/apache/pulsar/pull/5212 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: [Doc] Add Node.js Client Docs (#5212)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 7e63962 [Doc] Add Node.js Client Docs (#5212) 7e63962 is described below commit 7e6396214f0d95581f1ae3b011cafd7611a14a0a Author: Yuri Mizushima AuthorDate: Thu Sep 19 21:43:37 2019 +0900 [Doc] Add Node.js Client Docs (#5212) ## Motivation Currently, There is no official document for [Node.js client](https://github.com/apache/pulsar-client-node). We want to find information about Node.js client from Official docs. ## Modifications Add Node.js client docs. ## Result We can find information about Node.js client. --- site2/docs/client-libraries-node.md | 401 ++ site2/docs/security-tls-authentication.md | 19 ++ site2/docs/security-tls-transport.md | 15 +- site2/website/sidebars.json | 1 + 4 files changed, 435 insertions(+), 1 deletion(-) diff --git a/site2/docs/client-libraries-node.md b/site2/docs/client-libraries-node.md new file mode 100644 index 000..130f7a0 --- /dev/null +++ b/site2/docs/client-libraries-node.md @@ -0,0 +1,401 @@ +--- +id: client-libraries-node +title: The Pulsar Node.js client +sidebar_label: Node.js +--- + +The Pulsar Node.js client can be used to create Pulsar [producers](#producers), [consumers](#consumers), and [readers](#readers) in Node.js. + +## Installation + +You can install the [`pusar-client`](https://www.npmjs.com/package/pulsar-client) library via [npm](https://www.npmjs.com/). + +### Requirements +Pulsar Node.js client library is based on the C++ client library. +Follow [these instructions](client-libraries-cpp.md#compilation) and install the Pulsar C++ client library. + +### Compatibility + +Compatibility between each version of the Node.js client and the C++ client is as follows: + +| Node.js client | C++ client | +| :- | :- | +| 1.0.0 | 2.3.0 or later | + +If an incompatible version of the C++ client is installed, you may fail to build or run this library. + +### Installation using npm + +Install the `pulsar-client` library via [npm](https://www.npmjs.com/): + +```shell +$ npm install pulsar-client +``` + +> Note +> +> Also, this library works only in Node.js 10.x or later because it uses the [`node-addon-api`](https://github.com/nodejs/node-addon-api) module to wrap the C++ library. + +## Connection URLs +To connect to Pulsar using client libraries, you need to specify a [Pulsar protocol](developing-binary-protocol.md) URL. + +Pulsar protocol URLs are assigned to specific clusters, use the `pulsar` scheme and have a default port of 6650. Here is an example for `localhost`: + +```http +pulsar://localhost:6650 +``` + +A URL for a production Pulsar cluster may look something like this: + +```http +pulsar://pulsar.us-west.example.com:6650 +``` + +If you are using [TLS encryption](security-tls-transport.md) or [TLS Authentication](security-tls-authentication.md), the URL will look like something like this: + +```http +pulsar+ssl://pulsar.us-west.example.com:6651 +``` + +## Create a client + +In order to interact with Pulsar, you will first need a client object. You can create a client instance using a `new` operator and the `Client` method, passing in a client options object (more on configuration [below](#client-configuration)). + +Here is an example: + +```JavaScript +const Pulsar = require('pulsar-client'); + +(async () => { + const client = new Pulsar.Client({ +serviceUrl: 'pulsar://localhost:6650', + }); + + await client.close(); +})(); +``` + +### Client configuration + +The following configurable parameters are available for Pulsar clients: + +| Parameter | Description | Default | +| : | :-- | :-- | +| `serviceUrl` | The connection URL for the Pulsar cluster. See [above](#connection-urls) for more info. | | +| `authentication` | Configure the authentication provider. (default: no authentication). See [TLS Authentication](security-tls-authentication.md) for more info. | | +| `operationTimeoutSeconds` | The timeout for Node.js client operations (creating producers, subscribing to and unsubscribing from [topics](reference-terminology.md#topic)). Retries will occur until this threshold is reached, at which point the operation will fail. | 30 | +| `ioThreads` | The number of threads to use for handling connections to Pulsar [brokers](reference-terminology.md#broker). | 1 | +| `messageListenerThreads` | The number of threads used by message listeners ([consumers](#consumers) and [readers](#readers)). | 1 | +| `concurrentLookupRequest` | The number of concurrent lookup requests that can be sent on each broker connection. Setting a maximum helps to keep from overloading brokers. You should set values over the default
[GitHub] [pulsar] sijie commented on issue #5222: 2.4.1 pulsar-admin should have 8 schema compatibility check strategies
sijie commented on issue #5222: 2.4.1 pulsar-admin should have 8 schema compatibility check strategies URL: https://github.com/apache/pulsar/issues/5222#issuecomment-533112112 @pouledodue : you can use `NONE` for `always_compatible` check policy. The other 3 transitive policies are added recently. It seems that the pulsar-admin is not updated to include them. @codelipenghui @congbobo184 can you help with this issue? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5220: how to safely delete bookie of pulsar cluster?
sijie commented on issue #5220: how to safely delete bookie of pulsar cluster? URL: https://github.com/apache/pulsar/issues/5220#issuecomment-533110455 @sydnash can you share the `topic stats-internal` result? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5045: [client] Introduce `batchingMaxBytes` setting in pulsar producer
sijie commented on issue #5045: [client] Introduce `batchingMaxBytes` setting in pulsar producer URL: https://github.com/apache/pulsar/pull/5045#issuecomment-533106430 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: Modify the schema decode method can decode ByteBuf (#5123)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new ac13009 Modify the schema decode method can decode ByteBuf (#5123) ac13009 is described below commit ac13009c5990d99fc935f7b83ad10d4c79772990 Author: congbo <39078850+congbobo...@users.noreply.github.com> AuthorDate: Thu Sep 19 20:25:52 2019 +0800 Modify the schema decode method can decode ByteBuf (#5123) --- .../pulsar/client/api/schema/SchemaReader.java | 25 - .../pulsar/client/impl/schema/AbstractSchema.java | 64 ++ .../pulsar/client/impl/schema/BooleanSchema.java | 12 +++- .../pulsar/client/impl/schema/ByteBufSchema.java | 12 +++- .../client/impl/schema/ByteBufferSchema.java | 18 +- .../pulsar/client/impl/schema/ByteSchema.java | 20 ++- .../pulsar/client/impl/schema/BytesSchema.java | 16 +- .../pulsar/client/impl/schema/DateSchema.java | 14 - .../pulsar/client/impl/schema/DoubleSchema.java| 27 - .../pulsar/client/impl/schema/FloatSchema.java | 26 - .../pulsar/client/impl/schema/IntSchema.java | 27 - .../pulsar/client/impl/schema/LongSchema.java | 26 - .../pulsar/client/impl/schema/ShortSchema.java | 26 - .../pulsar/client/impl/schema/StringSchema.java| 29 +- .../pulsar/client/impl/schema/StructSchema.java| 22 +++- .../pulsar/client/impl/schema/TimeSchema.java | 14 - .../pulsar/client/impl/schema/TimestampSchema.java | 14 - .../impl/schema/generic/GenericAvroReader.java | 30 +- .../impl/schema/generic/GenericJsonReader.java | 27 - .../client/impl/schema/reader/AvroReader.java | 32 ++- .../client/impl/schema/reader/JsonReader.java | 24 +++- .../client/impl/schema/reader/ProtobufReader.java | 26 - .../pulsar/client/impl/schema/AvroSchemaTest.java | 26 + .../client/impl/schema/BooleanSchemaTest.java | 15 - .../pulsar/client/impl/schema/BytesSchemaTest.java | 7 +++ .../pulsar/client/impl/schema/DateSchemaTest.java | 17 +- .../client/impl/schema/DoubleSchemaTest.java | 13 - .../pulsar/client/impl/schema/FloatSchemaTest.java | 17 +- .../pulsar/client/impl/schema/IntSchemaTest.java | 15 - .../pulsar/client/impl/schema/JSONSchemaTest.java | 24 .../pulsar/client/impl/schema/LongSchemaTest.java | 17 +- .../client/impl/schema/PrimitiveSchemaTest.java| 7 ++- .../client/impl/schema/ProtobufSchemaTest.java | 16 ++ .../pulsar/client/impl/schema/ShortSchemaTest.java | 15 - .../client/impl/schema/StringSchemaTest.java | 23 .../pulsar/client/impl/schema/TimeSchemaTest.java | 10 +++- .../client/impl/schema/TimestampSchemaTest.java| 12 +++- 37 files changed, 708 insertions(+), 57 deletions(-) diff --git a/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/schema/SchemaReader.java b/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/schema/SchemaReader.java index 29155cc..d073387 100644 --- a/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/schema/SchemaReader.java +++ b/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/schema/SchemaReader.java @@ -18,9 +18,12 @@ */ package org.apache.pulsar.client.api.schema; +import java.io.InputStream; + /** * Deserialize messages from bytes. */ + public interface SchemaReader { /** @@ -29,5 +32,25 @@ public interface SchemaReader { * @param bytes the data * @return the serialized object */ -T read(byte[] bytes); +default T read(byte[] bytes) { +return read(bytes, 0, bytes.length); +} + +/** + * serialize bytes convert pojo. + * + * @param bytes the data + * @param offset the byte[] initial position + * @param length the byte[] read length + * @return the serialized object + */ +T read(byte[] bytes, int offset, int length); + +/** + * serialize bytes convert pojo. + * + * @param inputStream the stream of message + * @return the serialized object + */ +T read(InputStream inputStream); } diff --git a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/AbstractSchema.java b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/AbstractSchema.java new file mode 100644 index 000..f459d5c --- /dev/null +++ b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/AbstractSchema.java @@ -0,0 +1,64 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License
[GitHub] [pulsar] sijie merged pull request #5123: Modify the schema decode method can decode ByteBuf
sijie merged pull request #5123: Modify the schema decode method can decode ByteBuf URL: https://github.com/apache/pulsar/pull/5123 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (0014f6e -> a4587d2)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 0014f6e [Issue 5050][Docs] Adjust the content structure of the security chapter (#5201) add a4587d2 [doc] Improve Pulsar deployment bare metal multi cluster (#5135) No new revisions were added by this update. Summary of changes: site2/docs/deploy-bare-metal-multi-cluster.md | 185 -- 1 file changed, 83 insertions(+), 102 deletions(-)
[GitHub] [pulsar] sijie merged pull request #5135: [doc] Improve Pulsar deployment bare metal multi cluster
sijie merged pull request #5135: [doc] Improve Pulsar deployment bare metal multi cluster URL: https://github.com/apache/pulsar/pull/5135 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (0bc9dab -> 0014f6e)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 0bc9dab [Doc] Update *Debezium Connector Guide* (#5197) add 0014f6e [Issue 5050][Docs] Adjust the content structure of the security chapter (#5201) No new revisions were added by this update. Summary of changes: site2/docs/security-jwt.md | 240 site2/docs/security-overview.md | 21 ++-- site2/docs/security-token-admin.md | 152 --- site2/docs/security-token-client.md | 122 -- site2/website/sidebars.json | 3 +- 5 files changed, 248 insertions(+), 290 deletions(-) create mode 100644 site2/docs/security-jwt.md delete mode 100644 site2/docs/security-token-admin.md delete mode 100644 site2/docs/security-token-client.md
[GitHub] [pulsar] sijie merged pull request #5201: [Issue 5050][Docs] Adjust the content structure of the security chapter
sijie merged pull request #5201: [Issue 5050][Docs] Adjust the content structure of the security chapter URL: https://github.com/apache/pulsar/pull/5201 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie closed issue #5050: [Doc] 'JSON Web Token Authentication' is not documented in the Security chapter of Pulsar document
sijie closed issue #5050: [Doc] 'JSON Web Token Authentication' is not documented in the Security chapter of Pulsar document URL: https://github.com/apache/pulsar/issues/5050 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5227: [PIP-43] Support producer to send msg with different schema
sijie commented on issue #5227: [PIP-43] Support producer to send msg with different schema URL: https://github.com/apache/pulsar/pull/5227#issuecomment-533104337 @congbobo184 isn't #5165 already doing the same thing? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] vzhikserg commented on issue #5205: [doc] Fix broken links to the info about schemas and Schema Registry
vzhikserg commented on issue #5205: [doc] Fix broken links to the info about schemas and Schema Registry URL: https://github.com/apache/pulsar/pull/5205#issuecomment-533091788 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] massakam opened a new pull request #5228: [pulsar-broker] Fix error message when deleting topic failed
massakam opened a new pull request #5228: [pulsar-broker] Fix error message when deleting topic failed URL: https://github.com/apache/pulsar/pull/5228 There was a mistake in the error message when deleting a topic failed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 opened a new pull request #5227: [PIP-43] Support producer to send msg with different schema
congbobo184 opened a new pull request #5227: [PIP-43] Support producer to send msg with different schema URL: https://github.com/apache/pulsar/pull/5227 Master Issue: #4737 ### Motivation Implement of PIP-44. ### Modifications - Add two flag : 1. schema_auto_update_compatibility_strategy for whether producer automatically creates schema under compatibility checking conditions 2. schema_compatibility_strategy for pulsar schema compatibility check - Consumer and producer has their own schema compatibility check, specific in PIP-44 ### Verifying this change Add the tests for it Does this pull request potentially affect one of the following parts: If yes was chosen, please highlight the changes Dependencies (does it add or upgrade a dependency): (no) The public API: (no) The schema: (yes) The default values of configurations: (no) The wire protocol: (no) The rest endpoints: (no) The admin cli options: (yes) Anything that affects deployment: (no) ### Documentation Does this pull request introduce a new feature? (yes) If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) If a feature is not applicable for documentation, explain why? If a feature is not documented yet in this PR, please create a followup issue for adding the documentation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5221: [doc] Improve Pulsar Administration Pulsar dashboard
Anonymitaet commented on a change in pull request #5221: [doc] Improve Pulsar Administration Pulsar dashboard URL: https://github.com/apache/pulsar/pull/5221#discussion_r326101761 ## File path: site2/docs/administration-dashboard.md ## @@ -21,7 +21,7 @@ $ docker run -p 80:80 \ apachepulsar/pulsar-dashboard:{{pulsar:version}} ``` -The {@inject: github:`Dockerfile`:/dashboard/Dockerfile} can be found in `dashboard` directory, you can build a image from scratch also: +You can find the {@inject: github:`Dockerfile`:/dashboard/Dockerfile} in `dashboard` directory, you can build a image from scratch also: Review comment: ```suggestion You can find the {@inject: github:`Dockerfile`:/dashboard/Dockerfile} in the `dashboard` directory and build an image from scratch as well: ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5221: [doc] Improve Pulsar Administration Pulsar dashboard
Anonymitaet commented on a change in pull request #5221: [doc] Improve Pulsar Administration Pulsar dashboard URL: https://github.com/apache/pulsar/pull/5221#discussion_r326101560 ## File path: site2/docs/administration-dashboard.md ## @@ -8,7 +8,7 @@ The Pulsar dashboard is a web application that enables users to monitor current The dashboard is a data collector that polls stats from all the brokers in a Pulsar instance (across multiple clusters) and stores all the information in a [PostgreSQL](https://www.postgresql.org/) database. -A [Django](https://www.djangoproject.com) web app is used to render the collected data. +You can use a [Django](https://www.djangoproject.com) web app to render the collected data. Review comment: ```suggestion You can use the [Django](https://www.djangoproject.com) web APP to render the collected data. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5225: [doc] Improve Pulsar Administration Pulsar Statistics
Anonymitaet commented on a change in pull request #5225: [doc] Improve Pulsar Administration Pulsar Statistics URL: https://github.com/apache/pulsar/pull/5225#discussion_r326100668 ## File path: site2/docs/administration-stats.md ## @@ -23,41 +23,41 @@ sidebar_label: Pulsar statistics |my-subscription| The name of this subscription (client defined)| |msgBacklog| The count of messages in backlog for this subscription| |type| This subscription type| -|msgRateExpired| The rate at which messages were discarded instead of dispatched from this subscription due to TTL| +|msgRateExpired| The rate at which messages are discarded instead of dispatched from this subscription due to TTL| |consumers| The list of connected consumers for this subscription| |consumerName| Internal identifier for this consumer, generated by the client library| -|availablePermits| The number of messages this consumer has space for in the client library’s listen queue. A value of 0 means the client library’s queue is full and receive() isn’t being called. A nonzero value means this consumer is ready to be dispatched messages.| +|availablePermits| The number of messages this consumer has space for in the listen queue of client library. A value of 0 means the queue of client library is full and receive() is not being called. A nonzero value means this consumer is ready to be dispatched messages.| |replication| This section gives the stats for cross-colo replication of this topic| |replicationBacklog| The outbound replication backlog in messages| |connected| Whether the outbound replicator is connected| |replicationDelayInSeconds| How long the oldest message has been waiting to be sent through the connection, if connected is true| -|inboundConnection| The IP and port of the broker in the remote cluster’s publisher connection to this broker| -|inboundConnectedSince| The TCP connection being used to publish messages to the remote cluster. If there are no local publishers connected, this connection is automatically closed after a minute.| +|inboundConnection| The IP and port of the broker in the publisher connection of remote cluster to this broker | +|inboundConnectedSince| The TCP connection being used to publish messages to the remote cluster. If no local publishers are connected, this connection is automatically closed after a minute.| ## Topics |Stat|Description| |---|---| -|entriesAddedCounter| Messages published since this broker loaded this topic| +|entriesAddedCounter| Messages publishes since this broker loads this topic| Review comment: ```suggestion |entriesAddedCounter| Messages published since this broker loads this topic| ``` messages are published, use passive voice This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5226: [Doc] Add *HDFS2 sink connector guide*
Anonymitaet commented on a change in pull request #5226: [Doc] Add *HDFS2 sink connector guide* URL: https://github.com/apache/pulsar/pull/5226#discussion_r326097129 ## File path: site2/docs/io-hdfs2-sink.md ## @@ -0,0 +1,53 @@ +--- +id: io-hdfs2-sink +title: HDFS2 sink connector +sidebar_label: HDFS2 sink connector +--- + +The HDFS2 sink connector pulls the messages from Pulsar topics +and persists the messages to HDFS files. + +## Configuration + +The configuration of the HDFS2 sink connector has the following properties. + +### Property + +| Name | Type|Required | Default | Description +|--|--|--|-|-| +| `hdfsConfigResources` | String|true| | A file or a comma-separated list containing the Hadoop file system configuration.**Example**'core-site.xml''hdfs-site.xml' | +| `directory` | String | true | |The HDFS directory where files read from or written to. | +| `encoding` | String |false | |The character encoding for the files.**Example**UTF-8ASCII | +| `compression` | Compression |false | |The compression code used to compress or de-compress the files on HDFS. | +| `kerberosUserPrincipal` |String| false| |The principal account of Kerberos user used for authentication. | +| `keytab` | String|false|| The full pathname of the Kerberos keytab file used for authentication. | +| `filenamePrefix` |String| false | |The prefix of the files created inside the HDFS directory.**Example** The value of topicA result in files named topicA-. | +| `fileExtension` | String| false | | The extension added to the files written to HDFS.**Example**'.txt' '.seq' | +| `separator` | char|false | |The character used to separate records in a text file. If no value is provided, the contents from all records are concatenated together in one continuous byte array. | +| `syncInterval` | long| false || The interval between calls to flush data to HDFS disk in milliseconds. | +| `maxPendingRecords` |int| false|Integer.MAX_VALUE | The maximum number of records that hold in memory before acking. Setting this property to 1 makes every record send to disk before the record is acked.Setting this property to a higher value allows buffering records before flushing them to disk. + Review comment: @tuteng the properties from line 18 - 27 does not have default values? (I do not find their default values in [AbstractHdfsConfig.java](https://github.com/apache/pulsar/blob/master/pulsar-io/hdfs2/src/main/java/org/apache/pulsar/io/hdfs2/AbstractHdfsConfig.java)) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet edited a comment on issue #5015: [Doc] Update Pulsar Connector Guide
Anonymitaet edited a comment on issue #5015: [Doc] Update Pulsar Connector Guide URL: https://github.com/apache/pulsar/issues/5015#issuecomment-524149120 I've created the following PRs to resolve this issue: ## Overview * https://github.com/apache/pulsar/pull/5183 * https://github.com/apache/pulsar/pull/5224 ## Use * https://github.com/apache/pulsar/pull/5071 ## Built-in connector * https://github.com/apache/pulsar/pull/5168 * https://github.com/apache/pulsar/pull/5173 * https://github.com/apache/pulsar/pull/5197 * https://github.com/apache/pulsar/pull/5199 * https://github.com/apache/pulsar/pull/5203 * https://github.com/apache/pulsar/pull/5210 * https://github.com/apache/pulsar/pull/5211 * https://github.com/apache/pulsar/pull/5214 * https://github.com/apache/pulsar/pull/5226 ## CDC connector * https://github.com/apache/pulsar/pull/5198 ## Debug * https://github.com/apache/pulsar/pull/5041 ## Connector Admin CLI * https://github.com/apache/pulsar/pull/5028 * https://github.com/apache/pulsar/pull/5184 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] iravishah commented on issue #5206: TooLongFrameException : Adjusted frame length exceeds 5253120: 369295620 - discarded => Error while producing message with TLS on standalone setup
iravishah commented on issue #5206: TooLongFrameException : Adjusted frame length exceeds 5253120: 369295620 - discarded => Error while producing message with TLS on standalone setup URL: https://github.com/apache/pulsar/issues/5206#issuecomment-533065465 I have added `brokerServicePortTls=6651` and now i am getting `ERROR org.apache.pulsar.common.util.SslContextAutoRefreshBuilder - Execption while trying to refresh ssl Context Private key loading error` while starting standalone pulsar This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet opened a new pull request #5226: [Doc] Add *HDFS2 sink connector guide*
Anonymitaet opened a new pull request #5226: [Doc] Add *HDFS2 sink connector guide* URL: https://github.com/apache/pulsar/pull/5226 Fix https://github.com/apache/pulsar/issues/5015 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] iravishah edited a comment on issue #5206: TooLongFrameException : Adjusted frame length exceeds 5253120: 369295620 - discarded => Error while producing message with TLS on standalon
iravishah edited a comment on issue #5206: TooLongFrameException : Adjusted frame length exceeds 5253120: 369295620 - discarded => Error while producing message with TLS on standalone setup URL: https://github.com/apache/pulsar/issues/5206#issuecomment-533045354 @sijie I have followed [this](https://pulsar.apache.org/docs/en/security-tls-transport/) guide to generate certs for CA and Server. **Standalone config** `brokerServicePort=6651 webServicePort=8443 tlsEnabled=true tlsCertificateFilePath=/path/to/broker.cert.pem tlsKeyFilePath=/home/path/to/broker.key-pk8.pem tlsTrustCertsFilePath=/path/to/ca.cert.pem` **Client config** `useTls=true webServiceUrl=http://localhost:8443/ brokerServiceUrl=pulsar+ssl://localhost:6651/ authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls authParams=tlsCertFile:/path/to/broker.cert.pem,tlsKeyFile:/path/to/broker.key-pk8.pem tlsAllowInsecureConnection=false tlsEnableHostnameVerification=false tlsTrustCertsFilePath=/path/to/ca.cert.pem` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] iravishah edited a comment on issue #5206: TooLongFrameException : Adjusted frame length exceeds 5253120: 369295620 - discarded => Error while producing message with TLS on standalon
iravishah edited a comment on issue #5206: TooLongFrameException : Adjusted frame length exceeds 5253120: 369295620 - discarded => Error while producing message with TLS on standalone setup URL: https://github.com/apache/pulsar/issues/5206#issuecomment-533045354 @sijie I have followed [this](https://pulsar.apache.org/docs/en/security-tls-transport/) guide to generate certs for CA and Server. **Standalone config** ` brokerServicePort=6651 webServicePort=8443 tlsEnabled=true tlsCertificateFilePath=/path/to/broker.cert.pem tlsKeyFilePath=/home/path/to/broker.key-pk8.pem tlsTrustCertsFilePath=/path/to/ca.cert.pem` **Client config** ` useTls=true webServiceUrl=http://localhost:8443/ brokerServiceUrl=pulsar+ssl://localhost:6651/ authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls authParams=tlsCertFile:/path/to/broker.cert.pem,tlsKeyFile:/path/to/broker.key-pk8.pem tlsAllowInsecureConnection=false tlsEnableHostnameVerification=false tlsTrustCertsFilePath=/path/to/ca.cert.pem` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] iravishah commented on issue #5206: TooLongFrameException : Adjusted frame length exceeds 5253120: 369295620 - discarded => Error while producing message with TLS on standalone setup
iravishah commented on issue #5206: TooLongFrameException : Adjusted frame length exceeds 5253120: 369295620 - discarded => Error while producing message with TLS on standalone setup URL: https://github.com/apache/pulsar/issues/5206#issuecomment-533045354 @sijie I have followed [this](https://pulsar.apache.org/docs/en/security-tls-transport/) guide to generate certs for CA and Server. **Standalone config** `brokerServicePort=6651 webServicePort=8443 tlsEnabled=true tlsCertificateFilePath=/path/to/broker.cert.pem tlsKeyFilePath=/home/path/to/broker.key-pk8.pem tlsTrustCertsFilePath=/path/to/ca.cert.pem` **Client config** `useTls=true webServiceUrl=http://localhost:8443/ brokerServiceUrl=pulsar+ssl://localhost:6651/ authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls authParams=tlsCertFile:/path/to/broker.cert.pem,tlsKeyFile:/path/to/broker.key-pk8.pem tlsAllowInsecureConnection=false tlsEnableHostnameVerification=false tlsTrustCertsFilePath=/path/to/ca.cert.pem` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #5224: [Doc] Add *processing guarantee for connector*
Anonymitaet commented on issue #5224: [Doc] Add *processing guarantee for connector* URL: https://github.com/apache/pulsar/pull/5224#issuecomment-533042955 @tuteng could you please help review? Thank you This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Monica-zy commented on issue #5225: [doc] Improve Pulsar Administration Pulsar Statistics
Monica-zy commented on issue #5225: [doc] Improve Pulsar Administration Pulsar Statistics URL: https://github.com/apache/pulsar/pull/5225#issuecomment-533040143 @Anonymitaet please review~ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet edited a comment on issue #5015: [Doc] Update Pulsar Connector Guide
Anonymitaet edited a comment on issue #5015: [Doc] Update Pulsar Connector Guide URL: https://github.com/apache/pulsar/issues/5015#issuecomment-524149120 I've created the following PRs to resolve this issue: ## Overview * https://github.com/apache/pulsar/pull/5183 * https://github.com/apache/pulsar/pull/5224 ## Use * https://github.com/apache/pulsar/pull/5071 ## Built-in connector * https://github.com/apache/pulsar/pull/5168 * https://github.com/apache/pulsar/pull/5173 * https://github.com/apache/pulsar/pull/5197 * https://github.com/apache/pulsar/pull/5199 * https://github.com/apache/pulsar/pull/5203 * https://github.com/apache/pulsar/pull/5210 * https://github.com/apache/pulsar/pull/5211 * https://github.com/apache/pulsar/pull/5214 ## CDC connector * https://github.com/apache/pulsar/pull/5198 ## Debug * https://github.com/apache/pulsar/pull/5041 ## Connector Admin CLI * https://github.com/apache/pulsar/pull/5028 * https://github.com/apache/pulsar/pull/5184 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet edited a comment on issue #5015: [Doc] Update Pulsar Connector Guide
Anonymitaet edited a comment on issue #5015: [Doc] Update Pulsar Connector Guide URL: https://github.com/apache/pulsar/issues/5015#issuecomment-524149120 I've created the following PRs to resolve this issue: ## Overview * https://github.com/apache/pulsar/pull/5183 ## Use * https://github.com/apache/pulsar/pull/5071 ## Built-in connector * https://github.com/apache/pulsar/pull/5168 * https://github.com/apache/pulsar/pull/5173 * https://github.com/apache/pulsar/pull/5197 * https://github.com/apache/pulsar/pull/5199 * https://github.com/apache/pulsar/pull/5203 * https://github.com/apache/pulsar/pull/5210 * https://github.com/apache/pulsar/pull/5211 * https://github.com/apache/pulsar/pull/5214 * https://github.com/apache/pulsar/pull/5224 ## CDC connector * https://github.com/apache/pulsar/pull/5198 ## Debug * https://github.com/apache/pulsar/pull/5041 ## Connector Admin CLI * https://github.com/apache/pulsar/pull/5028 * https://github.com/apache/pulsar/pull/5184 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Monica-zy opened a new pull request #5225: [doc] Improve Pulsar Administration Pulsar Statistics
Monica-zy opened a new pull request #5225: [doc] Improve Pulsar Administration Pulsar Statistics URL: https://github.com/apache/pulsar/pull/5225 Motivation Improve the language and the overall descriptive style of the Pulsar Administration Pulsar statistics document (Administration-stats section): http://pulsar.apache.org/docs/en/administration-stats/ Modifications Adjust the tone, personal pronouns, voice also some typo errors of some sentences in the document. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet opened a new pull request #5224: [Doc] Add *processing guarantee for connector*
Anonymitaet opened a new pull request #5224: [Doc] Add *processing guarantee for connector* URL: https://github.com/apache/pulsar/pull/5224 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet opened a new issue #5223: [Doc] Add configs for broker
Anonymitaet opened a new issue #5223: [Doc] Add configs for broker URL: https://github.com/apache/pulsar/issues/5223 Document features related to auto topic creation (https://github.com/apache/pulsar/pull/4963) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] mileschao commented on issue #64: producer pending queue peek last with empty queue cause panic
mileschao commented on issue #64: producer pending queue peek last with empty queue cause panic URL: https://github.com/apache/pulsar-client-go/issues/64#issuecomment-533037384 comment on interface [PeekLast](https://github.com/apache/pulsar-client-go/blob/7a9aa2b36371437b1c409cb21cb2056ded7f198d/util/blocking_queue.go#L40) > PeekLast return last item in queue without dequeing, return nil if queue is empty @merlimat This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] mileschao opened a new issue #64: producer pending queue peek last with empty queue cause panic
mileschao opened a new issue #64: producer pending queue peek last with empty queue cause panic URL: https://github.com/apache/pulsar-client-go/issues/64 Expected behavior [pending queue peek last operator](https://github.com/apache/pulsar-client-go/blob/11755b26da59f58c2675708af1470fe493d92e37/pulsar/impl_partition_producer.go#L296) should deal with return value **nil** Actual behavior case panic: > panic: interface conversion: interface {} is nil, not *pulsar.pendingItem Steps to reproduce this is code error System configuration **Pulsar version**: master branch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] nkurihar commented on issue #5212: [Doc] Add Node.js Client Docs
nkurihar commented on issue #5212: [Doc] Add Node.js Client Docs URL: https://github.com/apache/pulsar/pull/5212#issuecomment-533032674 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] vzhikserg commented on issue #5205: [doc] Fix broken links to the info about schemas and Schema Registry
vzhikserg commented on issue #5205: [doc] Fix broken links to the info about schemas and Schema Registry URL: https://github.com/apache/pulsar/pull/5205#issuecomment-533027212 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] pouledodue closed issue #5208: Schema modification question
pouledodue closed issue #5208: Schema modification question URL: https://github.com/apache/pulsar/issues/5208 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] pouledodue opened a new issue #5222: 2.4.1 pulsar-admin should have 8 schema compat check strategy
pouledodue opened a new issue #5222: 2.4.1 pulsar-admin should have 8 schema compat check strategy URL: https://github.com/apache/pulsar/issues/5222 **Describe the bug** 2.4.1 `pulsar-admin` should have 8 schema compat check strategy, it only has 3. **To Reproduce** Steps to reproduce the behavior: ``` wget -c https://archive.apache.org/dist/pulsar/pulsar-2.4.1/apache-pulsar-2.4.1-bin.tar.gz tar mxvfz apache-pulsar-2.4.1-bin.tar.gz ./apache-pulsar-2.4.1/bin/pulsar-admin tenants create a ./apache-pulsar-2.4.1/bin/pulsar-admin namespaces create a/b ./apache-pulsar-2.4.1/bin/pulsar-admin namespaces set-schema-autoupdate-strategy --compatibility always_compatible a/b ``` gives: ``` null Reason: Either --compatibility or --disabled must be specified ``` **Expected behavior** A change of strategy **Screenshots** If applicable, add screenshots to help explain your problem. **Desktop (please complete the following information):** - OS: catalina This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] ChangWinde commented on issue #4721: Flink pulsar sink NotSerializableException
ChangWinde commented on issue #4721: Flink pulsar sink NotSerializableException URL: https://github.com/apache/pulsar/issues/4721#issuecomment-533009234 @vruc @idantony thanks for your help This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Monica-zy commented on issue #5221: [doc] Improve Pulsar Administration Pulsar dashboard
Monica-zy commented on issue #5221: [doc] Improve Pulsar Administration Pulsar dashboard URL: https://github.com/apache/pulsar/pull/5221#issuecomment-533004596 @Anonymitaet please review it, thank you~ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Monica-zy opened a new pull request #5221: [doc] Improve Pulsar Administration Pulsar dashboard
Monica-zy opened a new pull request #5221: [doc] Improve Pulsar Administration Pulsar dashboard URL: https://github.com/apache/pulsar/pull/5221 Motivation Improve the language and the overall descriptive style of the Pulsar Administration Pulsar dashboard document (Administration-dashboard section): http://pulsar.apache.org/docs/en/administration-dashboard/ Modifications Adjust the tone, personal pronouns, voice also some typo errors of some sentences in the document. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services