[GitHub] [pulsar] ntysdd opened a new pull request #5776: fix potential NPE
ntysdd opened a new pull request #5776: fix potential NPE URL: https://github.com/apache/pulsar/pull/5776 *(If this PR fixes a github issue, please add `Fixes #`.)* Fixes # *(or if this PR is one task of a github issue, please add `Master Issue: #` to link to the master issue.)* Master Issue: # ### Motivation fix potential NPE ### Modifications local changes ### Verifying this change - [ ] Make sure that the change passes the CI checks. This change is a trivial rework / code cleanup without any test coverage. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5764: [docs] Fix link issue for "Authentication and authorization in Pulsar"
Jennifer88huang commented on issue #5764: [docs] Fix link issue for "Authentication and authorization in Pulsar" URL: https://github.com/apache/pulsar/pull/5764#issuecomment-559920068 I was wondering why you close this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5764: [docs] Fix link issue for "Authentication and authorization in Pulsar"
Jennifer88huang commented on issue #5764: [docs] Fix link issue for "Authentication and authorization in Pulsar" URL: https://github.com/apache/pulsar/pull/5764#issuecomment-559919997 @belinda-wong In most cases, we only edit files in the master version(in the `./site2/docs` folder), and the changes will take effect in the next release. Our main purpose is to maintain the master version. If it's an important and urgent issue, and you want to apply it in the current version, for example, v2.4.1, you should apply the changes in the `./site2/website/versioned_docs/version-2.4.1` folder. That's why we create `versioned_docs` folder, and there are less files in those folders than that in the master. The changes you've made involves the files in the master and in `version-2.4.1`, if you want to apply it in v2.4.1, you should update the file in the `version-2.4.1` folder as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5771: [docs] Fix link issue for "Authentication and authorization in Pulsar"
Jennifer88huang commented on issue #5771: [docs] Fix link issue for "Authentication and authorization in Pulsar" URL: https://github.com/apache/pulsar/pull/5771#issuecomment-559918973 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5767: Support batch authorization of partitioned topic
tuteng commented on issue #5767: Support batch authorization of partitioned topic URL: https://github.com/apache/pulsar/pull/5767#issuecomment-559918478 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie closed issue #5686: NPE in RackwareEnsemblePlacementPolicyImpl.
sijie closed issue #5686: NPE in RackwareEnsemblePlacementPolicyImpl. URL: https://github.com/apache/pulsar/issues/5686 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5686: NPE in RackwareEnsemblePlacementPolicyImpl.
sijie commented on issue #5686: NPE in RackwareEnsemblePlacementPolicyImpl. URL: https://github.com/apache/pulsar/issues/5686#issuecomment-559915925 Close this issue since it is a dup of apache/bookkeeper#2208 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5713: avoid unsafe split when validate hostname which might be ipv6 address
sijie commented on issue #5713: avoid unsafe split when validate hostname which might be ipv6 address URL: https://github.com/apache/pulsar/pull/5713#issuecomment-559915771 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (035743f -> f0b34e7)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from 035743f [CLI] Add a broker tool for operations of a specific broker (#5768) add f0b34e7 Bump Lombok version to allow building with a JDK12>=12 (#5772) (#5773) No new revisions were added by this update. Summary of changes: pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[GitHub] [pulsar] sijie merged pull request #5773: Fixes #5772: Bump Lombok version to allow building with a JDK>=12
sijie merged pull request #5773: Fixes #5772: Bump Lombok version to allow building with a JDK>=12 URL: https://github.com/apache/pulsar/pull/5773 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5765: add Murmur3_32Hash private constructor
sijie commented on issue #5765: add Murmur3_32Hash private constructor URL: https://github.com/apache/pulsar/pull/5765#issuecomment-559915745 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie closed issue #5772: Bump Lombok version to allow building with a JDK >=12
sijie closed issue #5772: Bump Lombok version to allow building with a JDK >=12 URL: https://github.com/apache/pulsar/issues/5772 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5774: [website]Fix pulsar-manager download link
sijie commented on issue #5774: [website]Fix pulsar-manager download link URL: https://github.com/apache/pulsar/pull/5774#issuecomment-559915654 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5147: Deprecate getstatus subcommand for source command
sijie commented on issue #5147: Deprecate getstatus subcommand for source command URL: https://github.com/apache/pulsar/pull/5147#issuecomment-559915574 run cpp tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5635: [Issue 5612] Use Reactivestreams client for Mongo Sink
sijie commented on issue #5635: [Issue 5612] Use Reactivestreams client for Mongo Sink URL: https://github.com/apache/pulsar/pull/5635#issuecomment-559915552 run cpp tests run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5702: [pulsar-sql]Expose configurations of managed ledger and bookkeeper client.
sijie commented on issue #5702: [pulsar-sql]Expose configurations of managed ledger and bookkeeper client. URL: https://github.com/apache/pulsar/pull/5702#issuecomment-559915535 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5734: [Reopen][Issue 5597]retry when getPartitionedTopicMetadata failed
sijie commented on issue #5734: [Reopen][Issue 5597]retry when getPartitionedTopicMetadata failed URL: https://github.com/apache/pulsar/pull/5734#issuecomment-559915516 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5771: [docs] Fix link issue for "Authentication and authorization in Pulsar"
sijie commented on issue #5771: [docs] Fix link issue for "Authentication and authorization in Pulsar" URL: https://github.com/apache/pulsar/pull/5771#issuecomment-559915478 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5758: Bump jackson libraries to 2.10.1
sijie commented on issue #5758: Bump jackson libraries to 2.10.1 URL: https://github.com/apache/pulsar/pull/5758#issuecomment-559915461 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (e4fc609 -> 035743f)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git.
from e4fc609 [website]Add content of pulsar-manager in website(include
release notes, download link) (#5741)
add 035743f [CLI] Add a broker tool for operations of a specific broker
(#5768)
No new revisions were added by this update.
Summary of changes:
bin/pulsar | 3 +
pom.xml| 6 +
pulsar-broker/pom.xml | 5 +
.../pulsar/broker/loadbalance/BrokerHostUsage.java | 11 +-
.../impl/GenericBrokerHostUsageImpl.java | 34 +++---
.../loadbalance/impl/LinuxBrokerHostUsageImpl.java | 25 ++--
.../org/apache/pulsar/broker/tools/BrokerTool.java | 47
.../pulsar/broker/tools/LoadReportCommand.java | 127 +
.../broker/{protocol => tools}/package-info.java | 4 +-
9 files changed, 232 insertions(+), 30 deletions(-)
create mode 100644
pulsar-broker/src/main/java/org/apache/pulsar/broker/tools/BrokerTool.java
create mode 100644
pulsar-broker/src/main/java/org/apache/pulsar/broker/tools/LoadReportCommand.java
copy pulsar-broker/src/main/java/org/apache/pulsar/broker/{protocol =>
tools}/package-info.java (91%)
[GitHub] [pulsar] sijie opened a new issue #5775: [documentation] Document how to use broker-tool
sijie opened a new issue #5775: [documentation] Document how to use broker-tool URL: https://github.com/apache/pulsar/issues/5775 #5768 introduced a broker tool for troubleshooting broker-side issues such as load reporting. This issue is for tracking adding the documentation for broker-tool. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie merged pull request #5768: [CLI] Add a broker tool for operations of a specific broker
sijie merged pull request #5768: [CLI] Add a broker tool for operations of a specific broker URL: https://github.com/apache/pulsar/pull/5768 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5767: Support batch authorization of partitioned topic
sijie commented on issue #5767: Support batch authorization of partitioned topic URL: https://github.com/apache/pulsar/pull/5767#issuecomment-559915278 > I thought a draft pr would not be visible before. @zhaohaidao it is a good practice to add description (even it is a draft PR), so that people in the community knows the context of the pull request. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5642: Add Github workflow for gated checkin
aahmed-se commented on issue #5642: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5642#issuecomment-559912408 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5642: Add Github workflow for gated checkin
aahmed-se commented on issue #5642: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5642#issuecomment-559908229 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5642: Add Github workflow for gated checkin
aahmed-se commented on issue #5642: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5642#issuecomment-559907484 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5642: Add Github workflow for gated checkin
aahmed-se commented on issue #5642: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5642#issuecomment-559906331 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] zhaohaidao commented on issue #5739: Use message bytes instead of message number for producer pending messages
zhaohaidao commented on issue #5739: Use message bytes instead of message number for producer pending messages URL: https://github.com/apache/pulsar/issues/5739#issuecomment-559904567 Is there someone working on it? If not, I want to have a try. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] zhaohaidao commented on issue #5767: Support batch authorization of partitioned topic
zhaohaidao commented on issue #5767: Support batch authorization of partitioned topic URL: https://github.com/apache/pulsar/pull/5767#issuecomment-559904108 @tuteng Pls take a look at this pr. It looks like the failed case has nothing to do with my changes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] zhaohaidao commented on issue #5767: Support batch authorization of partitioned topic
zhaohaidao commented on issue #5767: Support batch authorization of partitioned topic URL: https://github.com/apache/pulsar/pull/5767#issuecomment-559903519 > @zhaohaidao can you update the description of the pull request? so the reviewers can understand what is the motivation of this pull request? @sijie thanks for reminding. I thought a draft pr would not be visible before. I have updated the descriptions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] belinda-wong commented on issue #5764: [docs] Fix link issue for "Authentication and authorization in Pulsar"
belinda-wong commented on issue #5764: [docs] Fix link issue for "Authentication and authorization in Pulsar" URL: https://github.com/apache/pulsar/pull/5764#issuecomment-559903381 ok, noted. so the edit should be applied to both the master version and the respective version (e.g. 2.4.1). is it correct? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5642: Add Github workflow for gated checkin
aahmed-se commented on issue #5642: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5642#issuecomment-559903217 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] alphashaw edited a comment on issue #5770: Pulsar Broker Sometimes does not return CommandSendReceipt/CommandSendError to prducer send CommandSend
alphashaw edited a comment on issue #5770: Pulsar Broker Sometimes does not return CommandSendReceipt/CommandSendError to prducer send CommandSend URL: https://github.com/apache/pulsar/issues/5770#issuecomment-559899675 @jiazhai Thanks for the swift response. It seems that my last questioned wasn't clear semantically. Let me rephrase it: > From the client end, I understand multiple producers (partitioned/not) might be using the same underlying TCP connection. Also I understand the **sequence_id** (for necessary commands) for each producer using the same connection must be unique. My question is, for these client connection-shared producers, does the broker serially handles their requests? In order words, if **p1** and **p2** represents two arbitrary producers sharing the same client connection and **(req1, res1),(req2, res2)** are at some point represents their request and response pairs respectively, if **req1 < req2** in time order, is it guaranteed that **res1 < res2** ?? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5642: Add Github workflow for gated checkin
aahmed-se commented on issue #5642: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5642#issuecomment-559899762 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5642: Add Github workflow for gated checkin
aahmed-se commented on issue #5642: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5642#issuecomment-559899733 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] alphashaw commented on issue #5770: Pulsar Broker Sometimes does not return CommandSendReceipt/CommandSendError to prducer send CommandSend
alphashaw commented on issue #5770: Pulsar Broker Sometimes does not return CommandSendReceipt/CommandSendError to prducer send CommandSend URL: https://github.com/apache/pulsar/issues/5770#issuecomment-559899675 @jiazhai Thanks for the swift response. It seems that my last questioned wasn't clear semantically. Let me rephrase it: > From the client end, I understand multiple producers (partitioned/not) might be using the same underlying TCP connection. Also I understand the **sequence_id** (for necessary commands) for each producer using the same connection must be unique. My question is, for these client connection-shared producers, does the broker serially handles their requests? In order words, if **p1** and **p2** represents two arbitrary producers sharing the same client connection and **(req1, res1),(req2, res2)** are at some point their request and response pairs respectively, if req1 < req2 in time order, is it guaranteed that res1 < res2 ?? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5774: [website]Fix pulsar-manager download link
codelipenghui commented on issue #5774: [website]Fix pulsar-manager download link URL: https://github.com/apache/pulsar/pull/5774#issuecomment-559893305 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5768: [CLI] Add a broker tool for operations of a specific broker
codelipenghui commented on issue #5768: [CLI] Add a broker tool for operations of a specific broker URL: https://github.com/apache/pulsar/pull/5768#issuecomment-559893200 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5680: [transaction-coordinator] Implementation of transaction coordinator client.
codelipenghui commented on issue #5680: [transaction-coordinator] Implementation of transaction coordinator client. URL: https://github.com/apache/pulsar/pull/5680#issuecomment-559893067 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service
codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service URL: https://github.com/apache/pulsar/pull/4955#issuecomment-559892857 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] cckellogg opened a new pull request #109: Fix subscriber bug and use Consumer interface for multi topic consumer.
cckellogg opened a new pull request #109: Fix subscriber bug and use Consumer interface for multi topic consumer. URL: https://github.com/apache/pulsar-client-go/pull/109 ### Motivation Fixes a bug in the subscriber function and converts the multi topic consumer to use the subscriber function. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar-client-go] cckellogg commented on a change in pull request #108: [Issue #107] SetWriteDeadline on connection
cckellogg commented on a change in pull request #108: [Issue #107]
SetWriteDeadline on connection
URL: https://github.com/apache/pulsar-client-go/pull/108#discussion_r352248292
##
File path: pulsar/internal/connection.go
##
@@ -309,7 +309,13 @@ func (c *connection) WriteData(data []byte) {
func (c *connection) internalWriteData(data []byte) {
c.log.Debug("Write data: ", len(data))
- if _, err := c.cnx.Write(data); err != nil {
+ err := c.cnx.SetWriteDeadline(time.Now().Add(time.Second * 15))
Review comment:
Can we make the timeout a variable and not hardcoded so we can make it
configurable in the future if needed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-go] cckellogg commented on a change in pull request #108: [Issue #107] SetWriteDeadline on connection
cckellogg commented on a change in pull request #108: [Issue #107]
SetWriteDeadline on connection
URL: https://github.com/apache/pulsar-client-go/pull/108#discussion_r352248292
##
File path: pulsar/internal/connection.go
##
@@ -309,7 +309,13 @@ func (c *connection) WriteData(data []byte) {
func (c *connection) internalWriteData(data []byte) {
c.log.Debug("Write data: ", len(data))
- if _, err := c.cnx.Write(data); err != nil {
+ err := c.cnx.SetWriteDeadline(time.Now().Add(time.Second * 15))
Review comment:
Should this be set once per connection or once per write? Also, this should
be a variable and not hardcoded so we can make it configurable in the future if
needed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5642: Add Github workflow for gated checkin
aahmed-se commented on issue #5642: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5642#issuecomment-559880958 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5642: Add Github workflow for gated checkin
aahmed-se commented on issue #5642: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5642#issuecomment-559866538 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5642: Add Github workflow for gated checkin
aahmed-se commented on issue #5642: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5642#issuecomment-559866507 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Lanayx commented on issue #5384: Topic compaction fails with 500 error
Lanayx commented on issue #5384: Topic compaction fails with 500 error URL: https://github.com/apache/pulsar/issues/5384#issuecomment-559815992 @zymap thank you, for checking this out. I don't remember doing anything with cluster, but today I've reinstalled it from ground and the error doesn't come up, so I'm closing the ticket. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Lanayx closed issue #5384: Topic compaction fails with 500 error
Lanayx closed issue #5384: Topic compaction fails with 500 error URL: https://github.com/apache/pulsar/issues/5384 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5774: [website]Fix pulsar-manager download link
tuteng commented on issue #5774: [website]Fix pulsar-manager download link URL: https://github.com/apache/pulsar/pull/5774#issuecomment-559814084 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5771: [docs] Fix link issue for "Authentication and authorization in Pulsar"
Jennifer88huang commented on issue #5771: [docs] Fix link issue for "Authentication and authorization in Pulsar" URL: https://github.com/apache/pulsar/pull/5771#issuecomment-559809037 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5764: [docs] Fix link issue for "Authentication and authorization in Pulsar"
Jennifer88huang commented on issue #5764: [docs] Fix link issue for "Authentication and authorization in Pulsar" URL: https://github.com/apache/pulsar/pull/5764#issuecomment-559808943 You can edit the link in this file as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rohts-patil commented on issue #3115: JSONSchema should works with generic (parameterized type) POJO/DTO.
rohts-patil commented on issue #3115: JSONSchema should works with generic (parameterized type) POJO/DTO. URL: https://github.com/apache/pulsar/issues/3115#issuecomment-559806629 @vamsi360 , I cloned your fork and built it to version 1.8.3-ppe-9.10. I still get the errror. ``` Caused by: org.apache.avro.AvroTypeException: Unknown type: T at org.apache.avro.specific.SpecificData.createSchema(SpecificData.java:312) at org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:720) at org.apache.avro.reflect.ReflectData.createFieldSchema(ReflectData.java:870) at org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:686) at org.apache.avro.specific.SpecificData$2.load(SpecificData.java:238) at org.apache.avro.specific.SpecificData$2.load(SpecificData.java:235) at avro.shaded.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568) at avro.shaded.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) at avro.shaded.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) at avro.shaded.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) ... 6 more ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Geal commented on issue #5720: authorize tenant level and namespace level access from the authorization provider
Geal commented on issue #5720: authorize tenant level and namespace level access from the authorization provider URL: https://github.com/apache/pulsar/issues/5720#issuecomment-559803206 I agree with the thread, the current access levels are problematic for multi tenant systems. The underlying point I'd like to add is that there should be check for individual operations or resources, not just for super user, tenant admin (we already have the `canProduceAsync` function and others that work like that). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Geal commented on issue #5718: [Issue 5712][broker] Auth provider superuser
Geal commented on issue #5718: [Issue 5712][broker] Auth provider superuser URL: https://github.com/apache/pulsar/pull/5718#issuecomment-559802132 thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] NotBad4U commented on issue #4055: could not determine kind of name for C.pulsar_configure_set_negative_ack_redelivery_delay_ms
NotBad4U commented on issue #4055: could not determine kind of name for C.pulsar_configure_set_negative_ack_redelivery_delay_ms URL: https://github.com/apache/pulsar/issues/4055#issuecomment-559794336 Hello ! Any update on this one ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar-manager.wiki] branch master updated: Updated Release process (markdown)
This is an automated email from the ASF dual-hosted git repository. guangning pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-manager.wiki.git The following commit(s) were added to refs/heads/master by this push: new 78ec3f1 Updated Release process (markdown) 78ec3f1 is described below commit 78ec3f13a2f98bb01613137d8d96d6929f7d4756 Author: tuteng AuthorDate: Fri Nov 29 21:35:41 2019 +0800 Updated Release process (markdown) --- Release-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Release-process.md b/Release-process.md index 576d6d6..d89f305 100644 --- a/Release-process.md +++ b/Release-process.md @@ -228,7 +228,7 @@ For Pulsar Manager release downloads, visit: http://pulsar.apache.org/en/download/ Release Notes are at: -https://github.com/apache/pulsar-manager/releases +http://pulsar.apache.org/en/pulsar-manager-release-notes/ We would like to thank the contributors that made the release possible.
[GitHub] [pulsar] tuteng opened a new pull request #5774: [website]Fix pulsar-manager download link
tuteng opened a new pull request #5774: [website]Fix pulsar-manager download link URL: https://github.com/apache/pulsar/pull/5774 ### Motivation Website download link error ### Modifications * Fix package of pulsar-manager link. Add version in the path. ### Verifying this change yarn pass This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service
codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service URL: https://github.com/apache/pulsar/pull/4955#issuecomment-559769243 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5680: [transaction-coordinator] Implementation of transaction coordinator client.
codelipenghui commented on issue #5680: [transaction-coordinator] Implementation of transaction coordinator client. URL: https://github.com/apache/pulsar/pull/5680#issuecomment-559768858 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] belinda-wong commented on issue #5764: [docs] Fix link issue for "Authentication and authorization in Pulsar"
belinda-wong commented on issue #5764: [docs] Fix link issue for "Authentication and authorization in Pulsar" URL: https://github.com/apache/pulsar/pull/5764#issuecomment-559766874 > @belinda-wong You don't need to close this one, you can modify directly in this PR, just as you've done in #5771. > > 1. We can modify the PR title and make it more clear to distinguish. > 2. If this occurs in version 2.4.1, we should modify the content in versioned-2.4.1 as well. Is there any changes I should make further? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc
Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#discussion_r352093121 ## File path: site2/docs/schema-understand.md ## @@ -539,8 +539,18 @@ This diagram illustrates how does schema work on the Producer side. 3. The broker looks up the schema in the schema storage to check if it is already a registered schema. 4. If yes, the broker skips the schema validation since it is a known schema, and returns the schema version to the producer. + +5. If no, the broker verifies whether a schema can be automatically created in this namespace: + + * If `isAllowAutoUpdateSchema` sets to **true**, then a schema can be created, and the broker validates the schema based on the schema compatibility check strategy defined for the topic. + + * If `isAllowAutoUpdateSchema` sets to **false**, then a schema can not be created, and the consumer is rejected to connect to the broker. -5. If no, the broker validates the schema based on the schema compatibility check strategy defined for the topic. +**Tip**: + +`isAllowAutoUpdateSchema` can be set via **Pulsar admin API** or **REST API.** + +For how to set `isAllowAutoUpdateSchema` via Pulsar admin API, see [Manage AutoUpdate Strategy](schema-manage.md/#manage-autoupdate-strategy). Review comment: Seems a little weird between #5 and #6? Do we need to add some contents like "If a schema is updated automatically:” after #5? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc
Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#discussion_r352072982 ## File path: site2/docs/schema-evolution-compatibility.md ## @@ -46,7 +46,7 @@ For more information, see [Schema compatibility check strategy](#schema-compatib ## Schema compatibility check strategy -Pulsar has 8 schema compatibility check strategies, which are summarized in the following table. +Pulsar has 9 schema compatibility check strategies, which are summarized in the following table. Review comment: ```suggestion Pulsar has 8 schema compatibility check strategies, which are summarized in the following table. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc
Anonymitaet commented on a change in pull request #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#discussion_r352091960 ## File path: site2/docs/schema-manage.md ## @@ -50,32 +58,47 @@ For a consumer, the `AutoUpdate` happens in the following cases: * If a **consumer connects to a topic without a schema** (which means the consumer receiving raw bytes), the consumer can connect to the topic successfully without doing any compatibility check. -* If a **consumer connects to a topic with a schema**: - - * If the **topic is idle** (no producers, no entries, no other consumers and no registered schemas), the broker registers a schema for the topic automatically. +* If a **consumer connects to a topic with a schema**. - * If the **topic is not idle**, the broker verifies if the schema provided by the consumer is compatible with the registered schema of the topic. +* If a topic does not have all of them (a schema/data/a local consumer and a local producer): -* If the **schema passes the compatibility check**, the consumer can connect to the topic and receive messages. +* If `isAllowAutoUpdateSchema` sets to **true**, then the consumer registers a schema and it is connected to a broker. + +* If `isAllowAutoUpdateSchema` sets to **false**, then the consumer is rejected to connect to a broker. + +* If a topic has one of them (a schema/data/a local consumer and a local producer), then the schema compatibility check is performed. -* If the **schema does not pass the compatibility check**, the consumer is rejected and disconnected. - - +* If the schema passes the compatibility check, then the consumer is connected to the broker. + +* If the schema does not pass the compatibility check, then the consumer is rejected to connect to the broker. + + + ### Manage AutoUpdate strategy You can use the `pulsar-admin` command to manage the `AutoUpdate` strategy as below: +* [Enable AutoUpdate](#enable-autoupdate) + * [Disable AutoUpdate](#disable-autoupdate) Review comment: As we discussed just now, please correct the commands for `enable` and `disable` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5768: [CLI] Add a broker tool for operations of a specific broker
codelipenghui commented on issue #5768: [CLI] Add a broker tool for operations of a specific broker URL: https://github.com/apache/pulsar/pull/5768#issuecomment-559758135 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #5764: Fixes #5753
Jennifer88huang commented on issue #5764: Fixes #5753 URL: https://github.com/apache/pulsar/pull/5764#issuecomment-559753917 @belinda-wong You don't need to close this one, you can modify directly in this PR, just as you've done in #5771. 1. We can modify the PR title and make it more clear to distinguish. 2. If this occurs in version 2.4.1, we should modify the content in versioned-2.4.1 as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui closed issue #5747: Release ByteBuf failed when send a over size message in message batch container
codelipenghui closed issue #5747: Release ByteBuf failed when send a over size message in message batch container URL: https://github.com/apache/pulsar/issues/5747 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5747: Release ByteBuf failed when send a over size message in message batch container
codelipenghui commented on issue #5747: Release ByteBuf failed when send a over size message in message batch container URL: https://github.com/apache/pulsar/issues/5747#issuecomment-559753099 Close this issue since #5748 is fixed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5734: [Reopen][Issue 5597]retry when getPartitionedTopicMetadata failed
wolfstudy commented on issue #5734: [Reopen][Issue 5597]retry when getPartitionedTopicMetadata failed URL: https://github.com/apache/pulsar/pull/5734#issuecomment-559752025 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] belinda-wong opened a new pull request #5764: Fixes #5753
belinda-wong opened a new pull request #5764: Fixes #5753 URL: https://github.com/apache/pulsar/pull/5764 Fixes #5753 [Doc] the dead link for TLS Authentication on Proxies ### Motivation The link is dead. ### Modifications Add ".md" to the link. That is: security-tls-authentication.md#on-proxies This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai merged pull request #5741: [website]Add content of pulsar-manager in website(include release notes, download link)
jiazhai merged pull request #5741: [website]Add content of pulsar-manager in website(include release notes, download link) URL: https://github.com/apache/pulsar/pull/5741 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (d9c2b8b -> e4fc609)
This is an automated email from the ASF dual-hosted git repository.
zhaijia pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git.
from d9c2b8b Add document for sticky consumer (#5745)
add e4fc609 [website]Add content of pulsar-manager in website(include
release notes, download link) (#5741)
No new revisions were added by this update.
Summary of changes:
site2/docs/administration-pulsar-manager.md| 132 +
site2/website/pages/en/download.js | 56 +
...se-notes.js => pulsar-manager-release-notes.js} | 4 +-
site2/website/pulsar-manager-release-notes.md | 19 +++
site2/website/pulsar-manager-release.json | 3 +
site2/website/sidebars.json| 1 +
6 files changed, 213 insertions(+), 2 deletions(-)
create mode 100644 site2/docs/administration-pulsar-manager.md
copy site2/website/pages/en/{release-notes.js =>
pulsar-manager-release-notes.js} (83%)
create mode 100644 site2/website/pulsar-manager-release-notes.md
create mode 100644 site2/website/pulsar-manager-release.json
[GitHub] [pulsar] PierreZ commented on issue #5735: Provide raw counters metrics
PierreZ commented on issue #5735: Provide raw counters metrics
URL: https://github.com/apache/pulsar/issues/5735#issuecomment-559748757
I started working on it. From what I see, the rate is computed through
[Rate](https://github.com/apache/pulsar/blob/master/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/util/Rate.java),
which seems to be hold by
[Producer](https://github.com/apache/pulsar/blob/master/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Producer.java)
which `Represents a currently connected producer`. The rate is [polled
frequently](https://github.com/apache/pulsar/blob/master/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java#L383-L386).
As the classes are representing by **connected** {Producer, Consumer} and the
fact the update is async, we can miss events between two polls if a {Producer,
Consumer} disconnects.
I'm wondering if the metrics can be generated from a place less volatile,
maybe at the opening of the managed ledger? What do you think?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5680: [transaction-coordinator] Implementation of transaction coordinator client.
codelipenghui commented on issue #5680: [transaction-coordinator] Implementation of transaction coordinator client. URL: https://github.com/apache/pulsar/pull/5680#issuecomment-559745938 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service
codelipenghui commented on issue #4955: [PIP-39] Introduce system topic and topic policies service URL: https://github.com/apache/pulsar/pull/4955#issuecomment-559745462 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #5686: NPE in RackwareEnsemblePlacementPolicyImpl.
codelipenghui commented on issue #5686: NPE in RackwareEnsemblePlacementPolicyImpl. URL: https://github.com/apache/pulsar/issues/5686#issuecomment-559745125 @sijie I have create issue https://github.com/apache/bookkeeper/issues/2208 in apache/bookkeeper, so can we close this issue now? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5762: Add the filesystem offload doc
Anonymitaet commented on a change in pull request #5762: Add the filesystem offload doc URL: https://github.com/apache/pulsar/pull/5762#discussion_r352071546 ## File path: site2/docs/cookbooks-tiered-storage.md ## @@ -186,6 +190,59 @@ Pulsar also provides some knobs to configure the size of requests sent to GCS. In both cases, these should not be touched unless you know what you are doing. +### "filesystem" Driver configuration + + + Configure connection address + +```conf +fileSystemURI="hdfs://127.0.0.1:9000" +``` + Configure Hadoop profile path + +In the configuration file under this path, you can configure the base path and authentication and so on. + +```conf +fileSystemProfilePath="../conf/filesystem_offload_core_site.xml" +``` + +The model for storing this topic data use `org.apache.hadoop.io.MapFile`, so we can use all of the configuration in Apache Hadoop for `org.apache.hadoop.io.MapFile`, the following example : + +```conf + + +fs.defaultFS + + + + +hadoop.tmp.dir +pulsar + + + +io.file.buffer.size +4096 + + + +io.seqfile.compress.blocksize +100 + + + +io.seqfile.compression.type +BLOCK + + + +io.map.index.interval +128 + + +``` + +A detailed explanation of the role of these configurations and other configurations can be found in [Filesystem Storage](http://hadoop.apache.org/). Review comment: ```suggestion For more information about the configurations in `org.apache.hadoop.io.MapFile`, see [Filesystem Storage](http://hadoop.apache.org/). ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5762: Add the filesystem offload doc
Anonymitaet commented on a change in pull request #5762: Add the filesystem offload doc URL: https://github.com/apache/pulsar/pull/5762#discussion_r352056573 ## File path: site2/docs/cookbooks-tiered-storage.md ## @@ -6,11 +6,14 @@ sidebar_label: Tiered Storage Pulsar's **Tiered Storage** feature allows older backlog data to be offloaded to long term storage, thereby freeing up space in BookKeeper and reducing storage costs. This cookbook walks you through using tiered storage in your Pulsar cluster. -Tiered storage currently uses [Apache Jclouds](https://jclouds.apache.org) to supports +* Tiered storage uses [Apache Jclouds](https://jclouds.apache.org) to supports [Amazon S3](https://aws.amazon.com/s3/) and [Google Cloud Storage](https://cloud.google.com/storage/)(GCS for short) for long term storage. With Jclouds, it is easy to add support for more [cloud storage providers](https://jclouds.apache.org/reference/providers/#blobstore-providers) in the future. +* Tiered storage uses [Apache Hadoop](http://hadoop.apache.org/) to supports filesystem for long term storage. Review comment: ```suggestion * Tiered storage uses [Apache Hadoop](http://hadoop.apache.org/) to support filesystem for long term storage. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5762: Add the filesystem offload doc
Anonymitaet commented on a change in pull request #5762: Add the filesystem offload doc URL: https://github.com/apache/pulsar/pull/5762#discussion_r352060316 ## File path: site2/docs/cookbooks-tiered-storage.md ## @@ -186,6 +190,59 @@ Pulsar also provides some knobs to configure the size of requests sent to GCS. In both cases, these should not be touched unless you know what you are doing. +### "filesystem" Driver configuration + + + Configure connection address + +```conf +fileSystemURI="hdfs://127.0.0.1:9000" +``` + Configure Hadoop profile path + +In the configuration file under this path, you can configure the base path and authentication and so on. Review comment: ```suggestion The configuration file is stored in the Hadoop profile path. It contains various settings, such as base path, authentication, and so on. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5762: Add the filesystem offload doc
Anonymitaet commented on a change in pull request #5762: Add the filesystem offload doc URL: https://github.com/apache/pulsar/pull/5762#discussion_r352058960 ## File path: site2/docs/cookbooks-tiered-storage.md ## @@ -186,6 +190,59 @@ Pulsar also provides some knobs to configure the size of requests sent to GCS. In both cases, these should not be touched unless you know what you are doing. +### "filesystem" Driver configuration + + + Configure connection address + Review comment: ```suggestion You can configure the connection address in the `broker.conf` file. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5762: Add the filesystem offload doc
Anonymitaet commented on a change in pull request #5762: Add the filesystem offload doc URL: https://github.com/apache/pulsar/pull/5762#discussion_r352056461 ## File path: site2/docs/cookbooks-tiered-storage.md ## @@ -6,11 +6,14 @@ sidebar_label: Tiered Storage Pulsar's **Tiered Storage** feature allows older backlog data to be offloaded to long term storage, thereby freeing up space in BookKeeper and reducing storage costs. This cookbook walks you through using tiered storage in your Pulsar cluster. -Tiered storage currently uses [Apache Jclouds](https://jclouds.apache.org) to supports +* Tiered storage uses [Apache Jclouds](https://jclouds.apache.org) to supports Review comment: ```suggestion * Tiered storage uses [Apache jclouds](https://jclouds.apache.org) to support ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5762: Add the filesystem offload doc
Anonymitaet commented on a change in pull request #5762: Add the filesystem offload doc URL: https://github.com/apache/pulsar/pull/5762#discussion_r352070627 ## File path: site2/docs/cookbooks-tiered-storage.md ## @@ -186,6 +190,59 @@ Pulsar also provides some knobs to configure the size of requests sent to GCS. In both cases, these should not be touched unless you know what you are doing. +### "filesystem" Driver configuration + + + Configure connection address + +```conf +fileSystemURI="hdfs://127.0.0.1:9000" +``` + Configure Hadoop profile path + +In the configuration file under this path, you can configure the base path and authentication and so on. + +```conf +fileSystemProfilePath="../conf/filesystem_offload_core_site.xml" +``` + +The model for storing this topic data use `org.apache.hadoop.io.MapFile`, so we can use all of the configuration in Apache Hadoop for `org.apache.hadoop.io.MapFile`, the following example : Review comment: ```suggestion The model for storing topic data uses `org.apache.hadoop.io.MapFile`. You can use all of the configurations in `org.apache.hadoop.io.MapFile` for Hadoop. **Example** ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5762: Add the filesystem offload doc
Anonymitaet commented on a change in pull request #5762: Add the filesystem offload doc URL: https://github.com/apache/pulsar/pull/5762#discussion_r352055188 ## File path: site2/docs/concepts-tiered-storage.md ## @@ -12,6 +12,6 @@ One way to alleviate this cost is to use Tiered Storage. With tiered storage, ol > Data written to BookKeeper is replicated to 3 physical machines by default. > However, once a segment is sealed in BookKeeper it becomes immutable and can > be copied to long term storage. Long term storage can achieve cost savings > by using mechanisms such as [Reed-Solomon error > correction](https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_correction) > to require fewer physical copies of data. -Pulsar currently supports S3 and Google Cloud Storage (GCS) for [long term store](https://pulsar.apache.org/docs/en/cookbooks-tiered-storage/). Offloading to long term storage triggered via a Rest API or command line interface. The user passes in the amount of topic data they wish to retain on BookKeeper, and the broker will copy the backlog data to long term storage. The original data will then be deleted from BookKeeper after a configured delay (4 hours by default). +Pulsar currently supports S3 and Google Cloud Storage (GCS) and filesystem for [long term store](https://pulsar.apache.org/docs/en/cookbooks-tiered-storage/). Offloading to long term storage triggered via a Rest API or command line interface. The user passes in the amount of topic data they wish to retain on BookKeeper, and the broker will copy the backlog data to long term storage. The original data will then be deleted from BookKeeper after a configured delay (4 hours by default). Review comment: ```suggestion Pulsar currently supports S3, Google Cloud Storage (GCS), and filesystem for [long term store](https://pulsar.apache.org/docs/en/cookbooks-tiered-storage/). Offloading to long term storage triggered via a Rest API or command line interface. The user passes in the amount of topic data they wish to retain on BookKeeper, and the broker will copy the backlog data to long term storage. The original data will then be deleted from BookKeeper after a configured delay (4 hours by default). ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] zymap commented on issue #5384: Topic compaction fails with 500 error
zymap commented on issue #5384: Topic compaction fails with 500 error URL: https://github.com/apache/pulsar/issues/5384#issuecomment-559735696 Hi @Lanayx I try running the pulsar 2.4.1 on the Kubernetes with Helm template. It works well after I trigger the compaction. I am sorry but I can not reproduce this problem. Did you do some other operations after you start the cluster? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] PierreZ opened a new pull request #5773: Fixes #5772: Bump Lombok version to allow building with a JDK12>=12
PierreZ opened a new pull request #5773: Fixes #5772: Bump Lombok version to allow building with a JDK12>=12 URL: https://github.com/apache/pulsar/pull/5773 *(If this PR fixes a github issue, please add `Fixes #`.)* Fixes #5772 ### Motivation I would like to be able to build Pulsar with at least JDK12. ### Modifications Bumping lombok dep. ### Verifying this change This change is a trivial rework / code cleanup without any test coverage. ### Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #5770: Pulsar Broker Sometimes does not return CommandSendReceipt/CommandSendError to prducer send CommandSend
jiazhai commented on issue #5770: Pulsar Broker Sometimes does not return CommandSendReceipt/CommandSendError to prducer send CommandSend URL: https://github.com/apache/pulsar/issues/5770#issuecomment-559727663 @alphashaw Great to know that you are working on an Erlang client. In current client, each partition is handled in an individual connection. so for several partitions in broker side the order between partitions is not guaranteed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5642: Add Github workflow for gated checkin
aahmed-se commented on issue #5642: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5642#issuecomment-559724930 run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5686: NPE in RackwareEnsemblePlacementPolicyImpl.
sijie commented on issue #5686: NPE in RackwareEnsemblePlacementPolicyImpl. URL: https://github.com/apache/pulsar/issues/5686#issuecomment-559722288 @codelipenghui Can you create an issue at apache/bookkeeper? I think it is a problem at bookkeeper. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie closed issue #5699: Expose last execution time of message expiration.
sijie closed issue #5699: Expose last execution time of message expiration. URL: https://github.com/apache/pulsar/issues/5699 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5699: Expose last execution time of message expiration.
sijie commented on issue #5699: Expose last execution time of message expiration. URL: https://github.com/apache/pulsar/issues/5699#issuecomment-559721136 This was fixed by #5721 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] aahmed-se commented on issue #5642: Add Github workflow for gated checkin
aahmed-se commented on issue #5642: Add Github workflow for gated checkin URL: https://github.com/apache/pulsar/pull/5642#issuecomment-559719967 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on issue #5627: [Issue 5626][pulsar-function-go] Fix a memory leak of pulsar-function-go library regarding time.NewTimer()
wolfstudy commented on issue #5627: [Issue 5626][pulsar-function-go] Fix a memory leak of pulsar-function-go library regarding time.NewTimer() URL: https://github.com/apache/pulsar/pull/5627#issuecomment-559719169 run java8 tests run cpp tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on a change in pull request #5757: [doc]Modify schema compatibility check doc
congbobo184 commented on a change in pull request #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#discussion_r352033217 ## File path: site2/docs/schema-understand.md ## @@ -539,14 +539,17 @@ This diagram illustrates how does schema work on the Producer side. 3. The broker looks up the schema in the schema storage to check if it is already a registered schema. 4. If yes, the broker skips the schema validation since it is a known schema, and returns the schema version to the producer. + +5. If no, the broker judge whether this namespace can automatically create schema. + isAllowAutoUpdateSchema can be modified by Admin-Api and REST-API -5. If no, the broker validates the schema based on the schema compatibility check strategy defined for the topic. +6. If yes, the broker validates the schema based on the schema compatibility check strategy defined for the topic. Review comment: yes, we can delete the line 546 :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] PierreZ opened a new issue #5772: Bump Lombok version to allow building with a JDK >=12
PierreZ opened a new issue #5772: Bump Lombok version to allow building with a JDK >=12 URL: https://github.com/apache/pulsar/issues/5772 **Describe the bug** Running `mvn clean install` on branch master using JDK13 results on a Lombok annotation failure. **To Reproduce** Steps to reproduce the behavior: 1. Pull JDK13 2. Run `mvn install` **Expected behavior** Build success **Screenshots** ``` [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile (default-testCompile) on project integration: Compilation failure [ERROR] Lombok annotation handler class lombok.javac.handlers.HandleBuilder failed on /home/pierrez/workspace/java/pulsar/tests/integration/src/test/java/org/apache/pulsar/tests/integration/topologies/PulsarClusterSpec.java: java.lang.IllegalStateException: Lombok TreeMaker frontend issue: no match when looking for method: com.sun.tools.javac.tree.JCTree$JCCase com.sun.tools.javac.tree.TreeMaker.Case(com.sun.tools.javac.tree.JCTree$JCExpression, com.sun.tools.javac.util.List) ``` It is an [closed Lombok issue](rzwitserloot/lombok/issues/1888), so we should be fine by upgrading Lombok. **Desktop (please complete the following information):** ```bash uname -a # Linux pierrez-xps 5.3.0-23-generic #25-Ubuntu SMP Tue Nov 12 09:22:33 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux java -version # openjdk version "13.0.1" 2019-10-15 # OpenJDK Runtime Environment AdoptOpenJDK (build 13.0.1+9) # OpenJDK 64-Bit Server VM AdoptOpenJDK (build 13.0.1+9, mixed mode, sharing) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] massakam commented on issue #5758: Bump jackson libraries to 2.10.1
massakam commented on issue #5758: Bump jackson libraries to 2.10.1 URL: https://github.com/apache/pulsar/pull/5758#issuecomment-559706881 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #5757: [doc]Modify schema compatibility check doc
congbobo184 commented on issue #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#issuecomment-559705442 Thanks, i will look carefully and add hidden flag to hide them. :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #5757: [doc]Modify schema compatibility check doc
sijie commented on issue #5757: [doc]Modify schema compatibility check doc URL: https://github.com/apache/pulsar/pull/5757#issuecomment-559704214 @congbobo184 this is one of the examples: https://github.com/apache/pulsar/blob/master/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdNamespaces.java#L67 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5741: [website]Add content of pulsar-manager in website(include release notes, download link)
tuteng commented on issue #5741: [website]Add content of pulsar-manager in website(include release notes, download link) URL: https://github.com/apache/pulsar/pull/5741#issuecomment-559701064 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] belinda-wong commented on issue #5764: Fixes #5753
belinda-wong commented on issue #5764: Fixes #5753 URL: https://github.com/apache/pulsar/pull/5764#issuecomment-559698992 @Jennifer88huang Thanks for the guidance. I've run the local build and test and both works. could you take a look at #5771 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] belinda-wong closed pull request #5764: Fixes #5753
belinda-wong closed pull request #5764: Fixes #5753 URL: https://github.com/apache/pulsar/pull/5764 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] belinda-wong opened a new pull request #5771: Fixes #5753
belinda-wong opened a new pull request #5771: Fixes #5753 URL: https://github.com/apache/pulsar/pull/5771 Fixes #5753 ### Motivation The link is dead. ### Modifications changed to: see ["Enabling TLS Authentication on Proxies"](security-tls-authentication.md#enable-tls-authentication-on-proxies) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
