[GitHub] [pulsar-client-python] eolivelli commented on a diff in pull request #81: Add a version attribute

[GitBox]

eolivelli commented on code in PR #81:
URL: 
https://github.com/apache/pulsar-client-python/pull/81#discussion_r1069048322


##
pulsar/__about__.py:
##
@@ -0,0 +1,19 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+__version__='3.1.0a1'

Review Comment:
   we would have to update the release procedure
   
   @BewareMyPower 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] tisonkun commented on issue #19196: [Doc] 2.10* docs provide incorrect command for `namespace set-backlog-quota`

[GitBox]

tisonkun commented on issue #19196:
URL: https://github.com/apache/pulsar/issues/19196#issuecomment-1381434775

   Perhaps also place a `--limit 2G` above?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] dave2wave opened a new issue, #17898: Only Index Docs from Currently Supported Versions

[GitBox]

dave2wave opened a new issue, #17898:
URL: https://github.com/apache/pulsar/issues/17898

   In Algolia there is a crawler configuration which skips "incubating" docs. I 
think that we should stop creating an index for versions of pulsar components 
which are no longer supporting.
   
   (1) We can either remove these pages from the site.
   
   or more likely
   
   (2) Include those versions in the exclude pattern.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] tisonkun commented on issue #19196: [Doc] 2.10* docs provide incorrect command for `namespace set-backlog-quota`

[GitBox]

tisonkun commented on issue #19196:
URL: https://github.com/apache/pulsar/issues/19196#issuecomment-1381433140

   cc @labuladong @nodece 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] tisonkun commented on issue #17898: Only Index Docs from Currently Supported Versions

[GitBox]

tisonkun commented on issue #17898:
URL: https://github.com/apache/pulsar/issues/17898#issuecomment-1381432449

   I notice that the version config contains a option `noIndex` 
https://docusaurus.io/docs/api/plugins/@docusaurus/plugin-content-docs#VersionsConfig
 to hint the search engine index those version, will investigate.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[pulsar-site] branch main updated: Use server side redirection: BUILD_ALL_VERSION=1 (#382)

[tison]

This is an automated email from the ASF dual-hosted git repository.

tison pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git


The following commit(s) were added to refs/heads/main by this push:
 new 6ad6454c904 Use server side redirection: BUILD_ALL_VERSION=1 (#382)
6ad6454c904 is described below

commit 6ad6454c904f845bd94684cc4778430f5e267ed5
Author: tison 
AuthorDate: Fri Jan 13 15:41:52 2023 +0800

Use server side redirection: BUILD_ALL_VERSION=1 (#382)

Signed-off-by: tison 
---
 docusaurus.config.js | 18 +-
 static/.htaccess | 19 ++-
 2 files changed, 15 insertions(+), 22 deletions(-)

diff --git a/docusaurus.config.js b/docusaurus.config.js
index fb1c24d2147..67a6eb0f7a3 100644
--- a/docusaurus.config.js
+++ b/docusaurus.config.js
@@ -161,8 +161,7 @@ module.exports = {
 position: "right",
 items: [
 {
-type: "doc",
-docId: "concepts-overview",
+to: `/docs/${versions[0]}/concepts-overview/`,
 label: "Pulsar Concepts",
 },
 {
@@ -352,21 +351,6 @@ module.exports = {
 ],
 ],
 plugins: [
-[
-'@docusaurus/plugin-client-redirects',
-/** @type {import('@docusaurus/plugin-client-redirects').Options} 
*/
-{
-createRedirects(existingPath) {
-const latestVersion = versions[0];
-if (existingPath.includes(`/docs/${latestVersion}`)) {
-return [
-existingPath.replace(`/docs/${latestVersion}`, 
'/docs/'),
-];
-}
-return undefined;
-},
-},
-],
 "./postcss-tailwind-loader",
 [
 'content-docs',
diff --git a/static/.htaccess b/static/.htaccess
index 4bf2b3bbdd7..7d831ca7447 100755
--- a/static/.htaccess
+++ b/static/.htaccess
@@ -4,7 +4,8 @@ RewriteRule "^en/(.+)$" "/$1" [R=301,DPI,L]
 RewriteRule "^docs/en/(.+)$" "/docs/$1" [R=301,DPI,L]
 RewriteRule "^docs/(zh-CN|zh-TW|ja|ko|fr)/(.+)$" "/$1/docs/$2" [R=301,DPI,L]
 RewriteRule "^(zh-CN|zh-TW|ja|ko|fr)/docs/(.+)$" "/docs/$2" [R=301,DPI,L]
-RewriteRule "^docs$" "/docs/" [R=301,DPI,L]
+
+RewriteRule "^docs/?$" "/docs/2.11.x" [R=301,DPI,L]
 RewriteRule "^docs/(.+)-incubating/(.+)$" 
"https://pulsar.staged.apache.org/docs/$1-incubating/$2; [R=301,DPI,L]
 
 RewriteRule "^pulsar-manager-release-notes(/)?$" "/release-notes" [R=301,DPI,L]
@@ -15,7 +16,15 @@ RewriteRule "^contributing(/)?$" "/contribute/" [R=301,DPI,L]
 RewriteCond %{REQUEST_FILENAME}\/index\.html -f
 RewriteRule ^ - [L]
 
-RewriteRule "^docs/(.+/)?develop-binary-protocol(/)?$" 
"/docs/$1developing-binary-protocol" [R=301,DPI,L]
-RewriteRule "^docs/(.+/)?administration-dashboard(/)?$" 
"/docs/$1administration-pulsar-manager" [R=301,DPI,L]
-RewriteRule "^docs/(.+/)?security-tls-keystore(/)?$" 
"/docs/$1security-tls-authentication" [R=301,DPI,L]
-RewriteRule "^docs/(.+/)?security-policy-and-supported-versions(/)?$" 
"/contribute/version-policy/" [R=301,DPI,L]
+RewriteCond expr "%{REQUEST_URI} !~ m|^/docs/next|"
+RewriteCond expr "%{REQUEST_URI} !~ m|^/docs/next\/.*|"
+RewriteCond expr "%{REQUEST_URI} !~ m|^/docs/\d+\.\d+\.\d+|"
+RewriteCond expr "%{REQUEST_URI} !~ m|^/docs/\d+\.\d+\.\d+/.*|"
+RewriteCond expr "%{REQUEST_URI} !~ m|^/docs/\d+\.\d+\.x|"
+RewriteCond expr "%{REQUEST_URI} !~ m|^/docs/\d+\.\d+\.x/.*|"
+RewriteRule "^docs/(.*)$" "/docs/2.11.x/$1" [R=301,DPI,L]
+
+RewriteRule "^docs/(.+/)?develop-binary-protocol/?$" 
"/docs/$1developing-binary-protocol" [R=301,DPI,L]
+RewriteRule "^docs/(.+/)?administration-dashboard/?$" 
"/docs/$1administration-pulsar-manager" [R=301,DPI,L]
+RewriteRule "^docs/(.+/)?security-tls-keystore/?$" 
"/docs/$1security-tls-authentication" [R=301,DPI,L]
+RewriteRule "^docs/(.+/)?security-policy-and-supported-versions/?$" 
"/contribute/version-policy" [R=301,DPI,L]

[GitHub] [pulsar-site] tisonkun merged pull request #382: Use server side redirection

[GitBox]

tisonkun merged PR #382:
URL: https://github.com/apache/pulsar-site/pull/382


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari opened a new pull request, #19225: [fix][ci] Fix CI issue where ubuntu apt repository azure.archive.ubuntu.com isn't available

[GitBox]

lhotari opened a new pull request, #19225:
URL: https://github.com/apache/pulsar/pull/19225

   ### Motivation
   
   There's a problem with azure.archive.ubuntu.com apt mirror that causes many 
builds to fail. I reported it by adding a comment to an existing issue: 
https://github.com/actions/runner-images/issues/675#issuecomment-1381389712 . 
However, there's a need to apply a fix so that apache/pulsar CI doesn't get 
blocked by the problem.
   
   ### Modifications
   
   - use apt-spy2 to pick the closest available ubuntu mirror
   - set the chosen mirror to UBUNTU_MIRROR environment variable to be used in 
docker image building
   
   ### Documentation
   
   
   
   - [ ] `doc` 
   - [ ] `doc-required` 
   - [x] `doc-not-needed` 
   - [ ] `doc-complete` 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-client-cpp] shibd commented on pull request #170: [feature] Support set ServiceUrlProvider when create client.

[GitBox]

shibd commented on PR #170:
URL: 
https://github.com/apache/pulsar-client-cpp/pull/170#issuecomment-1381406248

   > We only need to support passing a std::function. The 
initialize and close methods are redundant.
   
The initialize and close methods are useful. We need pass `Client&` to 
`ServiceUrlProvider` by `initialize` method.
   
   
https://github.com/apache/pulsar-client-cpp/blob/7d91638513405ed3e9bc5e7cd782d30e8703679d/lib/Client.cc#L43
   
   This way, if the user implements the `ServiceUrlProvider`. When it listens 
for a change in the service URL (such as through zk or other storage), it can 
invoke the `client.updateServiceUrl` method to update the URL.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-site] momo-jun commented on a diff in pull request #367: [blog] Add 2.10.3 release blog

[GitBox]

momo-jun commented on code in PR #367:
URL: https://github.com/apache/pulsar-site/pull/367#discussion_r1068992548


##
blog/2023-01-09-Apache-Pulsar-2-10-3.md:
##
@@ -0,0 +1,54 @@
+---
+title: "What’s New in Apache Pulsar 2.10.3"
+date: 2023-01-09
+author: "liangyepianzhou, momo-jun"
+---
+
+The Apache Pulsar community releases version 2.10.3! 50 contributors provided 
improvements and bug fixes that delivered 155 commits. Thanks for all your 
contributions.
+
+This blog walks through the most noteworthy changes. For the complete list 
including all feature enhancements and bug fixes, check out the [Pulsar 2.10.3 
Release 
Notes](https://pulsar.apache.org/release-notes/versioned/pulsar-2.10.3/).

Review Comment:
   Thanks for letting me know. Resolved.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-client-cpp] shibd commented on pull request #168: Fix segfault caused by socket I/O on a closed io_service

[GitBox]

shibd commented on PR #168:
URL: 
https://github.com/apache/pulsar-client-cpp/pull/168#issuecomment-1381385269

   @RobertIndie Can you take a look? This error occurs frequently.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-site] Technoboy- commented on pull request #373: Update release note for 2.11.0

[GitBox]

Technoboy- commented on PR #373:
URL: https://github.com/apache/pulsar-site/pull/373#issuecomment-1381384834

   > @Technoboy- I did a couple of fixes when previewing this page:)
   
   Thanks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[pulsar-site] branch main updated: Update release note for 2.11.0 (#373)

[technoboy]

This is an automated email from the ASF dual-hosted git repository.

technoboy pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git


The following commit(s) were added to refs/heads/main by this push:
 new c99157e6633 Update release note for 2.11.0 (#373)
c99157e6633 is described below

commit c99157e66331228ab9c8f26cd5c37895ed7fc00e
Author: Jiwei Guo 
AuthorDate: Fri Jan 13 14:44:24 2023 +0800

Update release note for 2.11.0 (#373)
---
 release-notes/versioned/client-java-2.11.0.md  |  48 ++
 release-notes/versioned/client-websocket-2.11.0.md |  10 +
 release-notes/versioned/pulsar-2.11.0.md   | 606 +
 3 files changed, 664 insertions(+)

diff --git a/release-notes/versioned/client-java-2.11.0.md 
b/release-notes/versioned/client-java-2.11.0.md
new file mode 100644
index 000..b3279f03864
--- /dev/null
+++ b/release-notes/versioned/client-java-2.11.0.md
@@ -0,0 +1,48 @@
+---
+id: client-java-2.11.0
+title: Client Java 2.11.0
+sidebar_label: Client Java 2.11.0
+---
+
+- [feature][Java] Support Reader Interceptor 
[#14729](https://github.com/apache/pulsar/pull/14729)
+- [improve][Java] Refactor SchemaHash to reduce call of hashFunction in 
SchemaHash [#17948](https://github.com/apache/pulsar/pull/17948)
+- [improve][Java] Improve performance of multi-topic consumer with more than 
one IO thread [#16336](https://github.com/apache/pulsar/pull/16336)
+- [improve][Java] Add initialization for the OpSendMsg 
[#16256](https://github.com/apache/pulsar/pull/16256)
+- [improve][Java] Replace ScheduledExecutor to improve performance of message 
consumption [#16236](https://github.com/apache/pulsar/pull/16236)
+- [improve][Java] Send CloseConsumer on client timeout 
[#16616](https://github.com/apache/pulsar/pull/16616)
+- [improve][Java] Make DeadLetterPolicy deserializable 
[#16513](https://github.com/apache/pulsar/pull/16513)
+- [improve][Java] Switch to rely on Netty for Hostname Verification 
[#15824](https://github.com/apache/pulsar/pull/15824)
+- [improve][Java] Remove sensitive msg from consumer/producer stats log 
[#15483](https://github.com/apache/pulsar/pull/15483)
+- [improve][Java] Add pending messages information while printing the producer 
stats [#15440](https://github.com/apache/pulsar/pull/15440)
+- [improve][Java] Check consumer schema null in advance 
[#15327](https://github.com/apache/pulsar/pull/15327)
+- [improve][Java] Returns immutable data set when using TableView 
[#14833](https://github.com/apache/pulsar/pull/14833)
+- [improve][Java] Avoid timer task run before previous subscribe complete 
[#14818](https://github.com/apache/pulsar/pull/14818)
+- [improve][Java] Optimize pause when creating sub consumers in multi-topic 
consumer [#14566](https://github.com/apache/pulsar/pull/14566)
+- [cleanup][Java] Reduce code duplication in admin client 
[#16377](https://github.com/apache/pulsar/pull/16377)
+- [cleanup][Java] Remove redundant check for chunked message TotalChunkMsgSize 
in ConsumerImpl [#16797](https://github.com/apache/pulsar/pull/16797)
+- [fix][Java] Fixed ack failure in ReaderImpl due to null messageId 
[#17728](https://github.com/apache/pulsar/pull/17728)
+- [fix][Java] Unwrap completion exception for Lookup Services 
[#17717](https://github.com/apache/pulsar/pull/17717)
+- [fix][Java] Fix scheduledExecutorProvider not shutdown 
[#17527](https://github.com/apache/pulsar/pull/17527)
+- [fix][Java] Fix the message present in incoming queue after going to DLQ 
[#17326](https://github.com/apache/pulsar/pull/17326)
+- [fix][Java] Fix reach redeliverCount client can't send batch messags 
[#17317](https://github.com/apache/pulsar/pull/17317)
+- [fix][Java] Fix reach redeliverCount client can't send messages to DLQ 
[#17287](https://github.com/apache/pulsar/pull/17287)
+- [fix][Java] LastBatchSendNanoTime initialization 
[#17058](https://github.com/apache/pulsar/pull/17058)
+- [fix][Java] Fixed inactive cnx channel causing the request to fail to time 
out and fail to return [#17051](https://github.com/apache/pulsar/pull/17051)
+- [fix][Java] Release semaphore before discarding messages in 
batchMessageContainer [#17019](https://github.com/apache/pulsar/pull/17019)
+- [fix][Java] Fix client memory limit currentUsage leak and semaphore release 
duplicated in ProducerImpl [#16837](https://github.com/apache/pulsar/pull/16837)
+- [fix][Java] Release memory usage for invalid messages 
[#16835](https://github.com/apache/pulsar/pull/16835)
+- [fix][Java] Fix subscription topic name error 
[#16719](https://github.com/apache/pulsar/pull/16719)
+- [fix][Java] Add message key if exists to deadLetter messages 
[#16615](https://github.com/apache/pulsar/pull/16615)
+- [fix][Java] Fix large messages sometimes cannot be split into chunks 
[#16196](https://github.com/apache/pulsar/pull/16196)
+- [fix][Java] Only trigger the batch receive timeout when having pending batch 
receives requests 

[GitHub] [pulsar-site] Technoboy- merged pull request #373: Update release note for 2.11.0

[GitBox]

Technoboy- merged PR #373:
URL: https://github.com/apache/pulsar-site/pull/373


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-site] momo-jun commented on pull request #373: Update release note for 2.11.0

[GitBox]

momo-jun commented on PR #373:
URL: https://github.com/apache/pulsar-site/pull/373#issuecomment-1381380238

   @Technoboy- I did a couple of fixes when previewing this page:)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[pulsar-site] branch release-2.11.0 updated (ac261cf8da1 -> 7d9612b4b0c)

[junma]

This is an automated email from the ASF dual-hosted git repository.

junma pushed a change to branch release-2.11.0
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git


from ac261cf8da1 delete cpp and py notes.
 add 7d9612b4b0c preview fixes

No new revisions were added by this update.

Summary of changes:
 release-notes/versioned/pulsar-2.11.0.md | 29 -
 1 file changed, 16 insertions(+), 13 deletions(-)

[pulsar-site] branch main updated: Add the Pulsar 2022 Year in Review blog (#371)

[tison]

This is an automated email from the ASF dual-hosted git repository.

tison pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git


The following commit(s) were added to refs/heads/main by this push:
 new 43cee1d9382 Add the Pulsar 2022 Year in Review blog (#371)
43cee1d9382 is described below

commit 43cee1d9382690cfa9bbf80b43e2f9396c30b416
Author: Sherlock Xu <65327072+sherlock...@users.noreply.github.com>
AuthorDate: Fri Jan 13 14:20:07 2023 +0800

Add the Pulsar 2022 Year in Review blog (#371)
---
 blog/2023-01-10-pulsar-2022-year-in-review.md | 106 ++
 static/img/pulsar-new-website-after-1.png | Bin 0 -> 157218 bytes
 static/img/pulsar-new-website-after-2.png | Bin 0 -> 751561 bytes
 static/img/pulsar-website-before.png  | Bin 0 -> 454982 bytes
 4 files changed, 106 insertions(+)

diff --git a/blog/2023-01-10-pulsar-2022-year-in-review.md 
b/blog/2023-01-10-pulsar-2022-year-in-review.md
new file mode 100644
index 000..a50a8da858e
--- /dev/null
+++ b/blog/2023-01-10-pulsar-2022-year-in-review.md
@@ -0,0 +1,106 @@
+---
+author: Apache Pulsar Community
+title: "Apache Pulsar 2022 Year in Review"
+date: 2023-01-10
+---
+
+2022 was an incredible year for Apache Pulsar as it witnessed strong community 
growth with the project becoming more stable, scalable, and secure.
+
+We are grateful to all of our community members and those in the wider 
open-source community who contributed to the project. They are the reasons 
behind every step Pulsar has made over the past year.
+
+
+
+Now, let’s take a look at some of the highlights in 2022.
+
+## New Pulsar website
+
+In May 2022, the Apache Pulsar community [launched its new 
website](https://lists.apache.org/thread/7nx3rm3cmpbw0ws1b62k17935xgyw2tj). The 
new appearance has a light blue background with the homepage structure 
redesigned. You can see [PIP-87](https://github.com/apache/pulsar/issues/12637) 
and [PIP-113](https://github.com/apache/pulsar/issues/13235) for more details. 
Following is a comparison of the websites.
+
+Before:
+
+![](/img/pulsar-website-before.png)
+
+After:
+
+![](/img/pulsar-new-website-after-1.png)
+
+![](/img/pulsar-new-website-after-2.png)
+
+Many community members [participated in the discussion and provided valuable 
feedback](https://github.com/apache/pulsar/issues/15538#issuecomment-1125602899).
 Currently, the Pulsar community is still working to improve the reader 
experience by making key information more visible and accessible.
+
+## Key events recap
+
+In 2022, the Apache Pulsar community put together a number of meetups and 
events across the globe to share the latest messaging and streaming 
technologies. Among others, two Pulsar Summits received the most attention from 
community members.
+
+- [Pulsar Summit San Francisco 
2022](https://streamnative.io/blog/community/2022-08-25-pulsar-summit-sf-2022-community-event-recap/):
 This event marked the first-ever, in-person Pulsar Summit in North America 
with 200+ attendees. It featured 20 speakers from companies like Google, AWS, 
and Yahoo!, 5 keynotes on Apache Pulsar and other streaming technologies, and 
12 breakout sessions on tech deep dives, use cases, and ecosystem talks. [Watch 
the sessions](https://streamnative.io/pulsar-su [...]
+- [Pulsar Summit Asia 
2022](pathname:///blog/2022/12/01/pulsar-summit-asia-2022-recap/): With 1,500+ 
registrations and 40,000+ views globally, this two-day virtual event gathered 
41 speakers giving 3 keynote addresses and 36 sessions. They came from 
companies like ByteDance, Huawei, Tencent, Nippon Telegraph and Telephone 
Corporation (NTT), Yum China, Netease, vivo, Nutanix, and StreamNative. [Watch 
the 
sessions](https://www.youtube.com/playlist?list=PLqRma1oIkcWgeNb3kgzqFyg5sywH8S4yy).
+
+To stay up to date with the latest events, you can join the following meetups 
or groups:
+
+- [New York City Apache Pulsar 
Meetup](https://www.meetup.com/new-york-city-apache-pulsar-meetup/)
+- [SF Bay Area Apache Pulsar 
Meetup](https://www.meetup.com/sf-bay-area-apache-pulsar-meetup/)
+- [Netherlands Apache Pulsar 
Meetup](https://www.meetup.com/netherlands-apache-pulsar-meetup/)
+- [Japan Pulsar User Group](https://japan-pulsar-user-group.connpass.com/)
+
+## Community growth
+
+The development of the Pulsar community would not be possible without our 
contributors. Among them, Pulsar Committers and PMC members have taken the lead 
in contributing to and promoting the project. In 2022, many new faces joined 
the community while we also welcomed old friends taking more responsibility. 
Let’s take a look at the Pulsar community by numbers.
+
+- **598** global contributors for the Pulsar main GitHub repository
+- **12.1K+** GitHub stars
+- **3.2K+** forks
+- **16** new Committers
+- **5** new PMC members
+- **9560** Pulsar Slack members
+- **4.3K+** downloads
+- **14M+** Docker pulls
+
+The Pulsar community welcomes all kinds of contributions. 

[GitHub] [pulsar-site] tisonkun merged pull request #371: [Blog] Add the Pulsar 2022 Year in Review blog

[GitBox]

tisonkun merged PR #371:
URL: https://github.com/apache/pulsar-site/pull/371


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-site] tisonkun commented on pull request #371: [Blog] Add the Pulsar 2022 Year in Review blog

[GitBox]

tisonkun commented on PR #371:
URL: https://github.com/apache/pulsar-site/pull/371#issuecomment-1381364166

   Merging...
   
   Thanks for your contribution @Sherlock113 and thanks to all reviewers!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[pulsar-site] branch main updated: Allow long modified file list

[tison]

This is an automated email from the ASF dual-hosted git repository.

tison pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git


The following commit(s) were added to refs/heads/main by this push:
 new 6119fdab365 Allow long modified file list
6119fdab365 is described below

commit 6119fdab365b696f0f0727a44afce63b3c13e7ad
Author: tison 
AuthorDate: Fri Jan 13 14:10:36 2023 +0800

Allow long modified file list

Signed-off-by: tison 
---
 tools/pytools/lib/execute/site_builder.py | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/tools/pytools/lib/execute/site_builder.py 
b/tools/pytools/lib/execute/site_builder.py
index d2ecf1638ae..9296b53f3ef 100644
--- a/tools/pytools/lib/execute/site_builder.py
+++ b/tools/pytools/lib/execute/site_builder.py
@@ -16,17 +16,20 @@
 # under the License.
 
 import shutil
+import tempfile
 from pathlib import Path
 
-from command import find_command, run_pipe, run
+from command import find_command, run
 from constant import site_path
 
 
 def execute(asf_site: Path):
 # 1. Get modified files
 git = find_command('git', msg="git is required")
-modified_files = run_pipe(git, 'diff', '--name-only', 'HEAD~1', 'HEAD', 
cwd=site_path()).read().strip()
-modified_files = modified_files.split('\n')
+with tempfile.TemporaryFile('w+') as f:
+run(git, 'diff', '--name-only', 'HEAD~1', 'HEAD', stdout=f, 
cwd=site_path())
+f.seek(0)
+modified_files = f.read().splitlines()
 for file in modified_files:
 print(f"{file} was modified")
 

[GitHub] [pulsar] nodece commented on a diff in pull request #19197: [feat][broker] Update AuthenticationProvider to simplify HTTP Authn

[GitBox]

nodece commented on code in PR #19197:
URL: https://github.com/apache/pulsar/pull/19197#discussion_r1068964150


##
pulsar-broker-common/src/main/java/org/apache/pulsar/broker/web/AuthenticationFilter.java:
##
@@ -52,54 +49,12 @@ public AuthenticationFilter(AuthenticationService 
authenticationService) {
 this.authenticationService = authenticationService;
 }
 
-private boolean isSaslRequest(HttpServletRequest request) {
-if (request.getHeader(SaslConstants.SASL_HEADER_TYPE) == null
-|| 
request.getHeader(SaslConstants.SASL_HEADER_TYPE).isEmpty()) {
-return false;
-}
-if (request.getHeader(SaslConstants.SASL_HEADER_TYPE)
-.equalsIgnoreCase(SaslConstants.SASL_TYPE_VALUE)) {
-return true;
-} else {
-return false;
-}
-}
-
 @Override
 public void doFilter(ServletRequest request, ServletResponse response, 
FilterChain chain)
 throws IOException, ServletException {
 try {
-HttpServletRequest httpRequest = (HttpServletRequest) request;
-HttpServletResponse httpResponse = (HttpServletResponse) response;
-
-if (!isSaslRequest(httpRequest)) {
-// not sasl type, return role directly.
-String authMethodName = 
httpRequest.getHeader(PULSAR_AUTH_METHOD_NAME);
-String role;
-if (authMethodName != null && 
authenticationService.getAuthenticationProvider(authMethodName) != null) {
-AuthenticationState authenticationState = 
authenticationService
-
.getAuthenticationProvider(authMethodName).newHttpAuthState(httpRequest);
-request.setAttribute(AuthenticatedDataAttributeName, 
authenticationState.getAuthDataSource());
-role = authenticationService.authenticateHttpRequest(
-(HttpServletRequest) request, 
authenticationState.getAuthDataSource());
-} else {
-request.setAttribute(AuthenticatedDataAttributeName,
-new AuthenticationDataHttps((HttpServletRequest) 
request));
-role = 
authenticationService.authenticateHttpRequest((HttpServletRequest) request);
-}
-request.setAttribute(AuthenticatedRoleAttributeName, role);
-
-if (LOG.isDebugEnabled()) {
-LOG.debug("[{}] Authenticated HTTP request with role {}", 
request.getRemoteAddr(), role);
-}
-chain.doFilter(request, response);
-return;
-}
-
 boolean doFilter = authenticationService
-.getAuthenticationProvider(SaslConstants.AUTH_METHOD_NAME)
-.authenticateHttpRequest(httpRequest, httpResponse);
-
+.authenticateHttpRequest((HttpServletRequest) request, 
(HttpServletResponse) response);

Review Comment:
   I think we should check the Sasl request to avoid breaking behavior.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] nodece commented on a diff in pull request #19197: [feat][broker] Update AuthenticationProvider to simplify HTTP Authn

[GitBox]

nodece commented on code in PR #19197:
URL: https://github.com/apache/pulsar/pull/19197#discussion_r1068952424


##
pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java:
##
@@ -160,6 +163,20 @@ public String authenticate(AuthenticationDataSource 
authData) throws Authenticat
 }
 }
 
+@Override
+public boolean authenticateHttpRequest(HttpServletRequest request, 
HttpServletResponse response) throws Exception {

Review Comment:
   Don't need to override `authenticateHttpRequest` method.
   
   See `newHttpAuthState` method, we have wrapped the  `request`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-site] tisonkun merged pull request #381: Manually generate 2.11.0 Javadoc

[GitBox]

tisonkun merged PR #381:
URL: https://github.com/apache/pulsar-site/pull/381


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] nodece commented on a diff in pull request #19197: [feat][broker] Update AuthenticationProvider to simplify HTTP Authn

[GitBox]

nodece commented on code in PR #19197:
URL: https://github.com/apache/pulsar/pull/19197#discussion_r1068942276


##
pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProvider.java:
##
@@ -138,10 +145,20 @@ default CompletableFuture 
authenticateHttpRequestAsync(HttpServletReque
 /**
  * Set response, according to passed in request.
  * and return whether we should do following chain.doFilter or not.
+ *
+ * Implementations of this method MUST modify the request by adding the 
{@link AuthenticatedRoleAttributeName}
+ * and the {@link AuthenticatedDataAttributeName} attributes.
+ *
+ * @return Set response, according to passed in request, and return 
whether we should do following chain.doFilter.
+ * @throws Exception when authentication failed
  * @deprecated use and implement {@link 
AuthenticationProvider#authenticateHttpRequestAsync} instead.
  */
 @Deprecated
 default boolean authenticateHttpRequest(HttpServletRequest request, 
HttpServletResponse response) throws Exception {
-throw new AuthenticationException("Not supported");
+AuthenticationState authenticationState = newHttpAuthState(request);

Review Comment:
   I think we should throw an exception, and keep the old design.
   
   ```
   throw new AuthenticationException("Not supported");
   ```
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-site] tisonkun opened a new pull request, #381: Manually generate 2.11.0 Javadoc

[GitBox]

tisonkun opened a new pull request, #381:
URL: https://github.com/apache/pulsar-site/pull/381

   Signed-off-by: tison 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] mattisonchao commented on a diff in pull request #19138: [feat][broker][PIP-195] Implement delayed message index bucket snapshot(merge/delete) - part8

[GitBox]

mattisonchao commented on code in PR #19138:
URL: https://github.com/apache/pulsar/pull/19138#discussion_r1068932778


##
pulsar-broker/src/main/java/org/apache/pulsar/broker/delayed/bucket/BucketDelayedDeliveryTracker.java:
##
@@ -214,11 +227,18 @@ public synchronized boolean addMessage(long ledgerId, 
long entryId, long deliver
 if (!existBucket && ledgerId > lastMutableBucket.endLedgerId
 && lastMutableBucket.size() >= minIndexCountPerBucket
 && !lastMutableBucket.isEmpty()) {
-sealBucket();
+Pair 
immutableBucketDelayedIndexPair =
+
lastMutableBucket.sealBucketAndAsyncPersistent(this.timeStepPerBucketSnapshotSegment,
+this.sharedBucketPriorityQueue);
+afterCreateImmutableBucket(immutableBucketDelayedIndexPair);
 lastMutableBucket.resetLastMutableBucketRange();
 
 if (immutableBuckets.asMapOfRanges().size() > maxNumBuckets) {
-// TODO merge bucket snapshot (synchronize operate)
+try {
+
asyncMergeBucketSnapshot().get(AsyncOperationTimeoutSeconds, TimeUnit.SECONDS);
+} catch (InterruptedException | ExecutionException | 
TimeoutException e) {

Review Comment:
   Maybe not. But I think we should persist in this state if we don't have any 
special purpose.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] mattisonchao commented on a diff in pull request #19138: [feat][broker][PIP-195] Implement delayed message index bucket snapshot(merge/delete) - part8

[GitBox]

mattisonchao commented on code in PR #19138:
URL: https://github.com/apache/pulsar/pull/19138#discussion_r1068929892


##
pulsar-broker/src/main/java/org/apache/pulsar/broker/delayed/bucket/BucketDelayedDeliveryTracker.java:
##
@@ -243,6 +263,53 @@ public synchronized boolean addMessage(long ledgerId, long 
entryId, long deliver
 return true;
 }
 
+private synchronized CompletableFuture asyncMergeBucketSnapshot() {
+List values = 
immutableBuckets.asMapOfRanges().values().stream().toList();
+long minNumberMessages = Long.MAX_VALUE;
+int minIndex = -1;
+for (int i = 0; i + 1 < values.size(); i++) {
+ImmutableBucket bucketL = values.get(i);
+ImmutableBucket bucketR = values.get(i + 1);
+long numberMessages = bucketL.numberBucketDelayedMessages + 
bucketR.numberBucketDelayedMessages;
+if (numberMessages < minNumberMessages) {
+minNumberMessages = (int) numberMessages;
+minIndex = i;
+}
+}
+return asyncMergeBucketSnapshot(values.get(minIndex), 
values.get(minIndex + 1));
+}
+
+private synchronized CompletableFuture 
asyncMergeBucketSnapshot(ImmutableBucket bucketA,
+  
ImmutableBucket bucketB) {
+immutableBuckets.remove(Range.closed(bucketA.startLedgerId, 
bucketA.endLedgerId));
+immutableBuckets.remove(Range.closed(bucketB.startLedgerId, 
bucketB.endLedgerId));
+
+CompletableFuture snapshotCreateFutureA =
+
bucketA.getSnapshotCreateFuture().orElse(CompletableFuture.completedFuture(null));
+CompletableFuture snapshotCreateFutureB =
+
bucketB.getSnapshotCreateFuture().orElse(CompletableFuture.completedFuture(null));
+
+return CompletableFuture.allOf(snapshotCreateFutureA, 
snapshotCreateFutureB).thenCompose(__ -> {
+
CompletableFuture>
 futureA =
+bucketA.getRemainSnapshotSegment();
+
CompletableFuture>
 futureB =
+bucketB.getRemainSnapshotSegment();
+return futureA.thenCombine(futureB, 
CombinedSegmentDelayedIndexQueue::wrap)
+.thenCompose(combinedDelayedIndexQueue -> {
+CompletableFuture removeAFuture = 
bucketA.asyncDeleteBucketSnapshot();
+CompletableFuture removeBFuture = 
bucketB.asyncDeleteBucketSnapshot();
+
+return CompletableFuture.allOf(removeAFuture, 
removeBFuture).thenRun(() -> {

Review Comment:
   Questions: 
   - When is the index rebuilt?
   - Should we unload the topic to rebuild the index if this operation fails?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] mattisonchao commented on a diff in pull request #19138: [feat][broker][PIP-195] Implement delayed message index bucket snapshot(merge/delete) - part8

[GitBox]

mattisonchao commented on code in PR #19138:
URL: https://github.com/apache/pulsar/pull/19138#discussion_r1068929892


##
pulsar-broker/src/main/java/org/apache/pulsar/broker/delayed/bucket/BucketDelayedDeliveryTracker.java:
##
@@ -243,6 +263,53 @@ public synchronized boolean addMessage(long ledgerId, long 
entryId, long deliver
 return true;
 }
 
+private synchronized CompletableFuture asyncMergeBucketSnapshot() {
+List values = 
immutableBuckets.asMapOfRanges().values().stream().toList();
+long minNumberMessages = Long.MAX_VALUE;
+int minIndex = -1;
+for (int i = 0; i + 1 < values.size(); i++) {
+ImmutableBucket bucketL = values.get(i);
+ImmutableBucket bucketR = values.get(i + 1);
+long numberMessages = bucketL.numberBucketDelayedMessages + 
bucketR.numberBucketDelayedMessages;
+if (numberMessages < minNumberMessages) {
+minNumberMessages = (int) numberMessages;
+minIndex = i;
+}
+}
+return asyncMergeBucketSnapshot(values.get(minIndex), 
values.get(minIndex + 1));
+}
+
+private synchronized CompletableFuture 
asyncMergeBucketSnapshot(ImmutableBucket bucketA,
+  
ImmutableBucket bucketB) {
+immutableBuckets.remove(Range.closed(bucketA.startLedgerId, 
bucketA.endLedgerId));
+immutableBuckets.remove(Range.closed(bucketB.startLedgerId, 
bucketB.endLedgerId));
+
+CompletableFuture snapshotCreateFutureA =
+
bucketA.getSnapshotCreateFuture().orElse(CompletableFuture.completedFuture(null));
+CompletableFuture snapshotCreateFutureB =
+
bucketB.getSnapshotCreateFuture().orElse(CompletableFuture.completedFuture(null));
+
+return CompletableFuture.allOf(snapshotCreateFutureA, 
snapshotCreateFutureB).thenCompose(__ -> {
+
CompletableFuture>
 futureA =
+bucketA.getRemainSnapshotSegment();
+
CompletableFuture>
 futureB =
+bucketB.getRemainSnapshotSegment();
+return futureA.thenCombine(futureB, 
CombinedSegmentDelayedIndexQueue::wrap)
+.thenCompose(combinedDelayedIndexQueue -> {
+CompletableFuture removeAFuture = 
bucketA.asyncDeleteBucketSnapshot();
+CompletableFuture removeBFuture = 
bucketB.asyncDeleteBucketSnapshot();
+
+return CompletableFuture.allOf(removeAFuture, 
removeBFuture).thenRun(() -> {

Review Comment:
   Question: 
   When is the index rebuilt?
   Should we unload the topic to rebuild the index if this operation fails?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] poorbarcode commented on a diff in pull request #19138: [feat][broker][PIP-195] Implement delayed message index bucket snapshot(merge/delete) - part8

[GitBox]

poorbarcode commented on code in PR #19138:
URL: https://github.com/apache/pulsar/pull/19138#discussion_r1068900015


##
pulsar-broker/src/main/java/org/apache/pulsar/broker/delayed/bucket/BucketDelayedDeliveryTracker.java:
##
@@ -214,11 +227,18 @@ public synchronized boolean addMessage(long ledgerId, 
long entryId, long deliver
 if (!existBucket && ledgerId > lastMutableBucket.endLedgerId
 && lastMutableBucket.size() >= minIndexCountPerBucket
 && !lastMutableBucket.isEmpty()) {
-sealBucket();
+Pair 
immutableBucketDelayedIndexPair =
+
lastMutableBucket.sealBucketAndAsyncPersistent(this.timeStepPerBucketSnapshotSegment,
+this.sharedBucketPriorityQueue);
+afterCreateImmutableBucket(immutableBucketDelayedIndexPair);
 lastMutableBucket.resetLastMutableBucketRange();
 
 if (immutableBuckets.asMapOfRanges().size() > maxNumBuckets) {
-// TODO merge bucket snapshot (synchronize operate)
+try {
+
asyncMergeBucketSnapshot().get(AsyncOperationTimeoutSeconds, TimeUnit.SECONDS);
+} catch (InterruptedException | ExecutionException | 
TimeoutException e) {

Review Comment:
   Hi @mattisonchao 
   I wonder why we should keep the `interrupted` state. Is this state to be 
handed over to netty?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-client-cpp] shibd commented on a diff in pull request #154: [feat] Support partitioned topic reader.

[GitBox]

shibd commented on code in PR #154:
URL: https://github.com/apache/pulsar-client-cpp/pull/154#discussion_r1068899826


##
include/pulsar/ConsumerConfiguration.h:
##
@@ -383,6 +383,26 @@ class PULSAR_PUBLIC ConsumerConfiguration {
  */
 InitialPosition getSubscriptionInitialPosition() const;
 
+/**
+ * Selects the subscription mode to be used when subscribing to a topic.
+ *
+ * Options are:
+ * 
+ *  {@link SubscriptionMode#Durable} (Default)
+ *  {@link SubscriptionMode#NonDurable}
+ * 
+ *
+ * @param subscriptionMode the subscription mode value
+ */
+ConsumerConfiguration& setSubscriptionMode(SubscriptionMode 
subscriptionMode);
+
+/**
+ * Get subscription mode.
+ *
+ * @return
+ */
+SubscriptionMode getSubscriptionMode() const;

Review Comment:
   done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] horizonzy commented on pull request #19177: [improve][branch-2.8]Upgrade the bk version to 4.14.6

[GitBox]

horizonzy commented on PR #19177:
URL: https://github.com/apache/pulsar/pull/19177#issuecomment-1381264712

   > Please hold on to the upgrade due to the RocksDB issue. 
[apache/bookkeeper#3734](https://github.com/apache/bookkeeper/issues/3734)
   
   The issue concerns the bookie server; upgrading the bookkeeper version is 
compatible with the old bookie server. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-site] tisonkun commented on pull request #371: [Blog] Add the Pulsar 2022 Year in Review blog

[GitBox]

tisonkun commented on PR #371:
URL: https://github.com/apache/pulsar-site/pull/371#issuecomment-1381257026

   Pending to merge...
   
   I'm going to merge this patch in hours if there are no more comments.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] congbobo184 commented on a diff in pull request #18273: [feat][txn] implement the SnapshotSegmentAbortedTxnProcessor

[GitBox]

congbobo184 commented on code in PR #18273:
URL: https://github.com/apache/pulsar/pull/18273#discussion_r1067649532


##
pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/buffer/impl/SnapshotSegmentAbortedTxnProcessorImpl.java:
##
@@ -0,0 +1,651 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,2
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.broker.transaction.buffer.impl;
+
+import io.netty.buffer.ByteBuf;
+import io.netty.buffer.Unpooled;
+import java.util.ArrayList;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentLinkedDeque;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.atomic.AtomicReferenceFieldUpdater;
+import java.util.function.Function;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.bookkeeper.mledger.AsyncCallbacks;
+import org.apache.bookkeeper.mledger.Entry;
+import org.apache.bookkeeper.mledger.ManagedLedgerException;
+import org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl;
+import org.apache.bookkeeper.mledger.impl.PositionImpl;
+import org.apache.bookkeeper.mledger.impl.ReadOnlyManagedLedgerImpl;
+import org.apache.commons.collections4.map.LinkedMap;
+import org.apache.commons.lang3.tuple.MutablePair;
+import org.apache.commons.lang3.tuple.Pair;
+import org.apache.pulsar.broker.service.BrokerServiceException;
+import org.apache.pulsar.broker.service.persistent.PersistentTopic;
+import org.apache.pulsar.broker.systopic.SystemTopicClient;
+import org.apache.pulsar.broker.transaction.buffer.AbortedTxnProcessor;
+import 
org.apache.pulsar.broker.transaction.buffer.metadata.v2.TransactionBufferSnapshotIndex;
+import 
org.apache.pulsar.broker.transaction.buffer.metadata.v2.TransactionBufferSnapshotIndexes;
+import 
org.apache.pulsar.broker.transaction.buffer.metadata.v2.TransactionBufferSnapshotIndexesMetadata;
+import 
org.apache.pulsar.broker.transaction.buffer.metadata.v2.TransactionBufferSnapshotSegment;
+import org.apache.pulsar.broker.transaction.buffer.metadata.v2.TxnIDData;
+import org.apache.pulsar.client.api.Message;
+import org.apache.pulsar.client.api.Schema;
+import org.apache.pulsar.client.api.transaction.TxnID;
+import org.apache.pulsar.client.impl.MessageIdImpl;
+import org.apache.pulsar.client.impl.PulsarClientImpl;
+import org.apache.pulsar.common.naming.SystemTopicNames;
+import org.apache.pulsar.common.naming.TopicDomain;
+import org.apache.pulsar.common.naming.TopicName;
+import org.apache.pulsar.common.protocol.Commands;
+import org.apache.pulsar.common.util.FutureUtil;
+
+@Slf4j
+public class SnapshotSegmentAbortedTxnProcessorImpl implements 
AbortedTxnProcessor {
+
+//Stored the unsealed aborted txn id, it will be persistent as a snapshot 
segment and reinit
+// when its size reach the capital of a snapshot segment.
+private LinkedList unsealedAbortedTxnIdSegment;
+
+//A mapping form the latest txn mark persistent position in a segment to 
its latest txn ID.
+//This is mainly used to trim expired snapshot segment and remove them 
form aborts.
+private final LinkedMap segmentIndex = new 
LinkedMap<>();
+
+//Store all aborted txn IDs check whether a txn is an aborted txn.
+private final LinkedMap aborts = new LinkedMap<>();
+//The indexes of the snapshot segments whose key is the aborted mark 
persistent position.
+private final LinkedMap 
indexes = new LinkedMap<>();
+//The latest persistent snapshot index. This is used to combine new 
segment indexes with the latest metadata and
+// indexes.
+private TransactionBufferSnapshotIndexes persistentSnapshotIndexes = new 
TransactionBufferSnapshotIndexes();
+
+private final PersistentTopic topic;
+
+private volatile long lastSnapshotTimestamps;
+
+private final int transactionBufferMaxAbortedTxnsOfSnapshotSegment;
+private final PersistentWorker persistentWorker;
+
+private static final String SNAPSHOT_PREFIX = "multiple-";
+
+public SnapshotSegmentAbortedTxnProcessorImpl(PersistentTopic topic) {
+

[GitHub] [pulsar-site] tisonkun commented on a diff in pull request #367: [blog] Add 2.10.3 release blog

[GitBox]

tisonkun commented on code in PR #367:
URL: https://github.com/apache/pulsar-site/pull/367#discussion_r1068868191


##
blog/2023-01-09-Apache-Pulsar-2-10-3.md:
##
@@ -0,0 +1,54 @@
+---
+title: "What’s New in Apache Pulsar 2.10.3"
+date: 2023-01-09
+author: "liangyepianzhou, momo-jun"
+---
+
+The Apache Pulsar community releases version 2.10.3! 50 contributors provided 
improvements and bug fixes that delivered 155 commits. Thanks for all your 
contributions.
+
+This blog walks through the most noteworthy changes. For the complete list 
including all feature enhancements and bug fixes, check out the [Pulsar 2.10.3 
Release 
Notes](https://pulsar.apache.org/release-notes/versioned/pulsar-2.10.3/).

Review Comment:
   Please add a `` marker to avoid showing the whole blog on 
the overview page.
   
   See https://docusaurus.io/docs/blog#blog-list.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] mattisonchao commented on a diff in pull request #19138: [feat][broker][PIP-195] Implement delayed message index bucket snapshot(merge/delete) - part8

[GitBox]

mattisonchao commented on code in PR #19138:
URL: https://github.com/apache/pulsar/pull/19138#discussion_r1068864518


##
pulsar-broker/src/main/java/org/apache/pulsar/broker/delayed/bucket/BucketDelayedDeliveryTracker.java:
##
@@ -243,6 +263,53 @@ public synchronized boolean addMessage(long ledgerId, long 
entryId, long deliver
 return true;
 }
 
+private synchronized CompletableFuture asyncMergeBucketSnapshot() {
+List values = 
immutableBuckets.asMapOfRanges().values().stream().toList();
+long minNumberMessages = Long.MAX_VALUE;

Review Comment:
   maximum?



##
pulsar-broker/src/main/java/org/apache/pulsar/broker/delayed/bucket/BucketDelayedDeliveryTracker.java:
##
@@ -243,6 +263,53 @@ public synchronized boolean addMessage(long ledgerId, long 
entryId, long deliver
 return true;
 }
 
+private synchronized CompletableFuture asyncMergeBucketSnapshot() {
+List values = 
immutableBuckets.asMapOfRanges().values().stream().toList();
+long minNumberMessages = Long.MAX_VALUE;

Review Comment:
   Maximum? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] mattisonchao commented on a diff in pull request #19138: [feat][broker][PIP-195] Implement delayed message index bucket snapshot(merge/delete) - part8

[GitBox]

mattisonchao commented on code in PR #19138:
URL: https://github.com/apache/pulsar/pull/19138#discussion_r1068862483


##
pulsar-broker/src/main/java/org/apache/pulsar/broker/delayed/bucket/BucketDelayedDeliveryTracker.java:
##
@@ -144,7 +155,12 @@ private synchronized long recoverBucketSnapshot() throws 
RuntimeException {
 }
 
 try {
-FutureUtil.waitForAll(futures).get(AsyncOperationTimeoutSeconds, 
TimeUnit.SECONDS);
+FutureUtil.waitForAll(futures).whenComplete((__, ex) -> {
+toBeDeletedBucketMap.forEach((k, immutableBucket) -> {
+immutableBuckets.remove(k);
+immutableBucket.asyncDeleteBucketSnapshot();

Review Comment:
   We need a log to show the exception.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] mattisonchao commented on a diff in pull request #19138: [feat][broker][PIP-195] Implement delayed message index bucket snapshot(merge/delete) - part8

[GitBox]

mattisonchao commented on code in PR #19138:
URL: https://github.com/apache/pulsar/pull/19138#discussion_r1068862368


##
pulsar-broker/src/main/java/org/apache/pulsar/broker/delayed/bucket/BucketDelayedDeliveryTracker.java:
##
@@ -308,6 +375,10 @@ public synchronized NavigableSet 
getScheduledMessages(int maxMessa
 try {
 
bucket.asyncLoadNextBucketSnapshotEntry().thenAccept(indexList -> {
 if (CollectionUtils.isEmpty(indexList)) {
+synchronized (immutableBuckets) {
+
immutableBuckets.remove(Range.closed(bucket.startLedgerId, bucket.endLedgerId));
+}
+bucket.asyncDeleteBucketSnapshot();

Review Comment:
   We need a log to show the exception.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-site] momo-jun commented on a diff in pull request #367: [blog] Add 2.10.3 release blog

[GitBox]

momo-jun commented on code in PR #367:
URL: https://github.com/apache/pulsar-site/pull/367#discussion_r1068861159


##
blog/2023-01-09-Apache-Pulsar-2-10-3.md:
##
@@ -0,0 +1,54 @@
+---
+title: "What’s New in Apache Pulsar 2.10.3"
+date: 2023-01-09
+author: "liangyepianzhou, momo-jun"
+---
+
+The Apache Pulsar community releases version 2.10.3! 50 contributors provided 
improvements and bug fixes that delivered 155 commits. Thanks for all your 
contributions.
+
+This blog walks through the most noteworthy changes. For the complete list 
including all feature enhancements and bug fixes, check out the [Pulsar 2.10.3 
Release 
Notes](https://pulsar.apache.org/release-notes/versioned/pulsar-2.10.3/).
+
+### Add `getState` in transactions for client APIs 
([PR-17423](https://github.com/apache/pulsar/pull/17423))
+
+ Issue
+In earlier versions, the state of transactions on the client side could not be 
obtained with the client APIs. Users have no way to check the state of 
transactions before ending them.

Review Comment:
   Thanks for raising this question. Here is my thought: users who are using 
the earlier versions still face the issue today. It was not past for those who 
may need an upgrade.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] mattisonchao commented on a diff in pull request #19138: [feat][broker][PIP-195] Implement delayed message index bucket snapshot(merge/delete) - part8

[GitBox]

mattisonchao commented on code in PR #19138:
URL: https://github.com/apache/pulsar/pull/19138#discussion_r1068860037


##
pulsar-broker/src/main/java/org/apache/pulsar/broker/delayed/bucket/ImmutableBucket.java:
##
@@ -132,12 +131,26 @@ private void recoverDelayedIndexBitMapAndNumber(int 
startSnapshotIndex,
 this.setNumberBucketDelayedMessages(numberMessages.getValue());
 }
 
+
CompletableFuture>
 getRemainSnapshotSegment() {
+return 
bucketSnapshotStorage.getBucketSnapshotSegment(getAndUpdateBucketId(), 
currentSegmentEntryId,
+lastSegmentEntryId);
+}
+
+CompletableFuture asyncDeleteBucketSnapshot() {
+return removeBucketCursorProperty(bucketKey()).thenCompose(__ ->
+
bucketSnapshotStorage.deleteBucketSnapshot(getAndUpdateBucketId()));
+}
+
 void clear(boolean delete) {
 delayedIndexBitMap.clear();
 getSnapshotCreateFuture().ifPresent(snapshotGenerateFuture -> {
 if (delete) {
 snapshotGenerateFuture.cancel(true);
-// TODO delete bucket snapshot
+try {
+
asyncDeleteBucketSnapshot().get(AsyncOperationTimeoutSeconds, TimeUnit.SECONDS);
+} catch (Exception e) {

Review Comment:
   Do you consider refactoring this code? Please catch the special exception if 
no.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-site] Technoboy- commented on a diff in pull request #373: Update release note for 2.11.0

[GitBox]

Technoboy- commented on code in PR #373:
URL: https://github.com/apache/pulsar-site/pull/373#discussion_r1068857083


##
release-notes/versioned/client-cpp-2.11.0.md:
##
@@ -0,0 +1,35 @@
+---
+id: client-cpp-2.11.0

Review Comment:
   Removed, thanks



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[pulsar-site] branch release-2.11.0 updated (203253d6d5b -> ac261cf8da1)

[technoboy]

This is an automated email from the ASF dual-hosted git repository.

technoboy pushed a change to branch release-2.11.0
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git


from 203253d6d5b address comment.
 add ac261cf8da1 delete cpp and py notes.

No new revisions were added by this update.

Summary of changes:
 release-notes/versioned/client-cpp-2.11.0.md| 35 --
 release-notes/versioned/client-python-2.11.0.md | 17 ---
 release-notes/versioned/pulsar-2.11.0.md| 40 -
 3 files changed, 92 deletions(-)
 delete mode 100644 release-notes/versioned/client-cpp-2.11.0.md
 delete mode 100644 release-notes/versioned/client-python-2.11.0.md

[GitHub] [pulsar] mattisonchao commented on a diff in pull request #19138: [feat][broker][PIP-195] Implement delayed message index bucket snapshot(merge/delete) - part8

[GitBox]

mattisonchao commented on code in PR #19138:
URL: https://github.com/apache/pulsar/pull/19138#discussion_r1068855546


##
pulsar-broker/src/main/java/org/apache/pulsar/broker/delayed/bucket/BucketDelayedDeliveryTracker.java:
##
@@ -214,11 +227,18 @@ public synchronized boolean addMessage(long ledgerId, 
long entryId, long deliver
 if (!existBucket && ledgerId > lastMutableBucket.endLedgerId
 && lastMutableBucket.size() >= minIndexCountPerBucket
 && !lastMutableBucket.isEmpty()) {
-sealBucket();
+Pair 
immutableBucketDelayedIndexPair =
+
lastMutableBucket.sealBucketAndAsyncPersistent(this.timeStepPerBucketSnapshotSegment,
+this.sharedBucketPriorityQueue);
+afterCreateImmutableBucket(immutableBucketDelayedIndexPair);
 lastMutableBucket.resetLastMutableBucketRange();
 
 if (immutableBuckets.asMapOfRanges().size() > maxNumBuckets) {
-// TODO merge bucket snapshot (synchronize operate)
+try {
+
asyncMergeBucketSnapshot().get(AsyncOperationTimeoutSeconds, TimeUnit.SECONDS);
+} catch (InterruptedException | ExecutionException | 
TimeoutException e) {
+throw new RuntimeException(e);

Review Comment:
   Do you think we should consider the specific exception?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] mattisonchao commented on a diff in pull request #19138: [feat][broker][PIP-195] Implement delayed message index bucket snapshot(merge/delete) - part8

[GitBox]

mattisonchao commented on code in PR #19138:
URL: https://github.com/apache/pulsar/pull/19138#discussion_r1068855318


##
pulsar-broker/src/main/java/org/apache/pulsar/broker/delayed/bucket/BucketDelayedDeliveryTracker.java:
##
@@ -214,11 +227,18 @@ public synchronized boolean addMessage(long ledgerId, 
long entryId, long deliver
 if (!existBucket && ledgerId > lastMutableBucket.endLedgerId
 && lastMutableBucket.size() >= minIndexCountPerBucket
 && !lastMutableBucket.isEmpty()) {
-sealBucket();
+Pair 
immutableBucketDelayedIndexPair =
+
lastMutableBucket.sealBucketAndAsyncPersistent(this.timeStepPerBucketSnapshotSegment,
+this.sharedBucketPriorityQueue);
+afterCreateImmutableBucket(immutableBucketDelayedIndexPair);
 lastMutableBucket.resetLastMutableBucketRange();
 
 if (immutableBuckets.asMapOfRanges().size() > maxNumBuckets) {
-// TODO merge bucket snapshot (synchronize operate)
+try {
+
asyncMergeBucketSnapshot().get(AsyncOperationTimeoutSeconds, TimeUnit.SECONDS);
+} catch (InterruptedException | ExecutionException | 
TimeoutException e) {

Review Comment:
   Please call `Thread.currentThread().interrupt();` if we get 
`InterruptedException`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] github-actions[bot] commented on pull request #17751: [fix][broker] Fix the markdelete position does not move forward when isAutoSkipNonRecoverableData=true and individual ack

[GitBox]

github-actions[bot] commented on PR #17751:
URL: https://github.com/apache/pulsar/pull/17751#issuecomment-1381219394

   The pr had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] github-actions[bot] commented on issue #18903: Support processingGuarantees "EFFECTIVELY_ONCE" in python function

[GitBox]

github-actions[bot] commented on issue #18903:
URL: https://github.com/apache/pulsar/issues/18903#issuecomment-1381218064

   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] github-actions[bot] commented on issue #18908: [configuration between Pulsar and MongoDB Atlas not working]

[GitBox]

github-actions[bot] commented on issue #18908:
URL: https://github.com/apache/pulsar/issues/18908#issuecomment-1381218031

   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] codecov-commenter commented on pull request #19222: [fix][test] Fix flaky test CompactionReaderImplTest

[GitBox]

codecov-commenter commented on PR #19222:
URL: https://github.com/apache/pulsar/pull/19222#issuecomment-1381214261

   # 
[Codecov](https://codecov.io/gh/apache/pulsar/pull/19222?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 Report
   > Merging 
[#19222](https://codecov.io/gh/apache/pulsar/pull/19222?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 (abb5658) into 
[master](https://codecov.io/gh/apache/pulsar/commit/d7f8f56b5be5ad34cf1d33dd89b7bfc6a9ab94ff?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 (d7f8f56) will **decrease** coverage by `10.55%`.
   > The diff coverage is `15.38%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/pulsar/pull/19222/graphs/tree.svg?width=650=150=pr=acYqCpsK9J_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pulsar/pull/19222?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
   
   ```diff
   @@  Coverage Diff  @@
   ## master   #19222   +/-   ##
   =
   - Coverage 47.89%   37.33%   -10.56% 
   + Complexity10871 1995 -8876 
   =
 Files   713  209  -504 
 Lines 6973014453-55277 
 Branches   7496 1578 -5918 
   =
   - Hits  33394 5396-27998 
   + Misses32631 8475-24156 
   + Partials   3705  582 -3123 
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `37.33% <15.38%> (-10.56%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click 
here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment)
 to find out more.
   
   | [Impacted 
Files](https://codecov.io/gh/apache/pulsar/pull/19222?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 | Coverage Δ | |
   |---|---|---|
   | 
[...va/org/apache/pulsar/client/impl/ProducerImpl.java](https://codecov.io/gh/apache/pulsar/pull/19222?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1Byb2R1Y2VySW1wbC5qYXZh)
 | `15.62% <0.00%> (-0.04%)` | :arrow_down: |
   | 
[...e/pulsar/client/impl/schema/reader/JsonReader.java](https://codecov.io/gh/apache/pulsar/pull/19222?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL3NjaGVtYS9yZWFkZXIvSnNvblJlYWRlci5qYXZh)
 | `0.00% <0.00%> (ø)` | |
   | 
[...r/client/impl/schema/reader/JacksonJsonReader.java](https://codecov.io/gh/apache/pulsar/pull/19222?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL3NjaGVtYS9yZWFkZXIvSmFja3Nvbkpzb25SZWFkZXIuamF2YQ==)
 | `66.66% <66.66%> (-6.67%)` | :arrow_down: |
   | 
[...g/apache/pulsar/broker/stats/MetricsGenerator.java](https://codecov.io/gh/apache/pulsar/pull/19222?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zdGF0cy9NZXRyaWNzR2VuZXJhdG9yLmphdmE=)
 | | |
   | 
[...ats/prometheus/PulsarPrometheusMetricsServlet.java](https://codecov.io/gh/apache/pulsar/pull/19222?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zdGF0cy9wcm9tZXRoZXVzL1B1bHNhclByb21ldGhldXNNZXRyaWNzU2VydmxldC5qYXZh)
 | | |
   | 
[...ar/broker/loadbalance/impl/BundleSplitterTask.java](https://codecov.io/gh/apache/pulsar/pull/19222?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9sb2FkYmFsYW5jZS9pbXBsL0J1bmRsZVNwbGl0dGVyVGFzay5qYXZh)
 | | |
   | 

[GitHub] [pulsar-client-node] shibd commented on pull request #282: [fix] Use OAuth2 on macOX arm64 failed.

[GitBox]

shibd commented on PR #282:
URL: 
https://github.com/apache/pulsar-client-node/pull/282#issuecomment-1381213324

   @ericallam Hi, 
   
   I 
[released](https://github.com/shibd/pulsar-client-node/actions/runs/3904096695) 
a hotfix version in my repository using this PR change, and I can use `oauth2 + 
ssl` normally on my arm64 machine. (Version 13.1)
   
   Could you refer to this 
[repository](https://github.com/shibd/pulsar-node-oauth2-ssl-test) README.md to 
see if it can be run on your host?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[pulsar-site] branch main updated: [fix][doc] Correct configuration requirements for the HDFS sinks (#364)

[liuyu]

This is an automated email from the ASF dual-hosted git repository.

liuyu pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git


The following commit(s) were added to refs/heads/main by this push:
 new 753bc8efc94 [fix][doc] Correct configuration requirements for the HDFS 
sinks (#364)
753bc8efc94 is described below

commit 753bc8efc943ae3e6154e9ef44d8d275bc56dd7d
Author: Kengo Seki 
AuthorDate: Fri Jan 13 10:52:21 2023 +0900

[fix][doc] Correct configuration requirements for the HDFS sinks (#364)
---
 docs/io-hdfs2-sink.md | 4 ++--
 docs/io-hdfs3-sink.md | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/docs/io-hdfs2-sink.md b/docs/io-hdfs2-sink.md
index 06ffd003c3a..1a7f851daf3 100644
--- a/docs/io-hdfs2-sink.md
+++ b/docs/io-hdfs2-sink.md
@@ -26,8 +26,8 @@ The configuration of the HDFS2 sink connector has the 
following properties.
 | `compression` | Compression |false |None |The compression code used to 
compress or de-compress the files on HDFS. Below are the available 
options:BZIP2DEFLATEGZIPLZ4SNAPPYZSTANDARD|
 | `kerberosUserPrincipal` |String| false| None|The principal account of 
Kerberos user used for authentication. |
 | `keytab` | String|false|None| The full pathname of the Kerberos keytab file 
used for authentication. |
-| `filenamePrefix` |String| true, if `compression` is set to `None`. | None 
|The prefix of the files created inside the HDFS directory.**Example** The value of topicA result in files named topicA-. |
-| `fileExtension` | String| true | None | The extension added to the files 
written to HDFS.**Example**'.txt' '.seq' |
+| `filenamePrefix` |String| true | None |The prefix of the files created 
inside the HDFS directory.**Example** The value of topicA 
result in files named topicA-. |
+| `fileExtension` | String| true, if `compression` is set to `None`. | None | 
The extension added to the files written to HDFS.**Example**'.txt' '.seq' |
 | `separator` | char|false |None |The character used to separate records in a 
text file. If no value is provided, the contents from all records 
are concatenated together in one continuous byte array. |
 | `syncInterval` | long| false |0| The interval between calls to flush data to 
HDFS disk in milliseconds. |
 | `maxPendingRecords` |int| false|Integer.MAX_VALUE |  The maximum number of 
records that hold in memory before acking. Setting this property to 
1 makes every record send to disk before the record is acked.Setting this property to a higher value allows buffering records before 
flushing them to disk. 
diff --git a/docs/io-hdfs3-sink.md b/docs/io-hdfs3-sink.md
index eff9bdfca42..6e9f3a2f3b8 100644
--- a/docs/io-hdfs3-sink.md
+++ b/docs/io-hdfs3-sink.md
@@ -26,8 +26,8 @@ The configuration of the HDFS3 sink connector has the 
following properties.
 | `compression` | Compression |false |None |The compression code used to 
compress or de-compress the files on HDFS. Below are the available 
options:BZIP2DEFLATEGZIPLZ4SNAPPYZSTANDARD|
 | `kerberosUserPrincipal` |String| false| None|The principal account of 
Kerberos user used for authentication. |
 | `keytab` | String|false|None| The full pathname of the Kerberos keytab file 
used for authentication. |
-| `filenamePrefix` |String| false |None |The prefix of the files created 
inside the HDFS directory.**Example** The value of topicA 
result in files named topicA-. |
-| `fileExtension` | String| false | None| The extension added to the files 
written to HDFS.**Example**'.txt' '.seq' |
+| `filenamePrefix` |String| true |None |The prefix of the files created inside 
the HDFS directory.**Example** The value of topicA result in 
files named topicA-. |
+| `fileExtension` | String| true, if `compression` is set to `None`. | None| 
The extension added to the files written to HDFS.**Example**'.txt' '.seq' |
 | `separator` | char|false |None |The character used to separate records in a 
text file. If no value is provided, the contents from all records 
are concatenated together in one continuous byte array. |
 | `syncInterval` | long| false |0| The interval between calls to flush data to 
HDFS disk in milliseconds. |
 | `maxPendingRecords` |int| false|Integer.MAX_VALUE |  The maximum number of 
records that hold in memory before acking. Setting this property to 
1 makes every record send to disk before the record is acked.Setting this property to a higher value allows buffering records before 
flushing them to disk. 

[GitHub] [pulsar-site] Anonymitaet merged pull request #364: [fix][doc] Correct configuration requirements for the HDFS sinks

[GitBox]

Anonymitaet merged PR #364:
URL: https://github.com/apache/pulsar-site/pull/364


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-site] Anonymitaet commented on a diff in pull request #367: [blog] Add 2.10.3 release blog

[GitBox]

Anonymitaet commented on code in PR #367:
URL: https://github.com/apache/pulsar-site/pull/367#discussion_r1068839021


##
blog/2023-01-09-Apache-Pulsar-2-10-3.md:
##
@@ -0,0 +1,54 @@
+---
+title: "What’s New in Apache Pulsar 2.10.3"
+date: 2023-01-09
+author: "liangyepianzhou, momo-jun"
+---
+
+The Apache Pulsar community releases version 2.10.3! 50 contributors provided 
improvements and bug fixes that delivered 155 commits. Thanks for all your 
contributions.
+
+This blog walks through the most noteworthy changes. For the complete list 
including all feature enhancements and bug fixes, check out the [Pulsar 2.10.3 
Release 
Notes](https://pulsar.apache.org/release-notes/versioned/pulsar-2.10.3/).
+
+### Add `getState` in transactions for client APIs 
([PR-17423](https://github.com/apache/pulsar/pull/17423))
+
+ Issue
+In earlier versions, the state of transactions on the client side could not be 
obtained with the client APIs. Users have no way to check the state of 
transactions before ending them.

Review Comment:
   Shall we consider using "past tense" since they were facts/actions that 
existed/took in the past?
   
   For example, https://pulsar.apache.org/blog/2022/04/08/Apache-Pulsar-2-9-2/
   
   https://user-images.githubusercontent.com/50226895/212218528-af6efa78-6193-4ca5-8565-0aac1e301f7d.png;>
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] codecov-commenter commented on pull request #19223: [improve][broker] Copy subscription properties during updating the topic partition number.

[GitBox]

codecov-commenter commented on PR #19223:
URL: https://github.com/apache/pulsar/pull/19223#issuecomment-1381209012

   # 
[Codecov](https://codecov.io/gh/apache/pulsar/pull/19223?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 Report
   > Merging 
[#19223](https://codecov.io/gh/apache/pulsar/pull/19223?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 (516d009) into 
[master](https://codecov.io/gh/apache/pulsar/commit/d7f8f56b5be5ad34cf1d33dd89b7bfc6a9ab94ff?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 (d7f8f56) will **decrease** coverage by `1.19%`.
   > The diff coverage is `100.00%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/pulsar/pull/19223/graphs/tree.svg?width=650=150=pr=acYqCpsK9J_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pulsar/pull/19223?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
   
   ```diff
   @@ Coverage Diff  @@
   ## master   #19223  +/-   ##
   
   - Coverage 47.89%   46.69%   -1.20% 
   + Complexity10871 6683-4188 
   
 Files   713  398 -315 
 Lines 6973043225   -26505 
 Branches   7496 4428-3068 
   
   - Hits  3339420182   -13212 
   + Misses3263120737   -11894 
   + Partials   3705 2306-1399 
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `46.69% <100.00%> (-1.20%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click 
here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment)
 to find out more.
   
   | [Impacted 
Files](https://codecov.io/gh/apache/pulsar/pull/19223?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 | Coverage Δ | |
   |---|---|---|
   | 
[...pulsar/broker/admin/impl/PersistentTopicsBase.java](https://codecov.io/gh/apache/pulsar/pull/19223?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9hZG1pbi9pbXBsL1BlcnNpc3RlbnRUb3BpY3NCYXNlLmphdmE=)
 | `58.90% <100.00%> (-0.76%)` | :arrow_down: |
   | 
[...pache/pulsar/client/impl/CompactionReaderImpl.java](https://codecov.io/gh/apache/pulsar/pull/19223?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL0NvbXBhY3Rpb25SZWFkZXJJbXBsLmphdmE=)
 | `0.00% <0.00%> (-90.00%)` | :arrow_down: |
   | 
[...g/apache/pulsar/client/impl/RawBatchConverter.java](https://codecov.io/gh/apache/pulsar/pull/19223?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1Jhd0JhdGNoQ29udmVydGVyLmphdmE=)
 | `3.12% <0.00%> (-89.07%)` | :arrow_down: |
   | 
[.../pulsar/compaction/StrategicTwoPhaseCompactor.java](https://codecov.io/gh/apache/pulsar/pull/19223?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NvbXBhY3Rpb24vU3RyYXRlZ2ljVHdvUGhhc2VDb21wYWN0b3IuamF2YQ==)
 | `0.00% <0.00%> (-76.20%)` | :arrow_down: |
   | 
[.../pulsar/broker/service/SharedConsumerAssignor.java](https://codecov.io/gh/apache/pulsar/pull/19223?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL1NoYXJlZENvbnN1bWVyQXNzaWdub3IuamF2YQ==)
 | `3.70% <0.00%> (-74.08%)` | :arrow_down: |
   | 
[...lsar/client/impl/RawBatchMessageContainerImpl.java](https://codecov.io/gh/apache/pulsar/pull/19223?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1Jhd0JhdGNoTWVzc2FnZUNvbnRhaW5lckltcGwuamF2YQ==)
 | `0.00% <0.00%> (-73.14%)` | :arrow_down: |
   | 

[GitHub] [pulsar] Demogorgon314 commented on a diff in pull request #18810: [improve][broker] PIP-192: Implement broker registry for new load manager

[GitBox]

Demogorgon314 commented on code in PR #18810:
URL: https://github.com/apache/pulsar/pull/18810#discussion_r1068833098


##
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/BrokerRegistryImpl.java:
##
@@ -0,0 +1,222 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.broker.loadbalance.extensions;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.Lists;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.RejectedExecutionException;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.BiConsumer;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.pulsar.broker.PulsarServerException;
+import org.apache.pulsar.broker.PulsarService;
+import org.apache.pulsar.broker.ServiceConfiguration;
+import org.apache.pulsar.broker.loadbalance.extensions.data.BrokerLookupData;
+import org.apache.pulsar.common.util.FutureUtil;
+import org.apache.pulsar.metadata.api.MetadataStoreException;
+import org.apache.pulsar.metadata.api.Notification;
+import org.apache.pulsar.metadata.api.NotificationType;
+import org.apache.pulsar.metadata.api.coordination.LockManager;
+import org.apache.pulsar.metadata.api.coordination.ResourceLock;
+
+/**
+ * The broker registry impl, base on the LockManager.
+ */
+@Slf4j
+public class BrokerRegistryImpl implements BrokerRegistry {
+
+protected static final String LOOKUP_DATA_PATH = "/loadbalance/brokers";
+
+private final PulsarService pulsar;
+
+private final ServiceConfiguration conf;
+
+private final BrokerLookupData brokerLookupData;
+
+private final LockManager brokerLookupDataLockManager;
+
+private final String brokerId;
+
+private final ScheduledExecutorService scheduler;
+
+private final List> listeners;
+
+private volatile ResourceLock brokerLookupDataLock;
+
+private volatile boolean registered = false;
+
+private final AtomicBoolean started = new AtomicBoolean(false);
+
+public BrokerRegistryImpl(PulsarService pulsar) {
+this.pulsar = pulsar;
+this.conf = pulsar.getConfiguration();
+this.brokerLookupDataLockManager = 
pulsar.getCoordinationService().getLockManager(BrokerLookupData.class);
+this.scheduler = pulsar.getLoadManagerExecutor();
+this.listeners = new ArrayList<>();
+this.brokerId = pulsar.getLookupServiceAddress();
+this.brokerLookupData = new BrokerLookupData(
+pulsar.getSafeWebServiceAddress(),
+pulsar.getWebServiceAddressTls(),
+pulsar.getBrokerServiceUrl(),
+pulsar.getBrokerServiceUrlTls(),
+pulsar.getAdvertisedListeners(),
+pulsar.getProtocolDataToAdvertise(),
+pulsar.getConfiguration().isEnablePersistentTopics(),
+pulsar.getConfiguration().isEnableNonPersistentTopics(),
+pulsar.getBrokerVersion());
+}
+
+@Override
+public void start() throws PulsarServerException {
+if (started.get()) {
+return;
+}
+
pulsar.getLocalMetadataStore().registerListener(this::handleMetadataStoreNotification);
+try {
+this.register();
+this.started.set(true);
+} catch (MetadataStoreException e) {
+throw new PulsarServerException(e);
+}
+}
+
+@Override
+public boolean isStarted() {
+return this.started.get();
+}
+
+@Override
+public synchronized void register() throws MetadataStoreException {
+if (!registered) {
+try {
+this.brokerLookupDataLock = 
brokerLookupDataLockManager.acquireLock(keyPath(brokerId), brokerLookupData)
+

[GitHub] [pulsar] heesung-sn commented on a diff in pull request #19102: [improve][broker] PIP-192: Implement extensible load manager

[GitBox]

heesung-sn commented on code in PR #19102:
URL: https://github.com/apache/pulsar/pull/19102#discussion_r1068833055


##
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/ExtensibleLoadManagerImpl.java:
##
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.broker.loadbalance.extensions;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ThreadLocalRandom;
+import java.util.concurrent.atomic.AtomicBoolean;
+import lombok.Getter;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.pulsar.broker.PulsarServerException;
+import org.apache.pulsar.broker.PulsarService;
+import org.apache.pulsar.broker.ServiceConfiguration;
+import org.apache.pulsar.broker.loadbalance.BrokerFilterException;
+import 
org.apache.pulsar.broker.loadbalance.extensions.channel.ServiceUnitStateChannel;
+import 
org.apache.pulsar.broker.loadbalance.extensions.channel.ServiceUnitStateChannelImpl;
+import org.apache.pulsar.broker.loadbalance.extensions.data.BrokerLookupData;
+import org.apache.pulsar.broker.loadbalance.extensions.filter.BrokerFilter;
+import 
org.apache.pulsar.broker.loadbalance.extensions.strategy.BrokerSelectionStrategy;
+import org.apache.pulsar.common.naming.ServiceUnitId;
+import org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap;
+
+@Slf4j
+public class ExtensibleLoadManagerImpl implements ExtensibleLoadManager {
+
+private PulsarService pulsar;
+
+private ServiceConfiguration conf;
+
+@Getter
+private BrokerRegistry brokerRegistry;
+
+private ServiceUnitStateChannel serviceUnitStateChannel;
+
+@Getter
+private LoadManagerContext context;
+
+@Getter
+private final BrokerSelectionStrategy brokerSelectionStrategy;
+
+@Getter
+private List brokerFilterPipeline;
+
+private final AtomicBoolean started = new AtomicBoolean(false);
+
+private final ConcurrentOpenHashMap>>
+lookupRequests = ConcurrentOpenHashMap.>>newBuilder()
+.build();
+
+/**
+ * Life cycle: Constructor -> initialize -> start -> close.
+ */
+public ExtensibleLoadManagerImpl() {
+this.brokerFilterPipeline = new ArrayList<>();
+this.brokerSelectionStrategy = (brokers, bundle, context) -> {
+if (brokers.isEmpty()) {
+return Optional.empty();
+}
+return 
Optional.of(brokers.get(ThreadLocalRandom.current().nextInt(brokers.size(;
+};
+}
+
+@Override
+public void start() throws PulsarServerException {
+if (this.started.get()) {
+return;
+}
+this.brokerRegistry = new BrokerRegistryImpl(pulsar);
+this.serviceUnitStateChannel = new ServiceUnitStateChannelImpl(pulsar);
+this.brokerRegistry.start();
+this.serviceUnitStateChannel.start();
+
+// TODO: Start the load data store.
+
+this.context = LoadManagerContextImpl.builder()
+.configuration(conf)
+.brokerRegistry(brokerRegistry)
+.brokerLoadDataStore(null)
+.topBundleLoadDataStore(null).build();
+// TODO: Start load data reporter.
+
+// TODO: Start unload scheduler and bundle split scheduler
+
+this.started.set(true);
+}
+
+@Override
+public void initialize(PulsarService pulsar) {
+this.pulsar = pulsar;
+this.conf = pulsar.getConfiguration();
+}
+
+@Override
+public CompletableFuture> 
assign(Optional topic,
+ServiceUnitId 
serviceUnit) {
+
+final String bundle = serviceUnit.toString();
+
+CompletableFuture> future = 
lookupRequests.computeIfAbsent(bundle, k -> {
+final CompletableFuture> owner;
+// Assign the bundle to channel owner if is internal topic, to 
avoid circular references.
+if (topic.isPresent() && isInternalTopic(topic.get().toString())) {
+owner = 

[GitHub] [pulsar] Technoboy- closed pull request #19222: [fix][test] Fix flaky test CompactionReaderImplTest

[GitBox]

Technoboy- closed pull request #19222: [fix][test] Fix flaky test 
CompactionReaderImplTest
URL: https://github.com/apache/pulsar/pull/19222


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] Demogorgon314 commented on a diff in pull request #19102: [improve][broker] PIP-192: Implement extensible load manager

[GitBox]

Demogorgon314 commented on code in PR #19102:
URL: https://github.com/apache/pulsar/pull/19102#discussion_r1068832278


##
pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java:
##
@@ -55,6 +55,7 @@
 import org.apache.pulsar.broker.loadbalance.LeaderElectionService;
 import org.apache.pulsar.broker.loadbalance.LoadManager;
 import org.apache.pulsar.broker.loadbalance.ResourceUnit;
+import 
org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImpl;

Review Comment:
   I tried to split it into two PRs, but `ExtensibleLoadManagerImpl ` test 
dependency topic lookup because we are using `TableView` in 
`ServiceUnitStateChannel`, so I have to implement the load manager function.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] mattisonchao closed pull request #19223: [improve][broker] Copy subscription properties during updating the topic partition number.

[GitBox]

mattisonchao closed pull request #19223: [improve][broker] Copy subscription 
properties during updating the topic partition number.
URL: https://github.com/apache/pulsar/pull/19223


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] dlg99 commented on a diff in pull request #19153: [improve] PIP-241: add TopicEventListener / topic events for the BrokerService

[GitBox]

dlg99 commented on code in PR #19153:
URL: https://github.com/apache/pulsar/pull/19153#discussion_r1068812324


##
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/TopicEventsListener.java:
##
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.broker.service;
+
+/**
+ * Listener for the Topic events.
+ */
+public interface TopicEventsListener {
+
+/**
+ * Types of events currently supported.
+ *  create/load/unload/delete
+ */
+enum TopicEvent {
+// create events included into load events
+CREATE,
+LOAD,
+UNLOAD,
+DELETE,
+}
+
+/**
+ * Stages of events currently supported.
+ *  before starting the event/successful completion/failed completion
+ */
+enum EventStage {
+BEFORE,
+SUCCESS,
+FAILURE
+}
+
+/**
+ * Handle topic event.
+ * Choice of the thread / maintenance of the thread pool is up to the 
event handlers.
+ * @param topicName - name of the topic
+ * @param event - TopicEvent
+ * @param stage - EventStage
+ * @param t - exception in case of FAILURE, if present/known
+ */
+void handleEvent(String topicName, TopicEvent event, EventStage stage, 
Throwable t);

Review Comment:
   prototyped cancellation of event and after discussion with @eolivelli we 
decided it is not needed. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] dlg99 commented on a diff in pull request #19153: [improve] PIP-241: add TopicEventListener / topic events for the BrokerService

[GitBox]

dlg99 commented on code in PR #19153:
URL: https://github.com/apache/pulsar/pull/19153#discussion_r1068811806


##
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/TopicEventsListener.java:
##
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.broker.service;
+
+/**
+ * Listener for the Topic events.
+ */
+public interface TopicEventsListener {

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] mattisonchao opened a new pull request, #19223: [improve][broker] Copy subscription properties during updating the topic partition number.

[GitBox]

mattisonchao opened a new pull request, #19223:
URL: https://github.com/apache/pulsar/pull/19223

   ### Motivation
   
   we have supported creating a subscription with properties since #15503. It 
is a good way to support copy subscription properties during updating the topic 
partition number.
   
   Because #15503 didn't support v1, this PR also just focused on v2. 
   
   ### Modifications
   
   - Support copy subscription properties during updating the topic partition 
number.
   
   ### Verifying this change
   
   - [x] Make sure that the change passes the CI checks.
   
   ### Documentation
   
   
   
   - [ ] `doc` 
   - [ ] `doc-required` 
   - [x] `doc-not-needed` 
   - [ ] `doc-complete` 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] dlg99 opened a new issue, #19224: PIP-241: TopicEventListener / topic events for the BrokerService

[GitBox]

dlg99 opened a new issue, #19224:
URL: https://github.com/apache/pulsar/issues/19224

   ### Motivation
   
   Some Protocol Handlers may need to know about the topic-specific events to 
update internal caches and/or state.
   
   These mechanisms will be useful also for core Pulsar components (like the 
Transactions subsystem) and probably we would be able to simplify the 
interaction between the internal components in the broker by using an unified 
mechanism to handle the lifecycle of topics.
   
   Specific use cases:
   
   KOP keeps some state for the topic and needs to handle such cases as:
   
   - Topic Unloaded: release resources dedicated to the topic
   - Topic Loaded: trigger loading of components tied to the partition 
(GroupCoordinator, TransactionManager)
   - Topic Deleted: remove any persistent state associated to the topic that is 
stored in additional side system topics
   - Topic Created: the same as “deleted” (ensure that there is no state on 
system topics related to the new topic) 
   
   
   ### Goal
   
   This PIP defines a set of events needed for the protocol handlers (and for 
internal broker components) to get notifications about topic-specific events as 
seen by BrokerService. PIP outlines changes needed for protocol handlers to 
keep/cache state consistent with BrokerService’s.
   
   The changes should not affect Pulsar running without protocol handlers or 
with protocol handlers that do not rely on the new events.
   
   
   ### API Changes
   
   ```java
   /**
* Listener for the Topic events.
*/
   @InterfaceAudience.LimitedPrivate
   @InterfaceStability.Evolving
   public interface TopicEventsListener {
   
   /**
* Types of events currently supported.
*  create/load/unload/delete
*/
   enum TopicEvent {
   // create events included into load events
   CREATE,
   LOAD,
   UNLOAD,
   DELETE,
   }
   
   /**
* Stages of events currently supported.
*  before starting the event/successful completion/failed completion
*/
   enum EventStage {
   BEFORE,
   SUCCESS,
   FAILURE
   }
   
   /**
* Outcome of the listener.
* Ignored for events at final stages (SUCCESS/FAILURE),
*
*/
   enum EventProcessingOutcome {
   OK,
   FAILURE,
   NOT_ALLOWED
   }
   
   /**
* POJO for event processing result (outcome, message)
*/
   @ToString(includeFieldNames=true)
   @Data(staticConstructor="of")
   class EventProcessingResult {
   private final EventProcessingOutcome outcome;
   private final String message;
   }
   
   /**
* Handle topic event.
* Choice of the thread / maintenance of the thread pool is up to the 
event handlers.
* @param topicName - name of the topic
* @param event - TopicEvent
* @param stage - EventStage
* @param t - exception in case of FAILURE, if present/known
* @return - EventProcessingResult.
*  EventProcessingResult.EventProcessingOutcome != OK indicates 
request to cancel
*   event at BEFORE stage.
*/
   EventProcessingResult handleEvent(String topicName, TopicEvent event, 
EventStage stage, Throwable t);
   }
   ```
   
   BrokerService:
   ```java
   public void addTopicEventListener(TopicEventsListener... listeners)
   
   public void removeTopicEventListener(TopicEventsListener... listeners) 
   ```
   
   ### Implementation
   
   See PR for the proposed implementation.
   https://github.com/apache/pulsar/pull/19153 
   
   
   ### Alternatives
   
   Add new methods to the BrokerInterceptor API
   
   ### Anything else?
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] codecov-commenter commented on pull request #19221: [fix][test] Fix ProxyPrometheusMetricsTest flakiness and consistently close JAX-RS clients in tests

[GitBox]

codecov-commenter commented on PR #19221:
URL: https://github.com/apache/pulsar/pull/19221#issuecomment-1381129556

   # 
[Codecov](https://codecov.io/gh/apache/pulsar/pull/19221?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 Report
   > Merging 
[#19221](https://codecov.io/gh/apache/pulsar/pull/19221?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 (714d7fc) into 
[master](https://codecov.io/gh/apache/pulsar/commit/d7f8f56b5be5ad34cf1d33dd89b7bfc6a9ab94ff?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 (d7f8f56) will **increase** coverage by `22.92%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/pulsar/pull/19221/graphs/tree.svg?width=650=150=pr=acYqCpsK9J_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pulsar/pull/19221?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
   
   ```diff
   @@  Coverage Diff  @@
   ## master   #19221   +/-   ##
   =
   + Coverage 47.89%   70.81%   +22.92% 
   + Complexity10871  439-10432 
   =
 Files   713   26  -687 
 Lines 69730 2251-67479 
 Branches   7496  245 -7251 
   =
   - Hits  33394 1594-31800 
   + Misses32631  484-32147 
   + Partials   3705  173 -3532 
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `70.81% <ø> (+22.92%)` | :arrow_up: |
   
   Flags with carried forward coverage won't be shown. [Click 
here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment)
 to find out more.
   
   | [Impacted 
Files](https://codecov.io/gh/apache/pulsar/pull/19221?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 | Coverage Δ | |
   |---|---|---|
   | 
[...va/org/apache/pulsar/client/impl/ProducerImpl.java](https://codecov.io/gh/apache/pulsar/pull/19221?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1Byb2R1Y2VySW1wbC5qYXZh)
 | | |
   | 
[...r/client/impl/schema/reader/JacksonJsonReader.java](https://codecov.io/gh/apache/pulsar/pull/19221?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL3NjaGVtYS9yZWFkZXIvSmFja3Nvbkpzb25SZWFkZXIuamF2YQ==)
 | | |
   | 
[...e/pulsar/client/impl/schema/reader/JsonReader.java](https://codecov.io/gh/apache/pulsar/pull/19221?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL3NjaGVtYS9yZWFkZXIvSnNvblJlYWRlci5qYXZh)
 | | |
   | 
[...tion/buffer/impl/TransactionBufferHandlerImpl.java](https://codecov.io/gh/apache/pulsar/pull/19221?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci90cmFuc2FjdGlvbi9idWZmZXIvaW1wbC9UcmFuc2FjdGlvbkJ1ZmZlckhhbmRsZXJJbXBsLmphdmE=)
 | | |
   | 
[.../apache/pulsar/broker/namespace/LookupOptions.java](https://codecov.io/gh/apache/pulsar/pull/19221?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9uYW1lc3BhY2UvTG9va3VwT3B0aW9ucy5qYXZh)
 | | |
   | 
[...r/service/schema/DefaultSchemaRegistryService.java](https://codecov.io/gh/apache/pulsar/pull/19221?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL3NjaGVtYS9EZWZhdWx0U2NoZW1hUmVnaXN0cnlTZXJ2aWNlLmphdmE=)
 | | |
   | 
[...apache/pulsar/broker/admin/v1/SchemasResource.java](https://codecov.io/gh/apache/pulsar/pull/19221?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9hZG1pbi92MS9TY2hlbWFzUmVzb3VyY2UuamF2YQ==)
 | | |
   | 

[GitHub] [pulsar] heesung-sn commented on issue #19214: Flaky-test: CompactionReaderImplTest.test

[GitBox]

heesung-sn commented on issue #19214:
URL: https://github.com/apache/pulsar/issues/19214#issuecomment-1381121566

   Raised a PR: https://github.com/apache/pulsar/pull/19222


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] heesung-sn opened a new pull request, #19222: [fix][test] Fix flaky test CompactionReaderImplTest

[GitBox]

heesung-sn opened a new pull request, #19222:
URL: https://github.com/apache/pulsar/pull/19222

   
   
   
   
   Fixes https://github.com/apache/pulsar/issues/19214
   
   
   
   ### Motivation
   
   Fixes https://github.com/apache/pulsar/issues/19214
   
   ### Modifications
   
   Created a consumer before publishing messages.
   
   ### Verifying this change
   I couldn't replicate the issue in my local env. However, we need to create a 
consumer first to run this test consistently. (This pre-step in other tests is 
searched by `// subscribe before ` comment.)
   
   - [x] Make sure that the change passes the CI checks.
   
   ### Does this pull request potentially affect one of the following parts:
   
   
   
   *If the box was checked, please highlight the changes*
   
   - [ ] Dependencies (add or upgrade a dependency)
   - [ ] The public API
   - [ ] The schema
   - [ ] The default values of configurations
   - [ ] The threading model
   - [ ] The binary protocol
   - [ ] The REST endpoints
   - [ ] The admin CLI options
   - [ ] The metrics
   - [ ] Anything that affects deployment
   
   ### Documentation
   
   
   
   - [ ] `doc` 
   - [ ] `doc-required` 
   - [x] `doc-not-needed` 
   - [ ] `doc-complete` 
   
   ### Matching PR in forked repository
   
   PR in forked repository: https://github.com/heesung-sn/pulsar/pull/21
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari opened a new pull request, #19221: [fix][test] Fix ProxyPrometheusMetricsTest flakiness and consistently close JAX-RS clients in tests

[GitBox]

lhotari opened a new pull request, #19221:
URL: https://github.com/apache/pulsar/pull/19221

   Fixes #19216
   
   ### Motivation
   
   See https://github.com/apache/pulsar/pull/19217#issuecomment-1381096161 for 
details of the remaining problem in ProxyPrometheusMetricsTest
   
   ### Modifications
   
   - Use Awaitility to wait until metrics are available
   - Consistently close JAX-RS clients in all tests
   
   ### Documentation
   
   
   
   - [ ] `doc` 
   - [ ] `doc-required` 
   - [x] `doc-not-needed` 
   - [ ] `doc-complete` 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari commented on issue #19216: Flaky-test: ProxyPrometheusMetricsTest.testMetrics

[GitBox]

lhotari commented on issue #19216:
URL: https://github.com/apache/pulsar/issues/19216#issuecomment-1381098258

   Reopening the issue. After applying #19217, the actual problem shows up.
   ```
 Error:  Tests run: 2, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 
15.921 s <<< FAILURE! - in 
org.apache.pulsar.proxy.server.ProxyPrometheusMetricsTest
 Error:  
testMetrics(org.apache.pulsar.proxy.server.ProxyPrometheusMetricsTest)  Time 
elapsed: 0.079 s  <<< FAILURE!
 java.lang.AssertionError: expected [1] but found [0]
at org.testng.Assert.fail(Assert.java:110)
at org.testng.Assert.failNotEquals(Assert.java:1413)
at org.testng.Assert.assertEqualsImpl(Assert.java:149)
at org.testng.Assert.assertEquals(Assert.java:131)
at org.testng.Assert.assertEquals(Assert.java:1240)
at org.testng.Assert.assertEquals(Assert.java:1274)
at 
org.apache.pulsar.proxy.server.ProxyPrometheusMetricsTest.testMetrics(ProxyPrometheusMetricsTest.java:118)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at 
org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
at 
org.testng.internal.invokers.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:47)
at 
org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:76)
at 
org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #19217: [improve][test] Prevent "Failed to register Collector of type Counter"

[GitBox]

lhotari commented on PR #19217:
URL: https://github.com/apache/pulsar/pull/19217#issuecomment-1381096161

   Now the actual flakiness is coming to surface
   
   ```
 Error:  Tests run: 2, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 
15.921 s <<< FAILURE! - in 
org.apache.pulsar.proxy.server.ProxyPrometheusMetricsTest
 Error:  
testMetrics(org.apache.pulsar.proxy.server.ProxyPrometheusMetricsTest)  Time 
elapsed: 0.079 s  <<< FAILURE!
 java.lang.AssertionError: expected [1] but found [0]
at org.testng.Assert.fail(Assert.java:110)
at org.testng.Assert.failNotEquals(Assert.java:1413)
at org.testng.Assert.assertEqualsImpl(Assert.java:149)
at org.testng.Assert.assertEquals(Assert.java:131)
at org.testng.Assert.assertEquals(Assert.java:1240)
at org.testng.Assert.assertEquals(Assert.java:1274)
at 
org.apache.pulsar.proxy.server.ProxyPrometheusMetricsTest.testMetrics(ProxyPrometheusMetricsTest.java:118)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at 
org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
at 
org.testng.internal.invokers.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:47)
at 
org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:76)
at 
org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] codecov-commenter commented on pull request #19220: [improve][test] Run cleanup methods even when failfast mode is enabled

[GitBox]

codecov-commenter commented on PR #19220:
URL: https://github.com/apache/pulsar/pull/19220#issuecomment-1381095343

   # 
[Codecov](https://codecov.io/gh/apache/pulsar/pull/19220?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 Report
   > Merging 
[#19220](https://codecov.io/gh/apache/pulsar/pull/19220?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 (cc9f150) into 
[master](https://codecov.io/gh/apache/pulsar/commit/d7f8f56b5be5ad34cf1d33dd89b7bfc6a9ab94ff?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 (d7f8f56) will **decrease** coverage by `10.56%`.
   > The diff coverage is `0.00%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/pulsar/pull/19220/graphs/tree.svg?width=650=150=pr=acYqCpsK9J_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pulsar/pull/19220?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
   
   ```diff
   @@  Coverage Diff  @@
   ## master   #19220   +/-   ##
   =
   - Coverage 47.89%   37.32%   -10.57% 
   + Complexity10871 1994 -8877 
   =
 Files   713  209  -504 
 Lines 6973014453-55277 
 Branches   7496 1576 -5920 
   =
   - Hits  33394 5395-27999 
   + Misses32631 8478-24153 
   + Partials   3705  580 -3125 
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `37.32% <0.00%> (-10.57%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click 
here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment)
 to find out more.
   
   | [Impacted 
Files](https://codecov.io/gh/apache/pulsar/pull/19220?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 | Coverage Δ | |
   |---|---|---|
   | 
[...va/org/apache/pulsar/client/impl/ProducerImpl.java](https://codecov.io/gh/apache/pulsar/pull/19220?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1Byb2R1Y2VySW1wbC5qYXZh)
 | `15.62% <0.00%> (-0.04%)` | :arrow_down: |
   | 
[.../org/apache/bookkeeper/mledger/impl/EntryImpl.java](https://codecov.io/gh/apache/pulsar/pull/19220?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-bWFuYWdlZC1sZWRnZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2Jvb2trZWVwZXIvbWxlZGdlci9pbXBsL0VudHJ5SW1wbC5qYXZh)
 | | |
   | 
[...oker/stats/prometheus/AggregatedProducerStats.java](https://codecov.io/gh/apache/pulsar/pull/19220?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zdGF0cy9wcm9tZXRoZXVzL0FnZ3JlZ2F0ZWRQcm9kdWNlclN0YXRzLmphdmE=)
 | | |
   | 
[...lsar/broker/service/RedeliveryTrackerDisabled.java](https://codecov.io/gh/apache/pulsar/pull/19220?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL1JlZGVsaXZlcnlUcmFja2VyRGlzYWJsZWQuamF2YQ==)
 | | |
   | 
[...r/service/schema/AvroSchemaCompatibilityCheck.java](https://codecov.io/gh/apache/pulsar/pull/19220?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL3NjaGVtYS9BdnJvU2NoZW1hQ29tcGF0aWJpbGl0eUNoZWNrLmphdmE=)
 | | |
   | 
[...tion/buffer/impl/InMemTransactionBufferReader.java](https://codecov.io/gh/apache/pulsar/pull/19220?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci90cmFuc2FjdGlvbi9idWZmZXIvaW1wbC9Jbk1lbVRyYW5zYWN0aW9uQnVmZmVyUmVhZGVyLmphdmE=)
 | | |
   | 

[pulsar] branch master updated: [improve][client] Use Jackson's ObjectReader in Client's JSON readers (#19215)

[lhotari]

This is an automated email from the ASF dual-hosted git repository.

lhotari pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new a56bd876f5a [improve][client] Use Jackson's ObjectReader in Client's 
JSON readers (#19215)
a56bd876f5a is described below

commit a56bd876f5acf328c7512715279212c650e52d43
Author: Lari Hotari 
AuthorDate: Fri Jan 13 01:01:25 2023 +0200

[improve][client] Use Jackson's ObjectReader in Client's JSON readers 
(#19215)
---
 .../pulsar/client/impl/schema/reader/JacksonJsonReader.java  | 9 +
 .../org/apache/pulsar/client/impl/schema/reader/JsonReader.java  | 9 +
 2 files changed, 10 insertions(+), 8 deletions(-)

diff --git 
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/reader/JacksonJsonReader.java
 
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/reader/JacksonJsonReader.java
index 478d40a291d..4220457dcef 100644
--- 
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/reader/JacksonJsonReader.java
+++ 
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/reader/JacksonJsonReader.java
@@ -19,6 +19,7 @@
 package org.apache.pulsar.client.impl.schema.reader;
 
 import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.ObjectReader;
 import java.io.IOException;
 import java.io.InputStream;
 import org.apache.pulsar.client.api.SchemaSerializationException;
@@ -33,17 +34,17 @@ import org.slf4j.LoggerFactory;
  */
 public class JacksonJsonReader implements SchemaReader {
 private final Class pojo;
-private final ObjectMapper objectMapper;
+private final ObjectReader objectReader;
 
 public JacksonJsonReader(ObjectMapper objectMapper, Class pojo) {
 this.pojo = pojo;
-this.objectMapper = objectMapper;
+this.objectReader = pojo != null ? objectMapper.readerFor(pojo) : 
objectMapper.reader();
 }
 
 @Override
 public T read(byte[] bytes, int offset, int length) {
 try {
-return objectMapper.readValue(bytes, offset, length, this.pojo);
+return objectReader.readValue(bytes, offset, length);
 } catch (IOException e) {
 throw new SchemaSerializationException(e);
 }
@@ -52,7 +53,7 @@ public class JacksonJsonReader implements SchemaReader {
 @Override
 public T read(InputStream inputStream) {
 try {
-return objectMapper.readValue(inputStream, pojo);
+return objectReader.readValue(inputStream, pojo);
 } catch (IOException e) {
 throw new SchemaSerializationException(e);
 } finally {
diff --git 
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/reader/JsonReader.java
 
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/reader/JsonReader.java
index 0b86cbcc84b..5e146698e8f 100644
--- 
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/reader/JsonReader.java
+++ 
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/reader/JsonReader.java
@@ -19,6 +19,7 @@
 package org.apache.pulsar.client.impl.schema.reader;
 
 import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.ObjectReader;
 import java.io.IOException;
 import java.io.InputStream;
 import org.apache.pulsar.client.api.SchemaSerializationException;
@@ -35,17 +36,17 @@ import org.slf4j.LoggerFactory;
 @Deprecated
 public class JsonReader implements SchemaReader {
 private final Class pojo;
-private final ObjectMapper objectMapper;
+private final ObjectReader objectReader;
 
 public JsonReader(ObjectMapper objectMapper, Class pojo) {
 this.pojo = pojo;
-this.objectMapper = objectMapper;
+this.objectReader = pojo != null ? objectMapper.readerFor(pojo) : 
objectMapper.reader();
 }
 
 @Override
 public T read(byte[] bytes, int offset, int length) {
 try {
-return objectMapper.readValue(bytes, offset, length, this.pojo);
+return objectReader.readValue(bytes, offset, length);
 } catch (IOException e) {
 throw new SchemaSerializationException(e);
 }
@@ -54,7 +55,7 @@ public class JsonReader implements SchemaReader {
 @Override
 public T read(InputStream inputStream) {
 try {
-return objectMapper.readValue(inputStream, pojo);
+return objectReader.readValue(inputStream);
 } catch (IOException e) {
 throw new SchemaSerializationException(e);
 } finally {

[GitHub] [pulsar] lhotari merged pull request #19215: [improve][client] Use Jackson's ObjectReader in Client's JSON readers

[GitBox]

lhotari merged PR #19215:
URL: https://github.com/apache/pulsar/pull/19215


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari commented on issue #19214: Flaky-test: CompactionReaderImplTest.test

[GitBox]

lhotari commented on issue #19214:
URL: https://github.com/apache/pulsar/issues/19214#issuecomment-1381085265

   thank you @heesung-sn .
   
   I faced yet another one: 
https://github.com/apache/pulsar/actions/runs/3906374418/jobs/6674698961#step:11:540


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #19217: [improve][test] Prevent "Failed to register Collector of type Counter"

[GitBox]

lhotari commented on PR #19217:
URL: https://github.com/apache/pulsar/pull/19217#issuecomment-1381084031

   There's a flaky assertion in the test case and after retry, the problem 
changed to the "Failed to register Collector of type Counter: 
test_counter_total is already in use by another Collector of type Counter" 
failure. 
   This issue was visible from the Unit-PROXY-surefire-reports.zip file and the 
`pulsar-proxy/target/surefire-reports/TEST-org.apache.pulsar.proxy.server.ProxyPrometheusMetricsTest.xml`
 xml file inside the zip file. The actual problem didn't get recorded in the 
console at all. 
   
   ```xml
   
   
   
   
   
   
   
   
   
   
   java.lang.IllegalArgumentException: 
Failed to register Collector of type Counter: test_counter_total is already in 
use by another Collector of type Counter at 
io.prometheus.client.CollectorRegistry.register(CollectorRegistry.java:57) at 
io.prometheus.client.Collector.register(Collector.java:307) at 
io.prometheus.client.Collector.register(Collector.java:300) at 
org.apache.pulsar.proxy.server.ProxyPrometheusMetricsTest.testMetrics(ProxyPrometheusMetricsTest.java:99)
 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method) at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
 at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.base/java.lang.reflect.Method.invoke(Method.jav
 a:568) at 
org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
 at 
org.testng.internal.invokers.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:47)
 at 
org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:76)
 at 
org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
 at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
 at java.base/java.lang.Thread.run(Thread.java:833) 
   
   
   
   
   
   
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari opened a new pull request, #19220: [improve][test] Run cleanup methods even when failfast mode is enabled

[GitBox]

lhotari opened a new pull request, #19220:
URL: https://github.com/apache/pulsar/pull/19220

   ### Motivation
   
   - cleanup didn't happen for tests after failures when failfast mode is 
activated
   
   ### Modifications
   
   - ignore the failfast solution for @After* methods 
   
   ### Documentation
   
   
   
   - [ ] `doc` 
   - [ ] `doc-required` 
   - [x] `doc-not-needed` 
   - [ ] `doc-complete` 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari closed issue #19216: Flaky-test: ProxyPrometheusMetricsTest.testMetrics

[GitBox]

lhotari closed issue #19216: Flaky-test: ProxyPrometheusMetricsTest.testMetrics
URL: https://github.com/apache/pulsar/issues/19216


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[pulsar] branch master updated: [fix][ci] Fix PulsarFunctionLocalRunTest that broke after ClusterData validation changes (#19212)

[lhotari]

This is an automated email from the ASF dual-hosted git repository.

lhotari pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 7f75993a84f [fix][ci] Fix PulsarFunctionLocalRunTest that broke after 
ClusterData validation changes (#19212)
7f75993a84f is described below

commit 7f75993a84f068febb92f63b338fb0614543affc
Author: Lari Hotari 
AuthorDate: Fri Jan 13 00:45:45 2023 +0200

[fix][ci] Fix PulsarFunctionLocalRunTest that broke after ClusterData 
validation changes (#19212)
---
 .../worker/PulsarFunctionLocalRunTest.java | 30 +-
 1 file changed, 23 insertions(+), 7 deletions(-)

diff --git 
a/pulsar-broker/src/test/java/org/apache/pulsar/functions/worker/PulsarFunctionLocalRunTest.java
 
b/pulsar-broker/src/test/java/org/apache/pulsar/functions/worker/PulsarFunctionLocalRunTest.java
index a8a1c8384dd..6555cc841e9 100644
--- 
a/pulsar-broker/src/test/java/org/apache/pulsar/functions/worker/PulsarFunctionLocalRunTest.java
+++ 
b/pulsar-broker/src/test/java/org/apache/pulsar/functions/worker/PulsarFunctionLocalRunTest.java
@@ -272,7 +272,7 @@ public class PulsarFunctionLocalRunTest {
 primaryHost = pulsar.getWebServiceAddress();
 
 // create cluster metadata
-ClusterData clusterData = 
ClusterData.builder().serviceUrl(urlTls.toString()).build();
+ClusterData clusterData = 
ClusterData.builder().serviceUrlTls(urlTls.toString()).build();
 admin.clusters().createCluster(config.getClusterName(), clusterData);
 
 ClientBuilder clientBuilder = PulsarClient.builder()
@@ -308,14 +308,30 @@ public class PulsarFunctionLocalRunTest {
 void shutdown() throws Exception {
 try {
 log.info("--- Shutting down ---");
-fileServer.stop();
-pulsarClient.close();
-admin.close();
-pulsar.close();
-bkEnsemble.stop();
+if (fileServer != null) {
+fileServer.stop();
+fileServer = null;
+}
+if (pulsarClient != null) {
+pulsarClient.close();
+pulsarClient = null;
+}
+if (admin != null) {
+admin.close();
+admin = null;
+}
+if (pulsar != null) {
+pulsar.close();
+pulsar = null;
+}
+if (bkEnsemble != null) {
+bkEnsemble.stop();
+bkEnsemble = null;
+}
 } finally {
 if (tempDirectory != null) {
 tempDirectory.delete();
+tempDirectory = null;
 }
 }
 }
@@ -1099,7 +1115,7 @@ public class PulsarFunctionLocalRunTest {
 public void testPulsarSinkWithFunction() throws Throwable {
 testPulsarSinkLocalRun(null, 1, StatsNullSink.class.getName(), 
"builtin://exclamation", 
"org.apache.pulsar.functions.api.examples.RecordFunction");
 }
-
+
 public static class TestErrorSink implements Sink {
 private Map config;
 @Override

[pulsar] branch master updated (7f75993a84f -> 3c584918146)

[lhotari]

This is an automated email from the ASF dual-hosted git repository.

lhotari pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


from 7f75993a84f [fix][ci] Fix PulsarFunctionLocalRunTest that broke after 
ClusterData validation changes (#19212)
 add 3c584918146 [improve][test] Prevent "Failed to register Collector of 
type Counter" (#19217)

No new revisions were added by this update.

Summary of changes:
 .../proxy/server/ProxyPrometheusMetricsTest.java   | 62 +-
 1 file changed, 36 insertions(+), 26 deletions(-)

[GitHub] [pulsar] lhotari merged pull request #19217: [improve][test] Prevent "Failed to register Collector of type Counter"

[GitBox]

lhotari merged PR #19217:
URL: https://github.com/apache/pulsar/pull/19217


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari merged pull request #19212: [fix][ci] Fix PulsarFunctionLocalRunTest that broke after ClusterData validation changes

[GitBox]

lhotari merged PR #19212:
URL: https://github.com/apache/pulsar/pull/19212


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] codecov-commenter commented on pull request #19219: [improve][ci] Limit cache step to 5 minutes to fail faster when it gets stuck

[GitBox]

codecov-commenter commented on PR #19219:
URL: https://github.com/apache/pulsar/pull/19219#issuecomment-1381059159

   # 
[Codecov](https://codecov.io/gh/apache/pulsar/pull/19219?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 Report
   > Merging 
[#19219](https://codecov.io/gh/apache/pulsar/pull/19219?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 (d8f256d) into 
[master](https://codecov.io/gh/apache/pulsar/commit/d7f8f56b5be5ad34cf1d33dd89b7bfc6a9ab94ff?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 (d7f8f56) will **decrease** coverage by `10.55%`.
   > The diff coverage is `0.00%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/pulsar/pull/19219/graphs/tree.svg?width=650=150=pr=acYqCpsK9J_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pulsar/pull/19219?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
   
   ```diff
   @@  Coverage Diff  @@
   ## master   #19219   +/-   ##
   =
   - Coverage 47.89%   37.33%   -10.56% 
   + Complexity10871 1994 -8877 
   =
 Files   713  209  -504 
 Lines 6973014453-55277 
 Branches   7496 1576 -5920 
   =
   - Hits  33394 5396-27998 
   + Misses32631 8476-24155 
   + Partials   3705  581 -3124 
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `37.33% <0.00%> (-10.56%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click 
here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment)
 to find out more.
   
   | [Impacted 
Files](https://codecov.io/gh/apache/pulsar/pull/19219?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 | Coverage Δ | |
   |---|---|---|
   | 
[...va/org/apache/pulsar/client/impl/ProducerImpl.java](https://codecov.io/gh/apache/pulsar/pull/19219?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1Byb2R1Y2VySW1wbC5qYXZh)
 | `15.62% <0.00%> (-0.04%)` | :arrow_down: |
   | 
[...apache/pulsar/client/impl/AutoClusterFailover.java](https://codecov.io/gh/apache/pulsar/pull/19219?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL0F1dG9DbHVzdGVyRmFpbG92ZXIuamF2YQ==)
 | `75.55% <0.00%> (-0.56%)` | :arrow_down: |
   | 
[...er/mledger/intercept/ManagedLedgerInterceptor.java](https://codecov.io/gh/apache/pulsar/pull/19219?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-bWFuYWdlZC1sZWRnZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2Jvb2trZWVwZXIvbWxlZGdlci9pbnRlcmNlcHQvTWFuYWdlZExlZGdlckludGVyY2VwdG9yLmphdmE=)
 | | |
   | 
[...n/coordinator/TransactionCoordinatorException.java](https://codecov.io/gh/apache/pulsar/pull/19219?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci90cmFuc2FjdGlvbi9leGNlcHRpb24vY29vcmRpbmF0b3IvVHJhbnNhY3Rpb25Db29yZGluYXRvckV4Y2VwdGlvbi5qYXZh)
 | | |
   | 
[...sar/broker/loadbalance/impl/LoadManagerShared.java](https://codecov.io/gh/apache/pulsar/pull/19219?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9sb2FkYmFsYW5jZS9pbXBsL0xvYWRNYW5hZ2VyU2hhcmVkLmphdmE=)
 | | |
   | 
[.../java/org/apache/pulsar/broker/admin/v3/Sinks.java](https://codecov.io/gh/apache/pulsar/pull/19219?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9hZG1pbi92My9TaW5rcy5qYXZh)
 | | |
   | 

[GitHub] [pulsar] lhotari commented on pull request #19219: [improve][ci] Limit cache step to 5 minutes to fail faster when it gets stuck

[GitBox]

lhotari commented on PR #19219:
URL: https://github.com/apache/pulsar/pull/19219#issuecomment-1381045171

   another case where this issue is happening: 
https://github.com/apache/pulsar/actions/runs/3905806139/jobs/6674164947#step:6:1875


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari opened a new pull request, #19219: [improve][ci] Limit cache step to 5 minutes to fail faster when it gets stuck

[GitBox]

lhotari opened a new pull request, #19219:
URL: https://github.com/apache/pulsar/pull/19219

   ### Motivation
   
   The cache step in GitHub Actions workflows gets sometimes stuck.
   
   Example at 
https://github.com/apache/pulsar/actions/runs/3905696188/jobs/6673646325#step:6:17
   ```
   Received 1493610815 of 1497805119 (99.7%), 0.5 MBs/sec
   Received 1493610815 of 1497805119 (99.7%), 0.5 MBs/sec
   Received 1493610815 of 1497805119 (99.7%), 0.5 MBs/sec
   Received 1493610815 of 1497805119 (99.7%), 0.5 MBs/sec
   Received 1493610815 of 1497805119 (99.7%), 0.5 MBs/sec
   Received 1493610815 of 1497805119 (99.7%), 0.5 MBs/sec
   Received 1493610815 of 1497805119 (99.7%), 0.5 MBs/sec
   Received 1493610815 of 1497805119 (99.7%), 0.5 MBs/sec
   Error: The operation was canceled.
   ```
   The cache step runs until the job times out after 60 minutes. 
   
   ### Modifications
   
   Add `timeout-minutes: 5` to cache steps to limit the execution time to 5 
minutes.
   
   ### Documentation
   
   
   
   - [ ] `doc` 
   - [ ] `doc-required` 
   - [x] `doc-not-needed` 
   - [ ] `doc-complete` 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #19215: [improve][client] Use Jackson's ObjectReader in Client's JSON readers

[GitBox]

lhotari commented on PR #19215:
URL: https://github.com/apache/pulsar/pull/19215#issuecomment-1380998956

   > LGTM. @lhotari - do you have documentation indicating this is the 
preferred approach? Thanks.
   
   Do you mean the use of `ObjectReader` instead of `ObjectMapper`?  Or the use 
of `readerFor`?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari commented on a diff in pull request #19160: [improve][misc] Use shared Jackson ObjectMapper to reduce overhead and remove ThreadLocal solution

[GitBox]

lhotari commented on code in PR #19160:
URL: https://github.com/apache/pulsar/pull/19160#discussion_r1068647149


##
pulsar-broker-common/src/main/java/org/apache/pulsar/broker/resources/LocalPoliciesResources.java:
##
@@ -66,7 +66,7 @@ public 
CompletableFuture>> getLocalPolici
 public CompletableFuture setLocalPoliciesWithVersion(NamespaceName 
ns, LocalPolicies policies,
Optional 
version) {
 try {
-byte[] content = 
ObjectMapperFactory.getThreadLocal().writeValueAsBytes(policies);
+byte[] content = 
ObjectMapperFactory.getMapper().writer().writeValueAsBytes(policies);

Review Comment:
   I was thinking of that, but the challenge is that there's also the Yaml 
versions and the ObjectMapper configured to filter null fields. That would 
result in a lot of similar methods. It's a tradeoff. 
   Leaving ObjectMapperFactory as-is and creating a completely new class would 
be another option. I'd suggest that it's left as another PR to continue 
refactoring. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] heesung-sn commented on issue #19214: Flaky-test: CompactionReaderImplTest.test

[GitBox]

heesung-sn commented on issue #19214:
URL: https://github.com/apache/pulsar/issues/19214#issuecomment-1380982436

   Ack. will work on this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari commented on issue #19214: Flaky-test: CompactionReaderImplTest.test

[GitBox]

lhotari commented on issue #19214:
URL: https://github.com/apache/pulsar/issues/19214#issuecomment-1380980081

   @heesung-sn This test seems to be flaky. Do you have a chance to take a look 
at it? it was added by #18195


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] codecov-commenter commented on pull request #19215: [improve][client] Use Jackson's ObjectReader in Client's JSON readers

[GitBox]

codecov-commenter commented on PR #19215:
URL: https://github.com/apache/pulsar/pull/19215#issuecomment-1380979114

   # 
[Codecov](https://codecov.io/gh/apache/pulsar/pull/19215?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 Report
   > Merging 
[#19215](https://codecov.io/gh/apache/pulsar/pull/19215?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 (9cc0894) into 
[master](https://codecov.io/gh/apache/pulsar/commit/d7f8f56b5be5ad34cf1d33dd89b7bfc6a9ab94ff?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 (d7f8f56) will **decrease** coverage by `36.22%`.
   > The diff coverage is `15.38%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/pulsar/pull/19215/graphs/tree.svg?width=650=150=pr=acYqCpsK9J_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pulsar/pull/19215?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
   
   ```diff
   @@  Coverage Diff  @@
   ## master   #19215   +/-   ##
   =
   - Coverage 47.89%   11.66%   -36.23% 
   + Complexity10871 2432 -8439 
   =
 Files   713  633   -80 
 Lines 6973059928 -9802 
 Branches   7496 6251 -1245 
   =
   - Hits  33394 6988-26406 
   - Misses3263152186+19555 
   + Partials   3705  754 -2951 
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `11.66% <15.38%> (-36.23%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click 
here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment)
 to find out more.
   
   | [Impacted 
Files](https://codecov.io/gh/apache/pulsar/pull/19215?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 | Coverage Δ | |
   |---|---|---|
   | 
[...va/org/apache/pulsar/client/impl/ProducerImpl.java](https://codecov.io/gh/apache/pulsar/pull/19215?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1Byb2R1Y2VySW1wbC5qYXZh)
 | `15.62% <0.00%> (-0.04%)` | :arrow_down: |
   | 
[...e/pulsar/client/impl/schema/reader/JsonReader.java](https://codecov.io/gh/apache/pulsar/pull/19215?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL3NjaGVtYS9yZWFkZXIvSnNvblJlYWRlci5qYXZh)
 | `0.00% <0.00%> (ø)` | |
   | 
[...r/client/impl/schema/reader/JacksonJsonReader.java](https://codecov.io/gh/apache/pulsar/pull/19215?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL3NjaGVtYS9yZWFkZXIvSmFja3Nvbkpzb25SZWFkZXIuamF2YQ==)
 | `66.66% <66.66%> (-6.67%)` | :arrow_down: |
   | 
[...n/java/org/apache/pulsar/client/api/RawReader.java](https://codecov.io/gh/apache/pulsar/pull/19215?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9hcGkvUmF3UmVhZGVyLmphdmE=)
 | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | 
[...ava/org/apache/pulsar/broker/admin/v1/Brokers.java](https://codecov.io/gh/apache/pulsar/pull/19215?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9hZG1pbi92MS9Ccm9rZXJzLmphdmE=)
 | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | 
[...ava/org/apache/pulsar/broker/admin/v2/Brokers.java](https://codecov.io/gh/apache/pulsar/pull/19215?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9hZG1pbi92Mi9Ccm9rZXJzLmphdmE=)
 | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | 

[GitHub] [pulsar] codecov-commenter commented on pull request #19217: [improve][test] Prevent "Failed to register Collector of type Counter"

[GitBox]

codecov-commenter commented on PR #19217:
URL: https://github.com/apache/pulsar/pull/19217#issuecomment-1380979022

   # 
[Codecov](https://codecov.io/gh/apache/pulsar/pull/19217?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 Report
   > Merging 
[#19217](https://codecov.io/gh/apache/pulsar/pull/19217?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 (8bce3ab) into 
[master](https://codecov.io/gh/apache/pulsar/commit/d7f8f56b5be5ad34cf1d33dd89b7bfc6a9ab94ff?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 (d7f8f56) will **decrease** coverage by `10.54%`.
   > The diff coverage is `0.00%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/pulsar/pull/19217/graphs/tree.svg?width=650=150=pr=acYqCpsK9J_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pulsar/pull/19217?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
   
   ```diff
   @@  Coverage Diff  @@
   ## master   #19217   +/-   ##
   =
   - Coverage 47.89%   37.34%   -10.55% 
   + Complexity10871 1995 -8876 
   =
 Files   713  209  -504 
 Lines 6973014453-55277 
 Branches   7496 1576 -5920 
   =
   - Hits  33394 5397-27997 
   + Misses32631 8476-24155 
   + Partials   3705  580 -3125 
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `37.34% <0.00%> (-10.55%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click 
here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment)
 to find out more.
   
   | [Impacted 
Files](https://codecov.io/gh/apache/pulsar/pull/19217?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 | Coverage Δ | |
   |---|---|---|
   | 
[...va/org/apache/pulsar/client/impl/ProducerImpl.java](https://codecov.io/gh/apache/pulsar/pull/19217?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1Byb2R1Y2VySW1wbC5qYXZh)
 | `15.62% <0.00%> (-0.04%)` | :arrow_down: |
   | 
[...ulsar/broker/service/PulsarChannelInitializer.java](https://codecov.io/gh/apache/pulsar/pull/19217?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL1B1bHNhckNoYW5uZWxJbml0aWFsaXplci5qYXZh)
 | | |
   | 
[...er/systopic/NamespaceEventsSystemTopicFactory.java](https://codecov.io/gh/apache/pulsar/pull/19217?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zeXN0b3BpYy9OYW1lc3BhY2VFdmVudHNTeXN0ZW1Ub3BpY0ZhY3RvcnkuamF2YQ==)
 | | |
   | 
[...ar/broker/loadbalance/extensions/models/Split.java](https://codecov.io/gh/apache/pulsar/pull/19217?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9sb2FkYmFsYW5jZS9leHRlbnNpb25zL21vZGVscy9TcGxpdC5qYXZh)
 | | |
   | 
[...ker/loadbalance/impl/LinuxBrokerHostUsageImpl.java](https://codecov.io/gh/apache/pulsar/pull/19217?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9sb2FkYmFsYW5jZS9pbXBsL0xpbnV4QnJva2VySG9zdFVzYWdlSW1wbC5qYXZh)
 | | |
   | 
[...e/pulsar/common/naming/NamespaceBundleFactory.java](https://codecov.io/gh/apache/pulsar/pull/19217?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NvbW1vbi9uYW1pbmcvTmFtZXNwYWNlQnVuZGxlRmFjdG9yeS5qYXZh)
 | | |
   | 

[GitHub] [pulsar] lhotari commented on issue #19214: Flaky-test: CompactionReaderImplTest.test

[GitBox]

lhotari commented on issue #19214:
URL: https://github.com/apache/pulsar/issues/19214#issuecomment-1380978378

   another one:
   
https://github.com/apache/pulsar/actions/runs/3905696188/jobs/6673192180#step:11:539


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] pgier opened a new issue, #19218: pulsar-shell CTRL-C should be interpreted as cancel the current command and should not exit the shell

[GitBox]

pgier opened a new issue, #19218:
URL: https://github.com/apache/pulsar/issues/19218

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) 
and found nothing similar.
   
   
   ### Motivation
   
   When running pulsar-shell, CTRL-C should be used to cancel the current 
pulsar command instead of a full exit of the program.  This would make the 
behavior more similar to bash and the python shell.
   
   For example, let's say I mistype a command.
   
   > admin topics list pblic/default
   
   Then I want to cancel the current pulsar command, but still stay in the 
pulsar-shell.
   
   ### Solution
   
   Catch the CTRL-C and just cancel the current pulsar command instead of 
exiting completely.
   
   ### Alternatives
   
   _No response_
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari opened a new pull request, #19217: [improve][test] Prevent "Failed to register Collector of type Counter"

[GitBox]

lhotari opened a new pull request, #19217:
URL: https://github.com/apache/pulsar/pull/19217

   Fixes #19216
   
   ### Motivation
   
   ProxyPrometheusMetricsTest is flaky. Please see #19216 for details.
   
   ### Modifications
   
   - unregister collector after the test
   - fix retries by adding null checks
   
   ### Documentation
   
   
   
   - [ ] `doc` 
   - [ ] `doc-required` 
   - [x] `doc-not-needed` 
   - [ ] `doc-complete` 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari opened a new issue, #19216: Flaky-test: ProxyPrometheusMetricsTest.testMetrics

[GitBox]

lhotari opened a new issue, #19216:
URL: https://github.com/apache/pulsar/issues/19216

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) 
and found nothing similar.
   
   
   ### Example failure
   
   
https://github.com/apache/pulsar/actions/runs/3905486374/jobs/6672708495#step:11:441
   
   ### Exception stacktrace
   
   ```
 Error:  Tests run: 6, Failures: 1, Errors: 0, Skipped: 5, Time elapsed: 
14.542 s <<< FAILURE! - in 
org.apache.pulsar.proxy.server.ProxyPrometheusMetricsTest
 Error:  
testMetrics(org.apache.pulsar.proxy.server.ProxyPrometheusMetricsTest)  Time 
elapsed: 0.031 s  <<< FAILURE!
 java.lang.IllegalArgumentException: Failed to register Collector of type 
Counter: test_counter_total is already in use by another Collector of type 
Counter
at 
io.prometheus.client.CollectorRegistry.register(CollectorRegistry.java:57)
at io.prometheus.client.Collector.register(Collector.java:307)
at io.prometheus.client.Collector.register(Collector.java:300)
at 
org.apache.pulsar.proxy.server.ProxyPrometheusMetricsTest.testMetrics(ProxyPrometheusMetricsTest.java:99)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at 
org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
at 
org.testng.internal.invokers.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:47)
at 
org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:76)
at 
org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
   ```
   
   ### Are you willing to submit a PR?
   
   - [X] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari commented on a diff in pull request #19215: [improve][client] Use Jackson's ObjectReader in Client's JSON readers

[GitBox]

lhotari commented on code in PR #19215:
URL: https://github.com/apache/pulsar/pull/19215#discussion_r1068606954


##
pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/reader/JacksonJsonReader.java:
##
@@ -33,17 +34,17 @@
  */
 public class JacksonJsonReader implements SchemaReader {
 private final Class pojo;
-private final ObjectMapper objectMapper;
+private final ObjectReader objectReader;
 
 public JacksonJsonReader(ObjectMapper objectMapper, Class pojo) {
 this.pojo = pojo;
-this.objectMapper = objectMapper;
+this.objectReader = pojo != null ? objectMapper.readerFor(pojo) : 
objectMapper.reader();

Review Comment:
   null handling is done this way because of backwards compatibility. The 
previous solution accepted null as pojo and a test breaks if this logic doesn't 
exist for the ObjectReader.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari opened a new pull request, #19215: [improve][client] Use Jackson's ObjectReader in Client's JSON readers

[GitBox]

lhotari opened a new pull request, #19215:
URL: https://github.com/apache/pulsar/pull/19215

   ### Motivation
   
   - benefits: 
 - no need to lookup the type when reading happens. 
- The type gets fetched from cache once. 
- Since a reference is kept, it won't overflow from a busy Jackson 
TypeFactory LRU cache.
   
   ### Modifications
   
   Replace ObjectMapper usage with ObjectReader in JacksonJsonReader and 
JsonReader classes.
   
   ### Documentation
   
   
   
   - [ ] `doc` 
   - [ ] `doc-required` 
   - [x] `doc-not-needed` 
   - [ ] `doc-complete` 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari opened a new issue, #19214: Flaky-test: CompactionReaderImplTest.test

[GitBox]

lhotari opened a new issue, #19214:
URL: https://github.com/apache/pulsar/issues/19214

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) 
and found nothing similar.
   
   
   ### Example failure
   
   
https://github.com/apache/pulsar/actions/runs/3904728193/jobs/6671101717#step:11:540
   
   ### Exception stacktrace
   
   ```
 Error:  Tests run: 6, Failures: 1, Errors: 0, Skipped: 5, Time elapsed: 
69.264 s <<< FAILURE! - in 
org.apache.pulsar.client.impl.CompactionReaderImplTest
 Error:  test(org.apache.pulsar.client.impl.CompactionReaderImplTest)  Time 
elapsed: 30.238 s  <<< FAILURE!
 java.util.concurrent.ExecutionException: 
org.apache.pulsar.client.api.PulsarClientException$TimeoutException: The 
subscription __compaction of the topic 
persistent://my-property/my-ns/my-compact-topic could not get the last message 
id withing configured timeout
at 
java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
at 
java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
at 
org.apache.pulsar.client.impl.CompactionReaderImplTest.test(CompactionReaderImplTest.java:97)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at 
org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
at 
org.testng.internal.invokers.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:47)
at 
org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:76)
at 
org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
 Caused by: 
org.apache.pulsar.client.api.PulsarClientException$TimeoutException: The 
subscription __compaction of the topic 
persistent://my-property/my-ns/my-compact-topic could not get the last message 
id withing configured timeout
at 
org.apache.pulsar.client.impl.ConsumerImpl.internalGetLastMessageIdAsync(ConsumerImpl.java:2406)
at 
org.apache.pulsar.client.impl.ConsumerImpl.lambda$internalGetLastMessageIdAsync$58(ConsumerImpl.java:2415)
at 
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at 
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at 
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 more
   ```
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] pgier opened a new issue, #19213: pulsar-shell should quit cleanly on EOF character (CTRL-D)

[GitBox]

pgier opened a new issue, #19213:
URL: https://github.com/apache/pulsar/issues/19213

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) 
and found nothing similar.
   
   
   ### Motivation
   
   It's common for command line shells to interpret the EOF (CTRL-D) character 
as a quit command.  For example `bash` and `python`.   The current behavior of 
pulsar-shell is to print an exception and then quit.
   ```
   Exception in thread "main" org.jline.reader.EndOfFileException
at 
org.jline.reader.impl.LineReaderImpl.readLine(LineReaderImpl.java:657)
at 
org.jline.reader.impl.LineReaderImpl.readLine(LineReaderImpl.java:468)
at org.apache.pulsar.shell.PulsarShell$1.readLine(PulsarShell.java:280)
at 
org.apache.pulsar.shell.PulsarShell.lambda$run$4(PulsarShell.java:398)
at org.apache.pulsar.shell.PulsarShell.run(PulsarShell.java:410)
at org.apache.pulsar.shell.PulsarShell.run(PulsarShell.java:221)
at org.apache.pulsar.shell.PulsarShell.main(PulsarShell.java:210)
   Goodbye!
   ```
   Since pulsar-shell is already doing the correct thing by quitting, this 
should just be done cleanly without the exception.
   
   
   ### Solution
   
   Catch the EOF character and quit with no error.
   
   ### Alternatives
   
   _No response_
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] codecov-commenter commented on pull request #19212: [fix][ci] Fix PulsarFunctionLocalRunTest that broke after ClusterData validation changes

[GitBox]

codecov-commenter commented on PR #19212:
URL: https://github.com/apache/pulsar/pull/19212#issuecomment-1380930939

   # 
[Codecov](https://codecov.io/gh/apache/pulsar/pull/19212?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 Report
   > Merging 
[#19212](https://codecov.io/gh/apache/pulsar/pull/19212?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 (9935f35) into 
[master](https://codecov.io/gh/apache/pulsar/commit/d7f8f56b5be5ad34cf1d33dd89b7bfc6a9ab94ff?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 (d7f8f56) will **decrease** coverage by `10.56%`.
   > The diff coverage is `0.00%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/pulsar/pull/19212/graphs/tree.svg?width=650=150=pr=acYqCpsK9J_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pulsar/pull/19212?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
   
   ```diff
   @@  Coverage Diff  @@
   ## master   #19212   +/-   ##
   =
   - Coverage 47.89%   37.32%   -10.57% 
   + Complexity10871 1993 -8878 
   =
 Files   713  209  -504 
 Lines 6973014453-55277 
 Branches   7496 1576 -5920 
   =
   - Hits  33394 5395-27999 
   + Misses32631 8478-24153 
   + Partials   3705  580 -3125 
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `37.32% <0.00%> (-10.57%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click 
here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment)
 to find out more.
   
   | [Impacted 
Files](https://codecov.io/gh/apache/pulsar/pull/19212?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
 | Coverage Δ | |
   |---|---|---|
   | 
[...va/org/apache/pulsar/client/impl/ProducerImpl.java](https://codecov.io/gh/apache/pulsar/pull/19212?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1Byb2R1Y2VySW1wbC5qYXZh)
 | `15.62% <0.00%> (-0.04%)` | :arrow_down: |
   | 
[...r/offload/jcloud/BlockAwareSegmentInputStream.java](https://codecov.io/gh/apache/pulsar/pull/19212?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-dGllcmVkLXN0b3JhZ2UvamNsb3VkL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9ib29ra2VlcGVyL21sZWRnZXIvb2ZmbG9hZC9qY2xvdWQvQmxvY2tBd2FyZVNlZ21lbnRJbnB1dFN0cmVhbS5qYXZh)
 | | |
   | 
[...rg/apache/pulsar/broker/lookup/v2/TopicLookup.java](https://codecov.io/gh/apache/pulsar/pull/19212?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9sb29rdXAvdjIvVG9waWNMb29rdXAuamF2YQ==)
 | | |
   | 
[...rvice/TransactionBufferSnapshotServiceFactory.java](https://codecov.io/gh/apache/pulsar/pull/19212?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL1RyYW5zYWN0aW9uQnVmZmVyU25hcHNob3RTZXJ2aWNlRmFjdG9yeS5qYXZh)
 | | |
   | 
[...oker/stats/prometheus/AggregatedConsumerStats.java](https://codecov.io/gh/apache/pulsar/pull/19212?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zdGF0cy9wcm9tZXRoZXVzL0FnZ3JlZ2F0ZWRDb25zdW1lclN0YXRzLmphdmE=)
 | | |
   | 
[...lsar/broker/loadbalance/impl/DeviationShedder.java](https://codecov.io/gh/apache/pulsar/pull/19212?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9sb2FkYmFsYW5jZS9pbXBsL0RldmlhdGlvblNoZWRkZXIuamF2YQ==)
 | | |
   | 

[GitHub] [pulsar] michaeljmarshall commented on a diff in pull request #19151: [improve][broker] Add parameter check for create/update cluster.

[GitBox]

michaeljmarshall commented on code in PR #19151:
URL: https://github.com/apache/pulsar/pull/19151#discussion_r1068579592


##
pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/ClusterDataImpl.java:
##
@@ -398,4 +400,30 @@ public ClusterDataImpl build() {
 migratedClusterUrl);
 }
 }
+
+/**
+ * Check cluster data properties by rule, if some property is illegal, it 
will throw
+ * {@link IllegalArgumentException}.
+ *
+ * @throws IllegalArgumentException exist illegal property.
+ */
+public void checkPropertiesIfPresent() throws IllegalArgumentException {
+URIPreconditions.checkURIIfPresent(getServiceUrl(),
+uri -> Objects.equals(uri.getScheme(), "http"),
+"Illegal service url, example: 
http://pulsar.example.com:8080;);
+URIPreconditions.checkURIIfPresent(getServiceUrlTls(),
+uri -> Objects.equals(uri.getScheme(), "https"),
+"Illegal service tls url, example: 
https://pulsar.example.com:8443;);
+URIPreconditions.checkURIIfPresent(getBrokerServiceUrl(),
+uri -> Objects.equals(uri.getScheme(), "pulsar"),
+"Illegal broker service url, example: 
pulsar://pulsar.example.com:6650");
+URIPreconditions.checkURIIfPresent(getBrokerServiceUrlTls(),
+uri -> Objects.equals(uri.getScheme(), "pulsar+ssl"),
+"Illegal broker service tls url, example: 
pulsar+ssl://pulsar.example.com:6651");
+URIPreconditions.checkURIIfPresent(getProxyServiceUrl(),
+uri -> Objects.equals(uri.getScheme(), "pulsar")
+|| Objects.equals(uri.getScheme(), "pulsar+ssl"),
+"Illegal proxy service url, example: 
pulsar+ssl://ats-proxy.example.com:4443 "
++ "or pulsar://ats-proxy.example.com:4080");
+}

Review Comment:
   Do we document these requirements? Also, what would happen if a user had 
already created a cluster in a bad state?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #19151: [improve][broker] Add parameter check for create/update cluster.

[GitBox]

lhotari commented on PR #19151:
URL: https://github.com/apache/pulsar/pull/19151#issuecomment-1380926203

   @mattisonchao  I wonder if a "https" url would need to be accepted in the 
serviceUrl just because of backwards compatibility reasons?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #19151: [improve][broker] Add parameter check for create/update cluster.

[GitBox]

lhotari commented on PR #19151:
URL: https://github.com/apache/pulsar/pull/19151#issuecomment-1380925462

   @mattisonchao Is this PR a potential breaking change? Please check #19212 . 
   ("Pulsar CI Flaky" jobs have been failing for several days with OOME after 
this PR was merged: 
https://github.com/apache/pulsar/actions/workflows/pulsar-ci-flaky.yaml . The 
OOME was a side-effect of the test class that broke because of the validation 
change.)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar] michaeljmarshall commented on a diff in pull request #19191: [fix][client] Fix producer could send timeout when enable batching

[GitBox]

michaeljmarshall commented on code in PR #19191:
URL: https://github.com/apache/pulsar/pull/19191#discussion_r1068553187


##
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java:
##
@@ -873,14 +869,23 @@ private boolean canAddToCurrentBatch(MessageImpl msg) {
 && batchMessageContainer.hasSameTxn(msg);
 }
 
+private void triggerSendIfFullOrScheduleFlush(boolean isBatchFull) {
+if (isBatchFull) {
+batchMessageAndSend(false);
+} else {
+maybeScheduleBatchFlushTask();
+}
+}
+
 private void doBatchSendAndAdd(MessageImpl msg, SendCallback callback, 
ByteBuf payload) {
 if (log.isDebugEnabled()) {
 log.debug("[{}] [{}] Closing out batch to accommodate large 
message with size {}", topic, producerName,
 msg.getUncompressedSize());
 }
 try {
 batchMessageAndSend(false);
-batchMessageContainer.add(msg, callback);
+boolean isBatchFull = batchMessageContainer.add(msg, callback);
+triggerSendIfFullOrScheduleFlush(isBatchFull);
 lastSendFuture = callback.getFuture();

Review Comment:
   Should this be before the call to `triggerSendIfFullOrScheduleFlush`? It is 
in the other usage of `triggerSendIfFullOrScheduleFlush`. However, we're in a 
`synchronized (this)` block, so it might be fine.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org