Re: [VOTE] PIP-327 Support force topic loading for unrecoverable errors

Let me try again to ping on this thread as it's already been more than 4 months for discussion and 2 weeks for the VOTE thread for such a small and necessary change. Thanks Rajan On Thu, Apr 25, 2024 at 12:12 AM 太上玄元道君 wrote: > +1 nonbinding > > > Rajan Dhabalia 于2024年4月1

Re: [VOTE] PIP-327 Support force topic loading for unrecoverable errors

Hi, Can we please close this VOTE thread as it's been more than 8 days to this thread and 4 months since we created a PIP. Please let me know if we have any concern and can we close the VOTE so we can move forward with this PIP. Thanks, Rajan On Tue, Apr 16, 2024 at 7:09 PM Rajan Dhabalia

Re: [VOTE] PIP-327 Support force topic loading for unrecoverable errors

f7zg7cdkns1om0q > > > Thanks, > sinan > > > Apurva Telang 于2024年4月17日 周三02:43写道： > > > +1 (non-binding) > > > > On Tue, Apr 16, 2024 at 10:11 AM Enrico Olivelli > > wrote: > > > > > +1 (binding) > > > > > >

Re: [DISCUSS] Solution after missing schema

d. > > > There is also content in PIP-327 that decides to skip processing when the > schema is lost. > > > Thanks, > sinan > > > Rajan Dhabalia 于2024年4月17日 周三02:34写道： > > > Hi, > > > > Broken schema ledgers and topic unavailability are the issues we h

Re: [DISCUSS] Solution after missing schema

Hi, Broken schema ledgers and topic unavailability are the issues we have been trying to resolve for multiple years now and We have been trying to add lot of patches to fix but we still find different usecases which can make topic unavailable when schema ledger is not recoverable (ledger is

[VOTE] PIP-327 Support force topic loading for unrecoverable errors

Hi, I would like to start voting thread for PIP-327 which also addresses issue # https://github.com/apache/pulsar/issues/21751 PIP design PR: https://github.com/apache/pulsar/pull/21752 Thread:

Re: [DISCUSS] PIP-335: Oxia metadata support

+1 Thanks, Rajan On Wed, Jan 31, 2024 at 3:58 PM Matteo Merli wrote: > https://github.com/apache/pulsar/pull/22009 > > === > > # PIP-335: Supporty Oxia metadata store plugin > > # Motivation > > Oxia is a scalable metadata store and coordination system that can be used > as the core

Re: [E] Re: [DISCUSS] PIP-188(BlueGreenClusterMigration) Remove redirectedClusterURI in Pulsar Client

;> requirement. I was under the impression that the migration happens for all >>> topics from the old to the new cluster. >>> >>> For this requirement, I think we should keep redirectedClusterURI in >>> HandlerState. >>> >>> Thanks for this quick r

Re: [DISCUSS] PIP-188(BlueGreenClusterMigration) Remove redirectedClusterURI in Pulsar Client

This might not work because same PulsarClient can be used to create multiple topics and producer/consumer and migration state can be different per topic and even for produce/consumer (of same topic) so, changing service url will not work for blue-green migration redirection for all topics

Re: [DISCUSS] PIP-327 Support force topic loading for unrecoverable errors

ad issues. The topic loading works fine, and the > > producer works fine. > > But the proposal said it would resolve the topic load issue, can you > > reproduce the topic load issue? > > > > Regards, > > Penghui > > > > > > > > On Wed, Dec

[DISCUSS] PIP-327 Support force topic loading for unrecoverable errors

Hi, We have an issue to fail loading topics in unrecoverable situation and impacting topic availability:: https://github.com/apache/pulsar/issues/21751 This PIP addresses the issue and allows brokers to handle such situations and maintain the topic availability: PIP:

[VOTE] PIP-313 Support force unsubscribe using consumer api

Hi, I would like to start voting thread for PIP-313 which also addresses issue # https://github.com/apache/pulsar/issues/21451 PIP design PR: https://github.com/apache/pulsar/pull/21452 Thread: https://lists.apache.org/thread/hptx8z9mktn94gvqtt4547wzcfcgdsrv Thanks, Rajan

Re: Re: [Discuss] PIP-319 Unblock stuck Key_Shared subscription after consumer reconnect

o you have plans for Key_Shared dispatcher rework? I would be happy to > contribute to such a project if I can help somehow. > > > Best Regards, > > Nikolai > > > > On 2023/12/01 06:34:59 Rajan Dhabalia wrote: > > Hi Nikolai, > > > > Thanks for bring

Re: [Discuss] PIP-319 Unblock stuck Key_Shared subscription after consumer reconnect

Hi Nikolai, Thanks for bringing this to the attention. I have seen multiple PIP and bug fixes for the Key-Shared subscription and I feel Key_Shared subscription dispatcher is developed with multiple fundamental issues which required to introduce scheduled-task to unblock stuck reads, introducing

Re: [DISCUSS] PIP-321 Split the responsibilities of namespace replication-clusters

Hi, I don't agree with your assumptions about allowed clusters and replication-cluster definition. Allowed clusters defined at tenant level will restrict tenants to create namespaces in regions/clusters where they are not allowed. and replication clusters will help tenant to set up full mesh

Re: [DISCUSS] PIP-310: Support custom publish rate limiters

Hi Lari/Girish, I am sorry for jumping late in the discussion but I would like to acknowledge the requirement of pluggable publish rate-limiter and I had also asked it during implementation of publish rate limiter as well. There are trade-offs between different rate-limiter implementations based

Re: [Discuss] Disable `Rebase and Merge` on Pulsar Repo

I remember, It was always disabled since we open sourced Pulsar. Who enabled it and what was the reason for enabling that option? It's definitely not a good option and we should disable it asap. Thanks, Rajan On Sun, Nov 5, 2023 at 7:36 PM Heesung Sohn wrote: > Hi, > > I recently merged my PR

Re: [DISCUSS] PIP-313 Support force unsubscribe using consumer api

nnection. > > Regards, > Penghui > > On Fri, Oct 27, 2023 at 8:25 AM Rajan Dhabalia > wrote: > > > Hi, > > > > We have an issue created regarding unsubscription of shared subscription > in > > Pulsar spout at : https://github.com/apache/pulsar/issue

[DISCUSS] PIP-313 Support force unsubscribe using consumer api

Hi, We have an issue created regarding unsubscription of shared subscription in Pulsar spout at : https://github.com/apache/pulsar/issues/21451 This PIP addresses the issue and allows users to unsubscribe subscription with additional force option. PIP:

Re: [VOTE] PIP-271: Add broker health check status into prometheus metrics #20147

Thanks for adding it. +1 Thanks, Rajan On Thu, Oct 26, 2023 at 3:52 PM vineeth p wrote: > Broker metrics don't have anything to indicate the health of the broker (to > indicate if the broker is active). In Prometheus broker metrics which are > used for monitoring, it will be useful if metrics

Re: [DISCUSS] PIP-296: Support storing broker internal client certificates in metadata store

ination cluster, are there any other recommendations you might have in > mind? > > Best, > Mattison > On 8 Sep 2023 at 16:47 +0800, Rajan Dhabalia , > wrote: > > Hi, > > > > Pulsar stores different types of metadata into a metadata store which > > conta

Re: [DISCUSS] PIP-296: Support storing broker internal client certificates in metadata store

Hi, Pulsar stores different types of metadata into a metadata store which contains tenant, namespaces and topic metadata. Metadata-store should store metadata definition and should avoid combining other non metadata related information especially certificates or keys like encryption/decryption

Re: [VOTE] PIP-268: Add support of topic stats/stats-internal using

the concerns and questions as it's been going for a long time and I would like to see it moving forward and would like to see any suggestions or feedback for API or any changes in the proposal. Thanks, Rajan On Wed, Aug 2, 2023 at 7:06 PM Rajan Dhabalia wrote: > >> If so, I think it could be a go

Re: [VOTE] PIP-268: Add support of topic stats/stats-internal using

sers want to > remove it. > > I want to say that from the perspective of improving performance, > it may not be more convincing than the above reason. > > Thanks, > Penghui > > > On Mon, Jun 26, 2023 at 3:37 PM Rajan Dhabalia > wrote: > > > > I do not deny

Re: [VOTE] PIP-268: Add support of topic stats/stats-internal using

jo-schema, and > accessing stats with consistent pojo definition. > > This is from the perspective of developers who are already very familiar > with Pulsar. > If you don't know it, just want to create a client based on Pulsar's API. > How to do it? For public API, I think a cl

Re: [VOTE] PIP-268: Add support of topic stats/stats-internal using

n > the REST API. > Well JSON would be the better option to avoid data conversion and copy for complex stats/internal-stats complex data-structures at broker and client side, maintaining consistency between stats/internal-stats pojo-schema, and accessing stats with consistent pojo definition

Re: [DISCUSS] PIP-267: Support multi-topic messageId deserialization to ack messages

Jun 20, 2023 at 9:39 AM Rajan Dhabalia > wrote: > > > > So you say in that sentence that you will add the topic name into > > MessageIdData. MessageIdData is defined in PulsarApi.proto and is > > transferred over the wire, so how can you add the topic to this class

[VOTE] PIP-268: Add support of topic stats/stats-internal using

Hi, I would like to start VOTE for : https://github.com/apache/pulsar/issues/20265 Thanks, Rajan

[VOTE] PIP-267: Support multi-topic messageId deserialization to ack messages

Hi. Pulsar api provides MessageId interface which is generally used by producer and consumer applications to manage topic offset. Sometimes, these applications would like to serialize and deserialize messageIds, specifically consumer app which would like to persist messageId and ack with those

Re: [DISCUSS] PIP-267: Support multi-topic messageId deserialization to ack messages

rred over the > wire, so how can you add the topic to this class without changing the wire > protocol? > > > > > > On Fri, Jun 16, 2023 at 10:47 PM Rajan Dhabalia > wrote: > > > Yes, the topic name will not be transferred and it's not part of the wire >

Re: [DISCUSS] PIP-267: Support multi-topic messageId deserialization to ack messages

ture > the > > client transmit to the server, so that means you are transmitting topic > to > > the server? > > > > > > On Fri, May 12, 2023 at 7:45 AM Rajan Dhabalia > > wrote: > > > > > Thank you for sharing your knowledge about the PIP which

Re: [DISCUSS] PIP-264: Enhanced OTel-based metric system

Hi, Are we proposing a change to break existing metrics compatibility (prometheus)? If that is the case then it's a big red flag as it will be a pain for any company to upgrade Pulsar as monitoring is THE most important part of the system and we don't even want to break compatibility for any

Re: [DISCUSS] PIP-273: Enable hostname verification by default

Hi, I am not sure enabling hostname verification by default is a good idea because it will cause deployment failures for many users. Hostname verification requires x509 cert changes and wildcard SNName in some cases and it's not easy when broker hostname patterns don't match in x509 certs and

Re: [DISCUSS] PIP-275: Introduce numWorkerThreadsForPersistentTopic to deprecate numWorkerThreadsForNonPersistentTopic in configuration

Hi, We already have a default number of threads for persistent topics but we have added a feature non-persistent topics and to isolate that path we introduced a number of worker threads which we can reduce or tune based on resources we would like to allocate for non-persistent topics. So, it

Re: [DISCUSS] PIP-271: Add broker health check status into prometheus metrics #20147

Thank you Vineeth for creating the PIP. This PIP will be useful to capture a broker's health as a part of metrics or dashboards and we can also set up various alerts on it. Generally we should check the status API of the broker for the HC/liveness probe in k8 instead of sanity because that can

Re: [DISCUSS] PIP-268: Add support of topic stats/stats-internal using client api

t; considered > to add to binary protocol? I mean, we'd better have a strategy to avoid > moving many > APIs to binary protocol. For example: > > - Tenant/Namespace level REST APIs, no > - Topic policies APIs, no > - Topic's internal info, maybe > - Expire messages, no > - T

Re: [DISCUSS] PIP-268: Add support of topic stats/stats-internal using client api

based binary TCP, if we end up writing > a JSON string, the only difference I see is the blocking I/O of writing the > response. > > WDYT? > > > On Fri, May 12, 2023 at 7:29 AM Rajan Dhabalia > wrote: > > > Communicating over binary protocol is more scal

Re: [DISCUSS] PIP-267: Support multi-topic messageId deserialization to ack messages

arate API. I'm concerned > > about whether existing applications would be affected, no matter if > > the existing implementation has the limitation. If yes, we should > > document it in the PIP so that users can know that. > > > > > it's a new optional field which wou

Re: [DISCUSS] PIP-268: Add support of topic stats/stats-internal using client api

accessibility in pulsar. Thanks, Rajan On Thu, May 11, 2023 at 6:24 AM Asaf Mesika wrote: > Before I dive into the PIP, I have several questions on the background > provided below: > > > On Tue, May 9, 2023 at 9:08 AM Rajan Dhabalia > wrote: > > > Hi, > > > > Right

Re: [DISCUSS] PIP-267: Support multi-topic messageId deserialization to ack messages

. it's better to add the API changes and potential breaking changes > in the proposal. > > Thanks, > Yunze > > On Tue, May 9, 2023 at 1:59 PM Rajan Dhabalia > wrote: > > > > Hi, > > > > Pulsar api provides MessageId interface which is generally u

[DISCUSS] PIP-268: Add support of topic stats/stats-internal using client api

Hi, Right now, Pulsar provides the topic's stats and stats-internal over HTTP admin API, and this stats data is used by user applications and also by Pulsar internal components such as Pulsar-functions to derive the certain states of the applications. for example, there are use cases where the

[DISCUSS] PIP-267: Support multi-topic messageId deserialization to ack messages

Hi, Pulsar api provides MessageId interface which is generally used by producer and consumer applications to manage topic offset. Sometimes, these applications would like to serialize and deserialize messageIds, specifically consumer app which would like to persist messageId and ack with those

Re: [DISCUSS] Is PIP required for small changes in metrics

Hi, Thank you Mattison for starting this thread because I am feeling some of the community members are making new contributors' life difficult and trying to enforce rules which were never discussed and discourage them from contributing to Pulsar. I really don't want to see the Pulsar community

Re: [DISCUSSION] Redesign the MessageId interface

Hi, I was reading the email thread why we want to change MessageId interface: https://lists.apache.org/thread/rdkqnkohbmkjjs61hvoqplhhngr0b0sd >> Currently we have the following 5 implementations of MessageId: >> These implementations are such a mess. For example, when users get a MessageId from

Re: [DISCUSS] Planning for Apache Pulsar 3.0

ve any criteria on > what kinds of changes would necessitate cutting a new major release but > would still be considered acceptable by the community? > > -- > Devin Bost > Sent from mobile > Cell: 801-400-4602 > > On Sat, Oct 8, 2022, 2:14 PM Rajan Dhabalia wrote: &g

Re: [DISCUSS] Planning for Apache Pulsar 3.0

ing it down. If you accumulate > > many "big" changes it introduces a lot of risk for instabilities and > > incompatibilities. Breaking it down in multiple steps helps to see the > > incremental changes and introduced them in a phased manner. > > > > > > -- &g

Re: [DISCUSS] Planning for Apache Pulsar 3.0

Hi, Can we get the list of changes at one place which we are planning to get as part of 3.0. One thing I would like to see as a part of a major release, it CAN NOT impact existing usecases and users in any way which can force them to upgrade the client library. Applications using < 3.0 version

Re: [DISCUSS] Enable message deduplication for replicator by default

Message deduplication always comes with memory and CPU cost and making it default means charging this penalty to every user without having this requirement. Enabling by default means you are impacting every user who is not aware about this feature after upgrading the release. This is purely

Re: [DISCUSS] Move PIPs to the codebase?

Hi, >>> I think we can move all the PIPs to the codebase and the new proposal and proposal without any reviews should happen with a PR first. So that we can review and comment easily. I didn't understand this part. You want one to create a PR before submitting a proposal? That's clearly not a

Re: [DISCUSS] PIP-192 New Pulsar Broker Load Balancer

Hi, I have gone through PIP but I don't see some basic information as part of PIP: 1. current issue in current load balancer strategy 2. Are there any performance and feature gaps in the current load balancer? Please provide data or metrics to show the impact. 3. what exactly we are solving in

Re: [DISCUSS] PIP-184: Cluster migration or Blue-Green cluster deployment support in Pulsar

s" or "migrationStarted"? > >> It's just a state to mark that broker has considered it for migration similar to the Terminated state. we don't need extra state for completion because the broker is going to delete the topic once all subscribers reach the end of the topic. > > O

Re: [DISCUSS] PIP-184: Cluster migration or Blue-Green cluster deployment support in Pulsar

t; > > On Tue, Jul 12, 2022 at 6:23 PM Rajan Dhabalia > wrote: > > > Hi, > > > > We have created PIP-184 which helps users to perform cluster migration > with > > Apache Pulsar. Cluster migration or Blue-Green cluster deployment is one > of > > the pr

Re: [DISCUSS] PIP-184: Cluster migration or Blue-Green cluster deployment support in Pulsar

; >> Sure, I have updated PIP in the last section where it lists down the features this PIP supports. I haven't seen any limitation yet with this PIP but we can add if we see any concerns with the PIP. > -joe > > > On Wed, Jul 13, 2022 at 9:31 AM Rajan Dhabalia > wrote: >

Re: [DISCUSS] PIP-184: Cluster migration or Blue-Green cluster deployment support in Pulsar

an now be deleted? > > > > "Broker sends topic migration message to client so, producer/consumer at > > client side can handle redirection accordingly" > > > > For producers, the message will be sent the moment the status of the > topic > > has changed

Re: [DISCUSS] PIP-184: Cluster migration or Blue-Green cluster deployment support in Pulsar

ection accordingly" > > For producers, the message will be sent the moment the status of the topic > has changed, so all messages from there on will be written to the new > cluster? > Yes. > For consumers, the message will be sent when there are no more messages to > read? >

[DISCUSS] PIP-184: Cluster migration or Blue-Green cluster deployment support in Pulsar

Hi, We have created PIP-184 which helps users to perform cluster migration with Apache Pulsar. Cluster migration or Blue-Green cluster deployment is one of the proven solutions to migrate live traffic from one cluster to another. One of the examples is applications running on Kubernetes sometimes

Re: [VOTE] PIP-136: Sync Pulsar policies across multiple clouds

; partitioned metadata > to the remote cluster's metadata store? > > Thanks, > Penghui > > On Wed, Mar 23, 2022 at 3:39 AM Rajan Dhabalia > wrote: > > > >> Do we need to provide the ability for users to decide to replicate the > > ACLs and replication cluster or n

Re: Build Pulsar Server on Java 17- too strict ?

Hi, I do not agree to force client applications to use jdk-17 and that will not be good Pulsar as a project because that will force users to find another alternative of Pulsar for their messaging usecases. In large org where Pulsar is being used as a managed service and used by a large number of

Re: Discuss about creating partitioned topic in V1

> Having said that, V1 admin API was in effect considered as "soft deprecated" since Pulsar 2.0. All documentation has been focused on v2 APIs. There are Orgs which have large scale Pulsar clusters and use V1 apis. There is no way we can deprecate V1 api and it has to be maintained until we find

Re: [VOTE] PIP-136: Sync Pulsar policies across multiple clouds

ion-in-apache-pulsar-part-2-patterns-and-practices.html > [2] > > https://github.com/apache/pulsar/blob/4dcb166e0bfcce7fc85fd8d59a25b881f6f9c6fa/pulsar-common/src/main/java/org/apache/pulsar/common/events/PulsarEvent.java#L36 > [3] > > https://github.com/apache/pulsar/wiki/PIP-92

Re: [VOTE] PIP-136: Sync Pulsar policies across multiple clouds

remote > cluster? > > Currently, we already have a PulsarEvent struct to define the pulsar system > events, > Looks like we can use a unified event definition by PulsarEvent. > > Others look good to me. > > Regards, > Penghui > > > > On Sat, Mar 19, 2022 at

[VOTE] PIP-136: Sync Pulsar policies across multiple clouds

Hi, I would like to start VOTE on PIP-136: https://github.com/apache/pulsar/issues/13728 Thanks, Rajan On Tue, Feb 8, 2022 at 4:58 PM Rajan Dhabalia wrote: > > >> How do we designate the host broker? Is it manual? How does it work > when the host broker is removed f

Re: [DISCUSS] PIP-136: Sync Pulsar policies across multiple clouds

> changes and publish the changes to the event topic. > > > > How do we designate the host broker? Is it manual? How does it work > > when the host broker is removed from the cluster? > > > > If we collocate the active consumer with the broker hosting the e

Re: [DISCUSS] PIP-136: Sync Pulsar policies across multiple clouds

resses this concern as well. > Thanks, > Michael > > [0] https://lists.apache.org/thread/pj4n4wzm3do8nkc52l7g7obh0sktzm17 > [1] https://lists.apache.org/thread/h4cbvwjdomktsq2jo66x5qpvhdrqk871 > [2] https://lists.apache.org/thread/0xkg0gpsobp0dbgb6tp9xq097lpm65bx > > > > On Sun

[DISCUSS] PIP-136: Sync Pulsar policies across multiple clouds

Hi, I would like to start a discussion about PIP-136: Sync Pulsar policies across multiple clouds. PIP documentation: https://github.com/apache/pulsar/issues/13728 *Motivation* Apache Pulsar is a cloud-native, distributed messaging framework which natively provides geo-replication. Many

[VOTE] PIP-100: Pulsar pluggable topic factory

I would like to start voting on the PIP. PIP: https://github.com/apache/pulsar/wiki/PIP-100:-Pulsar-pluggable-topic-factory PR: https://github.com/apache/pulsar/pull/12235 Thanks, Rajan On Tue, Sep 28, 2021 at 1:23 PM Rajan Dhabalia wrote: > Hi, > > I would like to propose a Plugga

PIP-100: Pulsar pluggable topic factory

Hi, I would like to propose a Pluggable topic factory in pulsar. Users select the topic type (persistent/non-persistent) based on the application use case and requirement of topic behavior. However, in some circumstances, users need some additional behavior on top of the existing implementation

Re: [DISCUSS] Return the message ID of the first chunk when sending chunked messages

Hi, Though chunked messages are sequential for a specific producer, it's not guaranteed that they will be contiguous when the broker receives them and writes them to a ledger. So, it will be a little tricky to find out the first message-id of any chunked-message at any given time unless broker

Re: [DISCUSS] Contribute "Reactive Streams adapter for the Apache Pulsar Java Client" to the Apache Pulsar project

*My two cents about developing and contributing adapters/connectors for Apache Pulsar. Apache Pulsar was open-sourced with Spark and Storm adapters initially and then such adapter families evolved with multiple other adapters, connectors and IO connectors (sink/source) into apache pulsar git repo.

Re: Proposing a meetup organizing committee

*I think the Apache project should avoid vendor-specific branding, in that way users will have a clear understanding about what they can receive from the free Apache project. In support of that, the Apache project should be driven by the community/PMC who are the real contributors to the project

Re: Treating lookup request timeout as equivalent to TooManyRequests

> so that timeouts on lookup type requests also closes the connection (and hopefully allows establishment of connection on a working broker). +1. Thanks, Rajan On Thu, Aug 12, 2021 at 11:30 AM Ivan Kelly wrote: > Hi folks, > > Rajan's opinion would be particularly useful here as they appears

Re: Lack of retries on TooManyRequests

*The history behind introducing TooManyRequest error is to handle backpressure for zookeeper by throttling a large number of concurrent topics loading during broker cold restart. Therefore, pulsar has lookup throttling at both client and server-side that slows down lookup because lookup ultimately

Re: [VOTE] Apache Pulsar Adapters Release 2.8.0 Candidate 1

ctions are not clear, see my answers > below > > Il Gio 1 Lug 2021, 21:27 Rajan Dhabalia ha scritto: > > > Hi, > > > > Right now, Build is failing with the below error which prevents us from > > building the project with all test execution. README page also suggests >

Re: [VOTE] Apache Pulsar Adapters Release 2.8.0 Candidate 1

Hi, Right now, Build is failing with the below error which prevents us from building the project with all test execution. README page also suggests to build project by skipping tests (using -DskipTests options) [ERROR] Failed to execute goal on project pulsar-kafka-compat-client-test: Could not

Re: [DISCUSS] PIP 83 : Pulsar client: Message consumption with pooled buffer

> > > About the API, my feeling is a RawConsumer is more suitable here, users > don’t need to specify the schema when using the RawConsumer since we might > occurs schema incompatible issue when using Schema.BYTEBUFFER. So > RawConsumer is more clear for this purpose? > > > >

[DISCUSS] PIP 83 : Pulsar client: Message consumption with pooled buffer

Hello, We have created a PIP-83 for Pulsar client: Message consumption with pooled buffer. https://github.com/apache/pulsar/wiki/PIP-83-:-Pulsar-client:-Message-consumption-with-pooled-buffer Thanks, Rajan

[ANNOUNCE] New committer: Enrico Olivelli

The Project Management Committee (PMC) for Apache Pulsar has invited Enrico Olivelli to become a committer and we are pleased to announce that he has accepted. Congratulations and welcome onboard Enrico Olivelli. Please join us to welcome Enrico Olivelli. Thanks

Re: First Pulsar Community Meeting - 2/2 & 2/4

Hi, >> 3. PIP-45: @Rajan Dhabalia and @Matteo Merli to create a task list to track the progress of the development of PIP-45. As per discussion during the meeting, I have created a github project and appropriate issues to document and track all the tasks related to PIP-45: Project:

Re: [E] Re: [PIP-78] Split the individual acknowledgments into multiple entries

o abstract, so that > > different people can choose. > > Your PR is an improvement to the current performance, there is no > conflict > > between them. > > > > Thanks > > > > On 2021/01/27 03:50:07, Rajan Dhabalia wrote: > > > I have created a PR which

Re: [E] Re: [PIP-78] Split the individual acknowledgments into multiple entries

hanks, > Sijie > > On Sat, Jan 23, 2021 at 7:46 PM Rajan Dhabalia > wrote: > > > *Hi, >> Delayed messages or certain user logic can introduce a lot of > > message-holes. We have seen this issue in quite a lot of customers' > > production environment.I agree that

Re: [E] Re: [PIP-78] Split the individual acknowledgments into multiple entries

*Hi, >> Delayed messages or certain user logic can introduce a lot of message-holes. We have seen this issue in quite a lot of customers' production environment.I agree that we need a larger buffer to store and recover individually deleted messages, and the existing 150K limit might not be enough

Re: Proposal for Consumer Filtering in Pulsar brokers

*I also have few concerns with the PIP by introducing server side filtering and processing in dispatch path. Filtering on unbounded tag list will make computation unpredictable and that will be definitely a problem to define the capacity model and can be a big problem of noisy neighbors in a

Re: [DISCUSS] PIP 64: Introduce REST endpoints for producing, consuming and reading messages

*Hi,I would like to add little history about Websocket vs REST API implementation in Pulsar. Before open-sourcing Pulsar and WebSocket proxy in Yahoo, we had implemented and evaluated both WebSocket and REST-HTTP APIs options and then decided to not introduce REST-API and kept Websocket proxy

Re: PIP-55: Refresh Authentication Credentials

Hi, We have the same ask and we had created PR to address the issue last year with the PR: https://github.com/apache/pulsar/pull/3705 Thanks, Rajan On Tue, Jan 14, 2020 at 3:21 PM Matteo Merli wrote: > > https://github.com/apache/pulsar/wiki/PIP-55%3A-Refresh-Authentication-Credentials > >

Re: [VOTE] Pulsar Release 2.4.0 Candidate 2

+1 binding Checked: - signatures and checksum - start standalone service and client-test for producer/consumer - rat check on src distribution Thanks, Rajan On Wed, Jun 26, 2019 at 4:15 AM PengHui Li wrote: > This is the second release candidate for Apache Pulsar, version 2.4.0. > > It

Re: PIP 36: Max message size

does it > make sense to move it to a separated topic? > > Also I have a few questions regarding chunking. I will leave it in your > doc. > > - Sijie > > On Fri, May 10, 2019 at 1:08 PM Rajan Dhabalia > wrote: > > > Hi, > > > > >> It can be easier to

Re: PIP 36: Max message size

Hi, >> It can be easier to achieve when Transaction is supported. A large message can be split into a txn of smaller messages. I would like to propose another approach "Chunking and combine" to publish large size of messages. Transaction deployment might not be convenient all the time specially

Re: [VOTE] Pulsar Release 2.3.0 Candidate 1

Checked: - signatures and checksum - start standalone service and client-test for producer/consumer (src/bin distribution) - rat check on src distribution - compile and unit test on src distribution Thanks, Rajan On Fri, Feb 15, 2019 at 12:10 AM Matteo Merli wrote: > This is the first

Re: [DRAFT] November report

+1. On Wed, Nov 14, 2018 at 11:36 AM Matteo Merli wrote: > Please take a look > > - > > ## Description: > > Pulsar is a highly scalable, low latency messaging platform running on > commodity hardware. It provides simple pub-sub semantics over topics, > guaranteed at-least-once delivery of

Re: Limitations on No.Of Producers and Consumers

y to contact cluster-1 and cluster-1 can redirect request to appropriate cluster into the same region. On Tue, Nov 13, 2018 at 12:00 PM Ivan Kelly wrote: > On Tue, Nov 13, 2018 at 8:23 PM, Rajan Dhabalia > wrote: > >> Is there any documentation on how to use it? > > > &g

Re: [VOTE] Pulsar Release 2.2.0 Candidate 2

+1 (binding) Environment: Mac-OS 10.13 Checked: - signatures and checksum - maven repository artifacts - start standalone service and client-test for producer/consumer (src/bin distribution) - rat check on src distribution - compile and unit test on src distribution Thanks, Rajan On Tue,

Re: Pulsar release 2.2

Hi Joe, This bug has been fixed in master with #2693 . So, rebasing with mater will fix it. Thanks, Rajan On Tue, Oct 2, 2018 at 7:16 PM, Joe F wrote: > Please, if you know of a problem that is bound to