Re: [DISCUSS] Planning for Apache Pulsar 3.0

Similarly to Lari, I hear your concerns about not breaking client APIs compatibility, but I share his view of being playful about the changes. IMO, this mindset is essential for brainstorming. When delivering we should then do that responsibly and according to a plan. The plan is the one ensuring

Re: [DISCUSS] Planning for Apache Pulsar 3.0

Il giorno mer 12 ott 2022 alle ore 00:40 Matteo Merli ha scritto: > > Agree, though let's make separate discussions. Putting all random > ideas into the same cauldron is a good recipe for making no one able > to follow or see a common line. > > That's what I meant when I started the proposal of

Re: [DISCUSS] Planning for Apache Pulsar 3.0

Agree, though let's make separate discussions. Putting all random ideas into the same cauldron is a good recipe for making no one able to follow or see a common line. That's what I meant when I started the proposal of having 3.0 completely detached from "features". If you start making a big

Re: [DISCUSS] Planning for Apache Pulsar 3.0

Let’s discuss any and all ideas for improvement. As each is discussed we can figure out how to make them non-breaking, We all want Pulsar to improve. We should encourage an open discussion where no idea is automatically bad or wrong. They can just be discussed without fear. Thanks, Dave > On

Re: [DISCUSS] Planning for Apache Pulsar 3.0

> If you want to work on cloud-events > integration, please do. I still don't see how's that related to Pulsar > 3.0 discussion. Would a change to SchemaInfo break the client? Devin G. Bost On Tue, Oct 11, 2022 at 11:58 AM Matteo Merli wrote: > > Companies want to be cloud agnostic since it

Re: [DISCUSS] Planning for Apache Pulsar 3.0

> Companies want to be cloud agnostic since it reduces their risk, and the idea of multi-cloud integration is very attractive to companies that want the best of both worlds or need to integrate data across platforms. If Pulsar is not part of this wave, I'm concerned we will be left behind. That

Re: [DISCUSS] Planning for Apache Pulsar 3.0

How about the gap between SchemaInfo and interoperability with CloudEvents? We're seeing a convergence between cloud providers on CloudEvents. I just learned that SAP Event Mesh now is working on integration with Azure EventGrid

Re: [DISCUSS] Planning for Apache Pulsar 3.0

I would prefer that we avoid using the term “breaking changes”, which is too vague to convey any specific meaning. So let me try to bring some clarity. There have been many changes to implementations, APIs and data storage formats in Pulsar (and book keeper also). I have deployed many of these

Re: [DISCUSS] Planning for Apache Pulsar 3.0

Rajan, Perhaps you missed this reply which lists one area of potential improvement. What are your thoughts about this type of client issue? Do you have an approach where this is not an issue for you? Thanks, Dave Sent from my iPhone > On Oct 7, 2022, at 6:42 PM, Lari Hotari wrote: > >

Re: [DISCUSS] Planning for Apache Pulsar 3.0

I would say first we should gather a list of changes which we want to target and find out which improvements really need major version release. We can take the Pulsar-1.0 to Pulsar-2.0 upgrade example to avoid major interruption and impact on existing systems and still achieve our goal. So, the

Re: [DISCUSS] Planning for Apache Pulsar 3.0

I'm noticing some pushback on the idea of pre-emptively proposing any kind of breaking upgrade that would necessitate cutting a 3.0 release. I do understand the concern about introducing a breaking change... For a distributed messaging application like Pulsar, if clients needed to be

Re: [DISCUSS] Planning for Apache Pulsar 3.0

This sounds like the current state of Apache Pulsar has a lot of issues and it requires fundamental design changes to make it promising which is definitely not true and I disagree with it. And I would be careful comparing with Kafka as I still don't think the Kafka release has anything to do with

Re: [DISCUSS] Planning for Apache Pulsar 3.0

Thanks Lari. I have a pain point that is about documentation and developer awareness where we can help improve quality and suggest improvements. 1) Work on clear documentation about each thread pool and how it is used by the classes in the message path. 2) Add thread pool impacts to the PR

Re: [DISCUSS] Planning for Apache Pulsar 3.0

Thank you, Matteo. I'm just adding that it all makes sense that what you said. I hope that my other email clarified the need to get loose of any restrictions for a while in the planning of Pulsar 3.0. Making the changes a reality is something where the practice of incremental changes that you

Re: [DISCUSS] Planning for Apache Pulsar 3.0

Thanks, Dave for sharing your views. I entirely agree. -Lari On 2022/10/08 01:41:47 Dave Fisher wrote: > To me the point of this discussion is for the community to discuss both > improvement ideas along with current pain points in the broker. Whether or > not the ideas and proposed solutions

Re: [DISCUSS] Planning for Apache Pulsar 3.0

We could all have our own favorite names for this work. :) There's advice that you should disrupt yourself before someone disrupts you. Shouldn't we follow that advice for Apache Pulsar? We can disrupt Pulsar together with our Apache hats on. The catch is that since we are doing this, we will

Re: [DISCUSS] Planning for Apache Pulsar 3.0

Thank you Rajan for explaining the crucial importance of client compatibility in a very clear way. Ensuring client compatibility is a "must" requirement. We happened to have some discussions about this topic at ApacheCon, and how to ensure protocol compatibility in the cases where there would

Re: [DISCUSS] Planning for Apache Pulsar 3.0

To me the point of this discussion is for the community to discuss both improvement ideas along with current pain points in the broker. Whether or not the ideas and proposed solutions are breaking or incremental is yet to be determined. Let’s not get hung up on versions or nomenclature. All

Re: [DISCUSS] Planning for Apache Pulsar 3.0

I actually disagree with the term "Pulsar Next Gen", because I haven't seen any proposal for which that would make sense to me to be called so. Rajan: That's the whole point of breaking it down. If you accumulate many "big" changes it introduces a lot of risk for instabilities and

Re: [DISCUSS] Planning for Apache Pulsar 3.0

Hi, Can we get the list of changes at one place which we are planning to get as part of 3.0. One thing I would like to see as a part of a major release, it CAN NOT impact existing usecases and users in any way which can force them to upgrade the client library. Applications using < 3.0 version

Re: [DISCUSS] Planning for Apache Pulsar 3.0

Thank you, Matteo. I agree that features should be delivered continuously when that is possible. In this case, that might not apply. I also agree that calling this Pulsar 3.0 isn't necessarily aligned with PIP-175 since an LTS release is when the major version is bumped. I'm fine in calling

Re: [DISCUSS] Planning for Apache Pulsar 3.0

Given the past experiences and the discussions that already happened around "PIP-175: Extend time based release process", the idea is to detach the 3.0 from "big-features" items or "incompatible changes". The changes are going to get included as they are ready, within feature releases, and in a

[DISCUSS] Planning for Apache Pulsar 3.0

Hi all, Greeting from ApacheCon North America 2022 from New Orleans! We had a great conference with a dedicated Pulsar track. Thanks to all presenters and everyone who attended. The talks weren't recorded, but the slides will be later on posted on the conference website [1]. At ApacheCon