Thanks for the proposal. It is a good idea. It is also not a bad idea to incubate it outside of Pulsar official repository at first.
On Tue, Jul 23, 2024 at 2:59 PM Aurora Twinkle <foreverlove...@gmail.com> wrote: > Hi: > I think this is a very good idea. > > In the native pulsar warehouse, there are some interfaces that have no > default implementation classes, such as ConsumerInterceptor, > ProducerInterceptor. Users must implement the interface themselves, > which increases the cost of use. If there is a plug-in library > mentioned in this PIP, we can provide some common interface > implementations in the plug-in library, such as mertics collection > interceptor for producer and consumer. > > Best Regards, > AuroraTwinkle > > xiangying meng <xiangyingme...@gmail.com> 于2024年7月23日周二 11:25写道： > > > > Hello Wen Zhi, > > > > This project is a new repository similar to a connector, and it will > > not make any modifications to Pulsar itself. [0] > > > > Its code part is simply a plugin library, based on Pulsar's external > > interfaces, providing a rich set of implementations. It only retains a > > "curated list", which is a series of pointers to personalized features > > and experimental features in personal repositories for reference and > > inspiration. If these experimental features are recognized by other > > users, they can also be gradually merged into the Pulsar main > > repository. However, it will not fork the content of the Pulsar > > repository for modification in the new repository. > > This is the improvement we made after discussion in the last email. I > > will synchronize it to the proposal later. > > Finally, thank you for your valuable suggestions, and we look forward > > to your continued participation. > > > > [0] - https://github.com/apache/pulsar-connectors > > > > Best Regards, > > Xiangying > > > > On Tue, Jul 23, 2024 at 10:36 AM WenZhi Feng <thetumb...@apache.org> > wrote: > > > > > > Hi, xiangying, > > > First of all, thanks for driving the proposal. > > > I have some questions. > > > 1. Will this repository maintained by committers/pmc? If the answer > is yes, there will be too many new modules introduced into pulsar and hard > to maintain. If the answer is no, i think we have better not to include it > into pulsar official repository. > > > 2. For those plugin-in feature, we may create a new repository like > various connectors for pulsar. But for those core code change on pulsar, > how can we seperate it with pulsar repository? > > > It is thrilling that users contribute various new features to apache > community, enriching the ecosystem. I think we should find out a good way > to realize it. > > > > > > Thanks, > > > Wenzhi Feng. > > > > > > On 2024/07/22 17:40:24 xiangying meng wrote: > > > > Thank you for your feedback, let's further clarify this matter. > > > > > > > > The original design of this proposal is a new contributor repository, > > > > and provides two ways to contribute. > > > > The first one: > > > > Contributors contribute through PR and accept code review, and can > > > > only be merged into the main branch after reviewed. The code merged > in > > > > this way is various plug-ins based on Pulsar's external interface. It > > > > does not contain any pulsar code, only Pulsar's dependencies. > > > > The second one: > > > > Features that require modifications to the Pulsar main repository > code > > > > and may conflict with later versions. My initial idea was for > > > > contributors to create branches and then write the commit id and > > > > features description into the document after the PR is reviewed. > > > > > > > > However, after the discussion just now, I agree that the second one > > > > may not be maintained in the branch of the new repository, but in the > > > > contributor's personal repository. > > > > > > > > Based on the above content, let's continue to look at the question > you raised. > > > > > > > > >- How can the Pulsar community maintain a fork of Pulsar itself ? > > > > > > > > If it is just plug-in library and a feature list document of the > > > > features in personal repository, then there is no need to fork > Pulsar > > > > itself. > > > > > > > > >- Do you want to "cut releases" out of this fork ? Who is going to > validate them ? Who is responsible for security bugs? > > > > > > > > We will only need to maintain a documents of feature and plugins list > > > > that have been reviewed by PR. We will not make changes to the Pulsar > > > > code itself. There is no fork of Pulsar. It will have its own version > > > > release process, just like Pulsar. > > > > > > > > -> Who is going to use this code ? > > > > > > > > IMO, providing various implementation plugins based on Pulsar's > > > > external interface is meaningful in itself. While introducing > Pulsar's > > > > own dependencies, introducing plugin libraries to use various already > > > > implemented plugins can reduce development costs. I think some people > > > > will be happy to use them. > > > > > > > > As for the other functions in the document, maintained in personal > > > > repositories, they can be used as a reference and provided to > > > > companies that are capable of solving security issues and bugs. > > > > > > > > Best regards, > > > > Xiangying > > > > > > > > On Tue, Jul 23, 2024 at 12:03 AM Enrico Olivelli < > eolive...@gmail.com> wrote: > > > > > > > > > > Il giorno lun 22 lug 2024 alle ore 17:34 xiangying meng < > > > > > xiangyingme...@gmail.com> ha scritto: > > > > > > > > > > > >thanks for your proposal, we need more and more energy in the > project. > > > > > > > > > > > > Yes, it requires a lot of participation, but sometimes we also > need a > > > > > > start. > > > > > > > > > > > > >Also we allow only "committers" to commit patches to the > repositories > > > > > > that are under the responsibility of the Apache Pulsar PMC, I > don't know > > > > > > how we can make this work with a "official fork" > > > > > > > > > > > > >An alternative is to have a "curated list" of links to personal > projects, > > > > > > but we keep the responsibility over the code on the code owners > and > > > > > > not on the Pulsar PMC > > > > > > > > > > > > Yes, this is a "curated list" link, but it points to the branch > and > > > > > > commit ID created by the contributor himself. In addition, for > some > > > > > > interface implementation classes, they can be placed in the main > > > > > > branch of the new contributor's repository and then added to the > > > > > > "curated list". Only committors or maintainers can add content > to the > > > > > > "curated list". I have detailed these in the proposal. > > > > > > > > > > > > But no matter which method, each function will be marked in the > > > > > > document, that is, in the "Featured List" who contributed it. I > > > > > > understand that this feature is more of a reference feature. > Allow > > > > > > users to choose these customized and risky features to customize > their > > > > > > own versions. This is very useful for some large companies with > > > > > > development capabilities. At the same time, its responsibility > should > > > > > > belong to the user, although we will also point out who is the > > > > > > contributor of each feature he chooses. But when using > innovative or > > > > > > customized features, users should also be clear about the risks. > I > > > > > > also wrote about the risks of contributor repositories in the > > > > > > proposal. > > > > > > > > > > > > > > > > The main point is that we cannot grant "write" permissions to > repositories > > > > > owned by the Pulsar project (the PMC) > > > > > to people who are not "committers". This is a legal thing, no > exceptions. > > > > > > > > > > So any contribution must be in the form of a PR, like for any > other patches > > > > > contributes to one of the github repositories owned by the Pulsar > project > > > > > > > > > > A couple of questions: > > > > > - How can the Pulsar community maintain a fork of Pulsar itself ? > > > > > - Do you want to "cut releases" out of this fork ? Who is going to > validate > > > > > them ? Who is responsible for security bugs ? > > > > > - Who is going to use this code ? > > > > > > > > > > I think that when you want to contribute a feature, you can do it > with a PR > > > > > or with the PIP process (if it is bigger), and it is fine to mark > it as > > > > > "experimental". > > > > > > > > > > Then the community decides whether to support it or not. > > > > > > > > > > When a patch is committed to our repository then it is the > responsibility > > > > > of all of us (the PMC and the committers) to maintain it almost > forever, > > > > > unless we decide to delete it. > > > > > > > > > > This is a legal thing, we cannot accumulate code without being > responsible > > > > > for that. > > > > > > > > > > I am happy to clarify more if needed, and I hope that other in the > > > > > community can help in understanding if this is doable of not > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > > > > > > > > In addition, thank you for your valuable feedback. > > > > > > > > > > > > Best regards > > > > > > Xiangying > > > > > > > > > > > > On Mon, Jul 22, 2024 at 10:55 PM Enrico Olivelli < > eolive...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > Hello, > > > > > > > thanks for your proposal, we need more and more energy in the > project. > > > > > > > > > > > > > > I don't think that this is a good idea. > > > > > > > > > > > > > > The main reason is that as an ASF project we must guarantee > the quality > > > > > > and > > > > > > > especially the security of what we provide to the public. > > > > > > > > > > > > > > Such kinds of repositories tend to become full of stale code > that can > > > > > > > become a source of security issues. > > > > > > > > > > > > > > Also we allow only "committers" to commit patches to the > repositories > > > > > > that > > > > > > > are under the responsibility of the Apache Pulsar PMC, I don't > know how > > > > > > we > > > > > > > can make this work with a "official fork" > > > > > > > > > > > > > > Maybe I misunderstood the goal ? > > > > > > > > > > > > > > An alternative is to have a "curated list" of links to > personal projects, > > > > > > > but we keep the responsibility over the code on the code > owners and not > > > > > > on > > > > > > > the Pulsar PMC > > > > > > > > > > > > > > > > > > > > > Best regards > > > > > > > Enrico Olivelli > > > > > > > > > > > > > > > > > > > > > > > > > > > > Il giorno lun 22 lug 2024 alle ore 15:15 steven lu < > > > > > > lushiji2...@gmail.com> > > > > > > > ha scritto: > > > > > > > > > > > > > > > We think this is a very good solution, > > > > > > > > Our XHS team(@liangyepianzhou < > https://github.com/liangyepianzhou> > > > > > > > > @AuroraTwinkle <https://github.com/AuroraTwinkle> > @StevenLuMT > > > > > > > > <https://github.com/StevenLuMT> @cai152 < > https://github.com/cai152>) > > > > > > will > > > > > > > > submit a few to pulsar-java-contrib > > > > > > > > <https://github.com/StevenLuMT/pulsar-java-contrib>( > > > > > > > > https://github.com/StevenLuMT/pulsar-java-contrib) as a > primer to let > > > > > > > > everyone know the role of this library > > > > > > > > > > > > > > > > xiangying meng <xiangyingme...@gmail.com> 于2024年7月22日周一 > 16:33写道： > > > > > > > > > > > > > > > > > Hello Pulsar Community, > > > > > > > > > > > > > > > > > > I hope this message finds you well. I'm reaching out to > propose the > > > > > > > > > establishment of a Pulsar Contributor Repository. > > > > > > > > > > > > > > > > > > This new initiative is designed to provide a dedicated > space for > > > > > > > > > experimental and community-driven features that complement > our core > > > > > > > > > offerings. This repository would serve as a space for > non-core > > > > > > > > > features and customizations, allowing for greater > flexibility and > > > > > > > > > community involvement. > > > > > > > > > > > > > > > > > > The proposal can be viewed at this GitHub link: > > > > > > > > > https://github.com/apache/pulsar/pull/23061/files > > > > > > > > > > > > > > > > > > I'm looking forward to your thoughts and feedback. Please > feel free > > > > > > to > > > > > > > > > share them on the Pulsar Developer Mailing List. > > > > > > > > > > > > > > > > > > Thank you in advance for your time and consideration. > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > > > > Xiangying > > > > > > > > > > > > > > > > > > > > > > > > > > > >
