Hi all, We are using PIP for adopting major changes to Pulsar. It was done in a lazy-consensus way. It was very efficient for growing the community and driving innovations in the Pulsar community. However, we don't have a clear workflow for PIP. It is actually causing some confusion and inconsistencies.
1) New contributors don't know where to start with new major contributions. 2) We don't have a clear definition of what are major changes. I'd like to start a thread to discuss a few things about PIP and try to formalize the PIP workflow. a) What is considered a "major change" that needs a PIP? b) What should be considered to be included in a PIP? c) What is the process? -- Here is my proposal. a) Any of the following should be considered a major change: - Any major new feature, subsystem, or piece of functionality - Any change that impacts the public interfaces of the project. What are the "public interfaces" of the project? All the following are public interfaces that people build around: - Client and Admin API. - Wire protocol. - Storage (both data and metadata) formats. - User-facing scripts/command-line tools, i.e. bin/pulsar, bin/pulsar-admin, bin/pulsar-daemon. - Docker images - Configuration settings - Exposed monitoring information (Prometheus metrics, topic stats and etc). For the most part monitoring, command-line tool changes, and configs are added with new features so these can be done with one single PIP. And they should be well documented. All these major changes should be shouted out in the release notes. b) I don't think we need to enforce a format for this PIP. However, we should encourage people including the following format. - Motivation: describe the problem to be solved. - Proposed changes: describe the new thing you want to do. - New / Changed Public Interfaces: impact on any of the "compatibility commitments" described above. The changes should be incorporated into documentation and called out in the release note. So everyone thinks about them. - Migration Plan and Compatibility: If this feature requires additional support for a non-downtime upgrade, describe how that will work. - Rejected Alternatives: What are the other alternatives you considered and why are they worse? c) What is the process? Anyone can initiate a PIP but you shouldn't do it unless you have the intention of getting the work done to implement. You can write a PIP in a google doc or a gist and send an email to dev mailing list to start a conversation with the community. The PIP is adopted via a lazy-consensus approval process. You can start working on the PIP unless you see concerns from other people in the community. Try to answer those comments and improve your PIP.
