On Tue, 24 Sept 2024 at 05:01, Rajan Dhabalia <rdhaba...@apache.org> wrote: > However, there are multiple other PRs related to key-shared sub, stats, > cursor performance, and other PRs are still blocked by others and people > just block it because they think they don't have this usecase. It's so > unfortunate that people easily merge implementations which only handle > small-scale usecases but the usecases for which Pulsar was built are > being blocked or take a long time to merge. It's just that I don't have > that energy to keep following up for useful and important changes for > Pulsar. And this is one of these examples as well. I have also started > discussion about improving the PIP process because it has become painful in > many cases.
It's not that individuals want to block changes for no reason. It seems that the main reason for blocking changes is the fear of regressions. Some areas of the Pulsar codebase aren't well covered in our test suites. For example, we don't have performance tests as part of the Apache Pulsar repositories. We have a lot of tests, but most of them are written in a way that tests the code as the author expects it to work. There are very few tests that evaluate features from the end-user API perspective or as system tests. Writing new tests is slow, and the developer experience is poor with the current test infrastructure. Adding more tests to the main build would slow down Pulsar CI even more. This isn't a new problem; it's been around for many years. I'd love to see more proposals and active contributions to improve the "safety nets" of Apache Pulsar so that we wouldn't fear change. I'm not saying that this is only a testing problem. Testability impacts architecture too. Balancing all different aspects of the system isn't easy, and it requires effort and dedication. We don't currently have enough contributors who are investing their time in enabling others to contribute effectively. I hope that we can improve together and address the problems we have that cause the fear of change. When that is addressed, there would be more confidence in accepting new PIPs and changes even when the reviewer doesn't have the use case or when they aren't familiar with the problem that the PIP is targeting to solve. -Lari
