GitHub user numinnex edited a discussion: Multi-phase, prolonged feature development discussion.
As we are approaching the end of development of the `io_uring_tpc` feature, I'd like to propose a different development framework for the next (even bigger) feature, clustering. ## Background: Current Approach For `io_uring_tpc` we used a long-lived feature branch, with no CI checks during merges (so we could merge changes incrementally from other contributors, without needing to fix the tests a priori) and branch protection enabled (1 approval). This initially sounded like a reasonable approach, with a few downsides (e.g., contributions to `master` that touch server had to be frozen). However, as development continued, it became clearer that this approach is problematic. ## Pain Points - Due to branch protection being enabled, it made it impossible to rebase it against `master`, as it requires pushing with `--force`. - Since `rebase` was not an option, we were forced to `merge` the master branch into the feature branch, which in the long run led to commit duplication (some of the commits on the feature branch from previous upstream merges were conflicting with the same commits that were brought by the current merge). - The assumption that we have to freeze the `master` branch from changes to the server was foolish. In reality, you can't predict what parts of the codebase will require changes when developing a feature. - Introducing external contributors to the workflow introduces extra obstacles (different workflow compared to `master` branch, created issues require a special label that directs towards the feature branch). ## Proposed Framework Using those lessons, I propose a different development framework. Deprecate the idea of a feature branch as the core development branch, use `master` branch instead. In order to avoid conflicts between fixes/patches that would usually go directly to `master`, the feature currently under development must be divided into the smallest possible atomic units, such that each of those units becomes a separate `crate`. Each of those units is assigned to one of the contributors (single responsibility). Contributors can upstream small changes frequently without conflicts, allowing us to move fast and identify interface incompatibilities during code review. There is no need for a `master` branch freeze anymore, as there should be no contention between the feature development and fixes/patches/other contributions. The only time when a branch freeze is required is during the "plumbing" (combining the modules together). ## Additional Benefits Another upside of this approach is moving towards our goal of modularizing the codebase, we are killing two birds with one stone. ## Addressing Concerns At first glance it looks like a lot of work to get into a workable solution (e.g., pre-planning the feature division into crates), but I'd argue that: - We need to do it anyway (goal of modularizing). - This won't be the last time we'll need to work on a feature for an extended period. So I deem the time investment worth it. ## Discussion If you have any other experiences working with such scenarios, please share them in this discussion. From my side, I've worked with long-lived feature branches twice, both times it turned out to be painful. GitHub link: https://github.com/apache/iggy/discussions/2281 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
