I have very high doubts about this working and I really hope I understood this wrong.
PRs sent to `kie` will be checked using a new GitHub Actions > hosted inside it and WON’T build `kie-extras` A new weekly bot is created to update the versioned `kie` SHA at > `kie-extras` via PR. Contributors to `kie-extras` start their week > taking a look at it and working together to get that merged ASAP. Each merged PR should have our code base in a state that works from top to bottom. This is people working on kie pushing work for those working on kie-extras. Kie dev/vendor is losing time and/or money for the kie-extra devs and vendors. We are independent developers from the community perspective. When the kie PR is merged there is a task for someone to do and no requirement for anyone to do it. There isn't even a requirement for anyone to revert any commit. So now kie-extras dev needs to touch kie and undo work that was already approved or not do that and we fight about who fixes it. There is no guarantee the change done at kie is fixable in kie-extras. Then it needs to be reverted in kie and that takes a week. The repositories will be out of sync for 2 weeks. I do like the idea of reducing the repositories. Most bug fixes I used to do hit 3-4 repositories and I spent weeks restarting the PR builds every workday to get a full green. With this cycle even a 2 repo change can take months. -1. Toni On Tue, Feb 18, 2025 at 10:34 AM Kennedy Bowers <kbow...@apache.org> wrote: > +1 > > On 2025/02/18 07:53:06 Yeser Amer wrote: > > +1 > > > > Thank you. > > > > On 2025/02/17 23:23:14 Alex Porcelli wrote: > > > +1 > > > > > > On Mon, Feb 17, 2025 at 6:22 PM Tiago Bento <tiagobe...@apache.org> > wrote: > > > > > > > Hi everyone, > > > > > > > > As a final attempt to shift the focus of the Apache KIE community > away > > > > from build systems and codebase structure and to allow us to move > > > > forward with a good release cadence while fostering innovation in the > > > > Open-Source Business Automation space, Alex and I wrote what we hope > > > > to be the last infrastructure-related proposal from us (for quite a > > > > while, at least). > > > > > > > > We sincerely believe that, if approved, this proposal will put Apache > > > > KIE in a very advantageous place to welcome new contributors, > > > > encourage current contributors to do their best work yet and multiply > > > > the reach of our technology to even more users. We hope everyone > > > > reading this feels heard as we try to incorporate all the feedback we > > > > gathered since our first day inside Apache. > > > > > > > > We opted to go for a [VOTE] directly to avoid risking an endless > > > > conversation again. While we understand this is not the preferred > > > > approach for contributing, this subject is still sensitive enough for > > > > us to have chosen this method, hopefully for the last time. > > > > > > > > Tiago and Alex. > > > > > > > > # > > > > # Problem scope > > > > > > > > Recently during the many threads that were raised in this mailing > > > > list, it became clear that there’s a general concern about the > > > > direction of the codebase structure and a continued feedback against > > > > the monorepo-ization of the Apache KIE codebase, which seems to be > > > > stealing our energy and making it impossible to focus on what we all > > > > do best: features and innovation. > > > > > > > > It’s also clear that the current infrastructure hasn’t been helpful. > > > > The level of complexity necessary to maintain the current > repositories > > > > is holding us back from a more streamlined approach to PR checks and > > > > CI and, most importantly, to our releases, preventing us from > > > > maintaining a good cadence. > > > > > > > > This proposal brings a compromise to the table, between the current > > > > poly-repo and a monorepo: a duo-repo, with one repo named `kie`, > > > > hosting all Java runtime code, and another repo named `kie-extras`, > > > > hosting everything else (including Docs, Examples, Websites, > Container > > > > images, Editors, Web apps, Quakus Dev UIs, Jenkins release jobs etc). > > > > > > > > The duo-repo-ization of the Apache KIE codebase will allow us to > > > > dramatically simplify the automations we have for PR checks, CI and > > > > release jobs as well as the operation for daily contributions, > > > > stopping the dependence we have on SNAPSHOTs, for example. The > > > > ultimate goal is to replace the complicated `build-chain`-based PR > > > > checks and the custom Jenkins framework we have for CI/releases. > > > > > > > > # > > > > # What is not in scope > > > > > > > > - Cross-repo PRs; as it leads to systems similar to `build-chain` > > > > and the custom Jenkins framework. We want to avoid going in that > > > > direction again. > > > > - Sparse-checkouts mechanism for `kie`; Due to the complexity of > > > > setting up something like this, and the other mechanisms that exist > to > > > > mitigate the increase in size of the repo, this won’t be pursued > > > > initially. > > > > - A complete review of all repositories under > `apache/incubator-kie-*` > > > > > > > > # > > > > # The proposal > > > > > > > > Changes to repositories: > > > > - `drools` is: > > > > - Renamed to `kie`, keeping the most-starred repository we > have on > > > > GitHub. > > > > - Updated to include `optaplanner`, `kogito-runtimes` and > > > > `kogito-apps`; preserving Git history. > > > > - New top-level modules will be created for each > repository, > > > > making things simpler at the beginning. > > > > - Top-level modules prefixed with `drools-` go to a new > > > > folder called `drools`, and top-level modules prefixed with `kie-` go > > > > to a new folder called `kie`. > > > > - `kie-tools` is: > > > > - Renamed to `kie-extras`; reflecting its multi-purposed nature > > > > of hosting a variety of different package types > > > > - `optaplanner`, `kogito-runtimes`, and `kogito-apps` are: > > > > - Updated with READMEs/descriptions pointing to the new `kie` > repo > > > > - Archived by Apache INFRA once `kie` is fully operational > > > > - `kogito-pipelines` is: > > > > - Updated with a disclaimer on its README saying it’s no > longer used. > > > > - Archived by Apache INFRA once `kie` is fully operational > > > > > > > > Changes to automations: > > > > - Jenkins is almost completely reset, except for a few jobs > > > > necessary for the Release Procedure [1]. > > > > - Timestamped SNAPSHOTs stop being published every Sunday. > > > > - Regular SNAPSHOTs stop being published every day. > > > > - `apache.snapshots` repository is cleaned up so that no SNAPSHOTs > > > > are available anymore. > > > > - PRs sent to `kie` will be checked using a new GitHub Actions > > > > hosted inside it and WON’T build `kie-extras` > > > > - PRs sent to `kie-extras` will be checked as they already are, > with > > > > a new step at the beginning that will build (or fetch from a cached > > > > GitHub Actions artifact) `kie` at a versioned commit SHA. > > > > - A new weekly bot is created to update the versioned `kie` SHA at > > > > `kie-extras` via PR. Contributors to `kie-extras` start their week > > > > taking a look at it and working together to get that merged ASAP. > > > > - A new release job stored in `kie-extras` is created to build and > > > > publish `kie` on a Release Candidate. > > > > > > > > Communication: > > > > - Apache KIE’s website and documentation are updated to reflect the > > > > new duo-repo structure. > > > > > > > > ### Impact on operation and contributions > > > > > > > > - Contributors to `optaplanner`, `kogito-runtimes` and > `kogito-apps` > > > > will move to `kie` > > > > - PR checks will take the same time (or less, if we’re able to > > > > capitalize more on parallel builds, given every module is going to be > > > > part of the same build in Maven’s reactor). > > > > - Unified Maven dependency management, less surface to manage > > > > external dependencies. > > > > - Contributions to `kie-extras` that depend on `kie` will continue > > > > being bi-phased, the same way they are today with the timestamped > > > > SNAPSHOTs. > > > > > > > > ### Notes on SNAPSHOTs > > > > > > > > - According to Apache’s guidelines on unreleased artifacts, we > > > > should be very conscious about our SNAPSHOT usage in regards to > > > > encouraging users to consume them or even to depend on them. See > > > > https://infra.apache.org/release-distribution.html#unreleased > > > > - Users will always have access to the latest release and will be > > > > encouraged to step on the development itself to access the bleeding > > > > edge (Building 999-SNAPSHOT locally from `kie` @ `main`). > > > > - Given the discrepancy between `kie` and `kie-extras`, to make CI > > > > simpler and not delve into cross-repo PR systems, it would be > > > > confusing to have some SNAPSHOTs published from `kie-extras` that are > > > > not aligned with SNAPSHOTs from `kie`. > > > > - Given the simplified structure of the codebase, publishing > > > > SNAPSHOTs would encourage usage that’s either not aligned with Apache > > > > or with our strategy of not having cross-repo PRs. We can revisit > that > > > > in the future if we find a justification for such. This proposal > > > > doesn’t want to get in the way of any future work, just establish a > > > > comprehensive baseline for us to evolve on top of. > > > > > > > > # > > > > # Actionable items (plan) > > > > > > > > Phase 1 (New structure) > > > > - A new GitHub Actions workflow is created on `drools` for PR > checks > > > > (without building any other repos) > > > > - `optaplanner` is incorporated as a top-level folder on `drools`, > > > > preserving Git history > > > > - `optaplanner`’s README is updated, pointing to `drools` as its > new > > > > home. Contributions to it are frozen. > > > > - `kogito-runtimes` is incorporated as a top-level folder on > > > > `drools`, preserving Git history > > > > - `kogito-runtimes`’s README is updated, pointing to `drools` as > its > > > > new home. Contributions to it are frozen. > > > > - `kogito-apps` is incorporated as a top-level folder on `drools`, > > > > preserving Git history > > > > - `kogito-apps`’s README is updated, pointing to `drools` as its > new > > > > home. Contributions to it are frozen. > > > > - `drools` is renamed to `kie` (GitHub redirects will make sure > > > > users accessing/cloning `drools` automatically go to `kie`) > > > > - `kie`’s README is updated to reflect its new top-level packages. > > > > - `kie-tools` is renamed to `kie-extras`. > > > > - `kie-tools` has its reference to `kie` updated to a versioned > > > > commit SHA, at the same time as its PR checks and CI are updated to > > > > build `kie` prior to building itself. > > > > > > > > Phase 2 (Cleanup) > > > > - `build-chain`-based GitHub Actions are deleted from all > repositories. > > > > - The custom Jenkins framework jobs are removed from Jenkins > > > > - Update Documentation and Websites to reflect new codebase > structure > > > > > > > > Phase 3 (Release) > > > > - A new release job is created on Jenkins (hosted on `kie-extras`) > > > > to release Maven modules from `kie`. > > > > - The Release Procedure for 10.2 is updated to accommodate the new > > > > codebase structure > > > > > > > > Phase 4 (Archive Repos) > > > > - kogito-pipelines, optaplanner, kogito-runtimes, and kogito-apps > > > > are archived by INFRA > > > > > > > > # > > > > # Commitment and rough timeline > > > > > > > > Alex and Tiago are fully committed to executing this proposal with a > > > > start date right after 10.1.0 is fully released (approved by IPMC and > > > > published to all external registries). > > > > > > > > The rough estimate to execute is about 4 weeks; with [HEADS UP]s > > > > coming to the mailing list whenever a change approaches. > > > > > > > > 10.2.0 will be the first release already with the new structure, > where > > > > the new release jobs will be validated. > > > > > > > > > > > > [1] > > > > > https://docs.google.com/document/d/15V-1e62bmmWeOdFUnSJHymPYoeb0TTwp0bhxPuJn4zo > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > > > > For additional commands, e-mail: dev-h...@kie.apache.org > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > > For additional commands, e-mail: dev-h...@kie.apache.org > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > For additional commands, e-mail: dev-h...@kie.apache.org > >
