Please let me amend my previous email, and add in some explanation. >From an outside contributor perspective, we're broken. We are the Apache KIE >project. Now that project has multiple sub areas (I won't call them projects) >that were at one point separate, but are now considered one. We have Drools, >Kogito/jBPM, OptaPlanner, Sonataflow, tooling (which itself has three >different parts Chrome, VS Code, and Stand Alone) and supporting repos (docs, >website, CI/CD, etc.). If I want to contribute something to Kogito one has to >work in kogito-runtimes, maybe kogito-apps, maybe docs, maybe website, maybe >examples, and maybe drools because of the interdependencies of each project. >Each one of these repositories has its own getting started and setup. Maybe it >is Java and Maven, maybe it's the whole kie-tools stack, maybe it is antora or >the website stack, just depends on what you need to do.
Once you have that all setup, you can then do the work and submit your PR(s). If that PR spans multiple repositories, they'll have to tie them all together so the team can take them all in as a whole. The contributor also needs to wait for CI to run, which may or may not run green. It's been a bit since we've had a fully green run on everything, which may make it seem like the contributor has done something wrong. That may not be the case, it might be something else down stream in the CI run that is off, even if the PR is only in one repository. This is a difficult first experience for a new contributor. I think this is broken. It doesn't lead to a good first impression of the project. I know we've beaten this idea many times about where to put things, how to do it, etc. Regardless of what anyone's preferred tool, repo strategy, or methodology is, please look at this project as a whole and think about how it is viewed as someone coming in fresh from the outside. As for my vote: I am modifying it to a +1 for moving things forward, and a 0 for how we get there. We need to be able to move forward. On 2025/02/03 19:51:03 Jason Porter wrote: > I'm a little confused why we're calling a vote at the same time we're > submitting a new proposal. Is it to simply move things along? I doubt we'll > make everyone happy with this proposal, however it plays out, but this seems > like it should be more of a discussion thread instead of a vote until we've > addressed issues that are brought up. > > As far as they methodology for getting to releasable docs and website, I > honestly don't care which way we go, where we do the work, or how it gets > done as long it is a sane process. I don't want to have to check five > different places to make sure it works, submit three different pull requests > to related things, and hope we have a green CI build (which we rarely do) for > basic changes. However, that is the current way of doing things, even if I > personally think it is broken, it has been working (sort of) for a while. > > If we're going to move forward with this being a vote: +0 for me. > > On 2025/02/03 16:30:00 Alex Porcelli wrote: > > Given the recent discussions about Docs [1] and Examples [2], Tiago > > Bento and I got together to write, in proposal form, the combination > > of our understanding of how we can create the baseline for content > > (Docs, Examples, and Websites) so that our users can consume Apache > > KIE releases in an expected, structured, and cohesive way. This > > proposal will take you < 15 min to read. > > > > - > > Alex & Tiago > > > > # Problem scope > > > > There has been a significant demand for proper guidance on using > > Apache KIE 10.0. Users are unsure how to adopt it, as neither > > Documentation nor Examples were available when 10.0.0 was released. An > > attempt to manually adjust the examples to the 10.0.0 release has > > happened for a couple of weeks now, showing how time-consuming it is > > to defer dealing with “satellite” repositories after releases are > > done. Even more complicated is that well-known websites (drools.org, > > jbpm.org, kogito.kie.org) are outdated and out of sync with Apache > > KIE’s repositories and releases, thus contributing to more confusion > > than anything else. > > > > To avoid being unclear about why Docs, Examples, and Websites are > > crucial for the community's success, they are all entry points where > > new users can start understanding Apache KIE, how to use it and > > whether or not it’s a fit for them. For existing users, they are all > > reference material, including code snippets, institutional texts > > and/or vision, licenses, migration guides, and legal material. > > Examples, in particular, also work as playgrounds, where users can > > start with pre-built applications and make minor tweaks to either > > check if something they want is supported or simulate a development > > workflow on a smaller scale. Our code is worth close to nothing if > > there isn’t good supporting material around it. > > > > This proposal aims to amend all those problems by bringing Docs, > > Examples, and Websites to the Release Procedure, PR checks, and a > > Continuous deployment (CD) mechanism, making them all be part of our > > daily contributions, so they are not treated as afterthoughts. > > > > > > # What is not in scope > > > > - A complete restructuring of all Apache KIE repositories > > - Refactoring the whole CI; this proposal aims to work with the > > current established CI already in place > > - Refactor content in Docs and Example applications; this proposal > > aims to move stuff as-is, except when something is broken, then we’ll > > fix it. > > - Revisit all Examples to check or validate whether they are fully > > functional. This is an important review that must be done > > incrementally. > > - Retroactively adjust the 10.0.x branch to include Docs, Examples, > > and Websites. > > - Provide a new blogging framework for Apache KIE in replacement for > > https://blog.kie.org/ > > > > > > # The proposal > > > > - The Apache KIE website gains a new section called "Docs" in its > > header, where users are able to select what stream of the Docs they > > want to see, for example: "Docs" -> "10.1.x" -> "Drools"; or "Docs" -> > > "10.2.x" -> "SonataFlow" etc. > > - A new read-only repository called `incubator-kie-examples` is > > created, containing all examples aligned with development stream > > branches, and releases candidates and releases tags. > > - A new release artifact (Examples ZIP) starts existing, and its > > contents then published to the read-only `incubator-kie-examples` > > repository. > > - The existing `incubator-kogito-examples` repository is archived > > after 10.1 is first released. > > - A new Continuous Deployment (CD) mechanism is created to ensure Docs > > and Websites are kept current. > > > > > > # The method > > > > Leverage `kie-tools` flexible build system to reduce complexity hidden > > in Jenkins/GitHub Actions automations, which have proven to become > > unmaintained and abandoned, by moving content (preserving Git history) > > from all Docs-, Examples-, and Websites-related repositories into new > > packages on `kie-tools`. Here’s the final structure this proposal aims > > to create inside the `kie-tools` repository: > > > > - kie-tools/docs/drools > > - kie-tools/docs/optaplanner > > - kie-tools/docs/jbpm > > - kie-tools/docs/kogito > > - kie-tools/docs/sonataflow > > - kie-tools/docs/tools > > - kie-tools/websites/kie-apache-org > > - kie-tools/examples/rules-* > > - kie-tools/examples/decisions-* > > - kie-tools/examples/process-* > > - kie-tools/examples/sonataflow-* > > - kie-tools/examples/pmml-* > > - kie-tools/examples/trusty-* > > > > For each example package, Spring Boot, Quarkus, and plain Java > > subdivisions would be done inside each example using different Maven > > modules. This would allow users to focus on the domain rather than the > > framework/implementation and avoid content duplication in READMEs. > > > > Being inside `kie-tools`, Examples would be able to benefit from > > graphical Editors, Runtime Consoles, Quarkus Dev UIs, and Container > > images, all crucial components for end-to-end solutions built with > > Apache KIE. This is currently impossible in the current > > `kogito-examples` repository due to the way our PR checks and releases > > are structured/automated, where `kogito-examples` builds before > > `kie-tools`. Making `kogito-examples` build after `kie-tools` for PR > > checks is non-trivial and completely out-of-scope of this proposal, as > > it would mean doubling-down on the existing CI systems like > > “build-chain” for PR checks and “the Kogito framework” on Apache > > Jenkins, both inherently wasteful on cross-repo PR checks and provenly > > hard to maintain/operate. > > > > Contributions to the proposed new packages would be made following the > > same commands we already use for all other packages on `kie-tools`. > > Contributors would need to have `pnpm` and Node.js installed or rely > > on Devbox to install everything automatically. After setting up the > > environment, cloning the repo, and running `pnpm bootstrap -F > > '@kie-docs/drools’ && pnpm -F @kie-docs/drools start` contributors > > would be able to develop the Drools documentation with live-reload > > capabilities. The very same would be true for Website(s). > > > > For the Examples applications, the process after cloning would be: > > > > 1. `pnpm bootstrap -F @kie-examples/some-example-app...` to install > > 3rd party dependencies and link packages. > > 2. `pnpm -F @kie-examples/some-example-app^... build:dev` to build > > this example’s 1st party dependencies. > > 3. `pnpm -F @kie-examples/some-example-app start` to start the example > > in its default mode. > > > > Each example is also encouraged to define in their README how to start > > itself using technology-specific commands, like `mvn quarkus:dev > > -Psome-profile` or even `docker-compose up`. The `start` script is > > only the default way to run an example, a convention already cohesive > > with all other existing packages on `kie-tools`. > > > > It’s important to note again that the `pnpm` commands would not be > > necessary in the `incubator-kie-examples` repository and users would > > be able to run plain `mvn` commands directly inside the example’s > > directory, for Maven-based examples at least. > > > > > > #### Impact on PR checks (CI) > > > > Each new package will contain a `package.json` file describing scripts > > and their dependency relationship with other packages in the > > repository. The important scripts are `build:dev`, `build:prod`, and > > `start`, to integrate them with the existing build system. > > > > PRs changing exclusively Docs packages will NOT build anything else in > > the repository, making PR checks efficient. The same is true for > > websites. > > > > PRs changing example applications will only build their dependency > > tree and the modified example applications, excluding unrelated > > packages from the checks while enforcing correctness. This is already > > how `kie-tools` works. > > > > If you need more information on how this mechanism works in practice, > > please come forward with clarifying questions. We’d love to show you > > how the partial and partitioned PR checks work on `kie-tools`. > > > > > > #### Impact on Release Process (“CI”) > > > > Docs and Websites wouldn’t be release artifacts and would be handled > > by a new “Continuous deployment (CD)” mechanism. This is covered in > > the next section of this proposal. > > > > With that said, only one new release artifact is being proposed here. > > A ZIP file containing all examples, stripping out all > > `kie-tools`-related boilerplate used to develop them. This new > > artifact (Examples ZIP) would be published to a new read-only > > repository (incubator-kie-examples) for all release candidates and > > releases we do. This repository would be THE consumable repository for > > users who wish to go straight to the point when testing out Apache KIE > > and who are not necessarily committers/developers. > > > > To do that, changing some release automations to add a new job that > > builds and publishes the Examples ZIP would be necessary: > > > > 1. Daily-dev > > (https://ci-builds.apache.org/job/KIE/job/kie-tools/job/main/job/kie-tools-daily-dev-publish/) > > 2. Release Candidates > > (https://ci-builds.apache.org/job/KIE/job/kie-tools/job/main/job/release/job/kie-tools-release-candidate/) > > 3. Release Promote > > (https://ci-builds.apache.org/job/KIE/job/kie-tools/job/main/job/release/job/kie-tools-release-promote/ > > ) > > > > > > #### New Continuous Deployment (CD) > > > > Websites and Docs would be treated as continually deployed packages > > for the development stream we declare as “live”. Also, old development > > streams would be continually built and published to make documentation > > updates possible for older versions. Referring to Toshiya's thread on > > documentation [1], our Docs would be “mutable”, and the granularity > > would be “per stream”. > > > > Two new automations would be created for making sure our content is > > up-to-date at all times: > > > > `publish-kie-apache-org-website`: Responsible for updating live Apache > > KIE’s website (https://kie.apache.org) and also capable of generating > > ad-hoc immutable copies of the website for Release Candidates. It > > would be able to also be executed for the `main` stream for > > development purposes. > > > > `publish-older-docs-to-live-website` ensures that older development > > streams can have their content updated on the live website. For > > example, if we find a seriously incorrect statement in an older > > version, this job would ensure it’s published after being fixed. We > > can eventually remove this for versions that become too old. > > > > > > # Actionable items (plan) > > > > Phase 1 (Development and contributions): > > > > - Move all examples to `kie-tools`, already in the new > > structure/organization, and integrate them as part of the build system > > - Move all documentation content to `kie-tools` and integrate the > > new packages as part of the build system > > - Move `incubator-kie-website` content to `kie-tools` and integrate > > it as part of the build system > > - Create new landing pages for each major component: Drools, jBPM, > > SonataFlow, Optaplanner, and Kogito. > > > > Phase 2 (Release Process) > > > > - Create the `incubator-kie-examples` repository > > - Update `kie-tools` daily-dev job to publish the Examples ZIP to > > `incubator-kie-examples@main` > > - Update `kie-tools` Release Candidate job to publish the Examples > > ZIP to https://dist.apache.org/repos/dist/dev/incubator/kie/ > > - Update `kie-tools` Release Promote job to publish the Examples ZIP > > content to incubator-kie-examples@{version}` > > - Archive the `incubator-kie-kogito-examples` repository (post 10.1 > > release) > > - Update the Release Procedure for 10.1 to include Docs, Examples, > > and Websites where necessary > > > > Phase 3 (Continuous deployments) > > > > - Create and configure new `publish-kie-apache-org-website` job > > - Create and configure new `publish-docs-to-live-website` job > > > > Phase 4 (Cleanup) > > > > - Depublish current `drools.org`, `jbpm.org`, `optaplanner.org`, > > `kie.org`, `sonataflow.org` and all its subdomains. > > - Redirect all old domains to the proper landing page of Apache KIE’s > > widely known components (E.g., https://drools.org → > > https://kie.apache.org/drools) > > - Preserve Docs from older versions somewhere accessible under Apache > > KIE’s website. > > > > > > # Commitment and rough timeline > > > > Alex Porcelli and Tiago Bento are fully committed to executing this > > plan in collaboration with other members who want to help. > > > > We are committed to working on it as soon as the proposal is approved. > > Although it’s hard to predict how long it will take, Phases 1, 2, and > > 3 will take around 6 weeks based on the information we have today. If > > anything changes or issues are found during execution, we can update > > this thread. > > > > Phase 4 is impossible to estimate, as there might be many different > > people and/or organizations to chase so that these websites can be > > decommissioned and properly forward users to KIE’s new home in Apache. > > > > > > # Observations on `kie-tools` > > > > This proposal makes Docs, Examples, and Websites available to all > > users without touching the complex, unmaintained, black-box systems > > currently in Apache’s Jenkins, known as “build-chain” and “the Kogito > > CI framework”. > > > > We know that `kie-tools` is not unanimously popular among Apache KIE > > committers, but it’s actively maintained and has been proven to scale > > well both in package count and technology diversity. We invite > > everyone to take a look at the repo’s User Manual at > > https://github.com/apache/incubator-kie-tools/blob/main/repo/MANUAL.md > > > > > > ## Non-obvious positive outcomes > > > > - Drools and OptaPlanner repositories won’t have dependencies on > > JavaScript anymore. > > - Fonts and other “binary” files won’t need to be committed as a > > source anymore, as `kie-tools`’s build system can properly depend on > > NPM artifacts. Also helps towards graduation, as Category X > > dependencies won’t be present. > > > > > > [1] https://lists.apache.org/thread/zjt156tk5zjw7463xsfdkojwzndwd1mn > > [2] https://lists.apache.org/thread/qc9vhod96mdoppo5ssj4f0pkqhzt4ghd > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > > For additional commands, e-mail: dev-h...@kie.apache.org > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > For additional commands, e-mail: dev-h...@kie.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org For additional commands, e-mail: dev-h...@kie.apache.org
