-0 What I lack in the proposal is how the move of examples into kie-tools resolves the need for running cross-repository PRs or how to solve this problem WITHOUT build-chain, which is positioned as a selling point for the examples move (apart for solving the images-related gaps the pipelines have).
If the answer reads that kogito-examples is not meant to test the repositories up the chain then I won't agree on that one, albeit current shape is only checking compilation rather than testing, but at least that (side note we could consider testing the actual readme commands, whole different story, e.g. https://github.com/orangemug/readme-tester/ ). Also there are incorrect assumptions about the reason for build-chain existence in the first place - take a look at the repository name https://github.com/kiegroup/github-action-build-chain - this is primarily a GitHub action implemented to enable cross-repository executions (for PRs, branch builds, full downstream, etc) in GitHub Actions, jenkins pipelines would easily live without the tool, though goal was perhaps to unify the means. It also allows local execution in the same way that the CI does, so pnpm vs build-chain when it comes to ease of build locally, they seem similar to me, except for the single/distributed location of code. As proposal states: <cite> Making `kogito-examples` build after `kie-tools` for PR checks is non-trivial and completely out-of-scope of this proposal, as it would mean doubling-down on the existing CI systems like “build-chain” for PR checks and “the Kogito framework” on Apache Jenkins, both inherently wasteful on cross-repo PR checks and provenly hard to maintain/operate. </cite> Which needs more explanation on how we serve the cross-repository PR builds when a change in kogito-runtimes requires a change in kie-tools examples, and that without the build-chain. I guess the response leads to weekly SNAPSHOT, which is on its own an artificial construct we invented for kie-tools. This way the examples inevitably end as just a project depending on drools and kogito of a pre-defined version, not a part of the environment as such. kogito-pipelines CI framework assessment does not seem to be strictly tied to the given topic, that's a "pipelines as code" framework, which has its pros and cons, none of which were actually mentioned as part of the proposal. Not entirely sure why it's covered in the text while CI refactoring is stated as out of scope. I am happy to share more on the topic as time allows, though I can't commit to any counter-proposal in the short term. Regards Jan On Tue, 4 Feb 2025 at 13:56, Daniel José dos Santos <djdossan...@live.com> wrote: > +1 > > -----Mensagem original----- > De: Eder Ignatowicz <ignatow...@gmail.com> > Enviada em: segunda-feira, 3 de fevereiro de 2025 13:53 > Para: dev@kie.apache.org > Assunto: Re: [PROPOSAL][VOTE] Including Docs, Examples, and Website(s) in > Apache KIE 10.1 and beyond > > +1 > _____________ > Eder Ignatowicz > ignatow...@gmail.com > > > On Mon, Feb 3, 2025 at 11:52 AM Thiago Lugli <thiago...@apache.org> wrote: > > > +1 > > > > Em seg., 3 de fev. de 2025 às 13:44, Tiago Bento > > <tiagobe...@apache.org> escreveu: > > > > > > +1 > > > > > > On Mon, Feb 3, 2025 at 11:39 AM Gabriele Cardosi > > > <gabriele.card...@gmail.com> wrote: > > > > > > > > -1 > > > > > > > > > > > > Il giorno lun 3 feb 2025 alle ore 17:37 Eduardo Cerqueira < > > > > eduardocerque...@apache.org> ha scritto: > > > > > > > > > +1 > > > > > > > > > > On Mon, Feb 3, 2025 at 11:31 AM Alex Porcelli > > > > > <porce...@apache.org> > > wrote: > > > > > > > > > > > Given the recent discussions about Docs [1] and Examples [2], > > > > > > Tiago Bento and I got together to write, in proposal form, the > > combination > > > > > > of our understanding of how we can create the baseline for > > > > > > content (Docs, Examples, and Websites) so that our users can > > > > > > consume Apache KIE releases in an expected, structured, and > > > > > > cohesive way. This proposal will take you < 15 min to read. > > > > > > > > > > > > - > > > > > > Alex & Tiago > > > > > > > > > > > > # Problem scope > > > > > > > > > > > > There has been a significant demand for proper guidance on > > > > > > using Apache KIE 10.0. Users are unsure how to adopt it, as > > > > > > neither Documentation nor Examples were available when 10.0.0 > > > > > > was > > released. An > > > > > > attempt to manually adjust the examples to the 10.0.0 release > > > > > > has happened for a couple of weeks now, showing how > > > > > > time-consuming it > > is > > > > > > to defer dealing with “satellite” repositories after releases > > > > > > are done. Even more complicated is that well-known websites ( > > drools.org, > > > > > > jbpm.org, kogito.kie.org) are outdated and out of sync with > > > > > > Apache KIE’s repositories and releases, thus contributing to > > > > > > more > > confusion > > > > > > than anything else. > > > > > > > > > > > > To avoid being unclear about why Docs, Examples, and Websites > > > > > > are crucial for the community's success, they are all entry > > > > > > points > > where > > > > > > new users can start understanding Apache KIE, how to use it > > > > > > and whether or not it’s a fit for them. For existing users, > > > > > > they are > > all > > > > > > reference material, including code snippets, institutional > > > > > > texts and/or vision, licenses, migration guides, and legal > material. > > > > > > Examples, in particular, also work as playgrounds, where users > > > > > > can start with pre-built applications and make minor tweaks to > > > > > > either check if something they want is supported or simulate a > > > > > > development workflow on a smaller scale. Our code is worth > > > > > > close to nothing if there isn’t good supporting material around > it. > > > > > > > > > > > > This proposal aims to amend all those problems by bringing > > > > > > Docs, Examples, and Websites to the Release Procedure, PR > > > > > > checks, and a Continuous deployment (CD) mechanism, making > > > > > > them all be part of > > our > > > > > > daily contributions, so they are not treated as afterthoughts. > > > > > > > > > > > > > > > > > > # What is not in scope > > > > > > > > > > > > - A complete restructuring of all Apache KIE repositories > > > > > > - Refactoring the whole CI; this proposal aims to work with > > > > > > the current established CI already in place > > > > > > - Refactor content in Docs and Example applications; this > > > > > > proposal aims to move stuff as-is, except when something is > > > > > > broken, then > > we’ll > > > > > > fix it. > > > > > > - Revisit all Examples to check or validate whether they are > > > > > > fully functional. This is an important review that must be > > > > > > done incrementally. > > > > > > - Retroactively adjust the 10.0.x branch to include Docs, > > > > > > Examples, and Websites. > > > > > > - Provide a new blogging framework for Apache KIE in > > > > > > replacement > > for > > > > > > https://blog.kie.org/ > > > > > > > > > > > > > > > > > > # The proposal > > > > > > > > > > > > - The Apache KIE website gains a new section called "Docs" in > > > > > > its header, where users are able to select what stream of the > > > > > > Docs they want to see, for example: "Docs" -> "10.1.x" -> > > > > > > "Drools"; or > > "Docs" -> > > > > > > "10.2.x" -> "SonataFlow" etc. > > > > > > - A new read-only repository called `incubator-kie-examples` > > > > > > is created, containing all examples aligned with development > > > > > > stream branches, and releases candidates and releases tags. > > > > > > - A new release artifact (Examples ZIP) starts existing, and > > > > > > its contents then published to the read-only > > > > > > `incubator-kie-examples` repository. > > > > > > - The existing `incubator-kogito-examples` repository is > > > > > > archived after 10.1 is first released. > > > > > > - A new Continuous Deployment (CD) mechanism is created to > > > > > > ensure > > Docs > > > > > > and Websites are kept current. > > > > > > > > > > > > > > > > > > # The method > > > > > > > > > > > > Leverage `kie-tools` flexible build system to reduce > > > > > > complexity > > hidden > > > > > > in Jenkins/GitHub Actions automations, which have proven to > > > > > > become unmaintained and abandoned, by moving content > > > > > > (preserving Git > > history) > > > > > > from all Docs-, Examples-, and Websites-related repositories > > > > > > into > > new > > > > > > packages on `kie-tools`. Here’s the final structure this > > > > > > proposal > > aims > > > > > > to create inside the `kie-tools` repository: > > > > > > > > > > > > - kie-tools/docs/drools > > > > > > - kie-tools/docs/optaplanner > > > > > > - kie-tools/docs/jbpm > > > > > > - kie-tools/docs/kogito > > > > > > - kie-tools/docs/sonataflow > > > > > > - kie-tools/docs/tools > > > > > > - kie-tools/websites/kie-apache-org > > > > > > - kie-tools/examples/rules-* > > > > > > - kie-tools/examples/decisions-* > > > > > > - kie-tools/examples/process-* > > > > > > - kie-tools/examples/sonataflow-* > > > > > > - kie-tools/examples/pmml-* > > > > > > - kie-tools/examples/trusty-* > > > > > > > > > > > > For each example package, Spring Boot, Quarkus, and plain Java > > > > > > subdivisions would be done inside each example using different > > Maven > > > > > > modules. This would allow users to focus on the domain rather > > > > > > than > > the > > > > > > framework/implementation and avoid content duplication in > READMEs. > > > > > > > > > > > > Being inside `kie-tools`, Examples would be able to benefit > > > > > > from graphical Editors, Runtime Consoles, Quarkus Dev UIs, and > > > > > > Container images, all crucial components for end-to-end > > > > > > solutions built with Apache KIE. This is currently impossible > > > > > > in the current `kogito-examples` repository due to the way our > > > > > > PR checks and > > releases > > > > > > are structured/automated, where `kogito-examples` builds > > > > > > before `kie-tools`. Making `kogito-examples` build after > > > > > > `kie-tools` for > > PR > > > > > > checks is non-trivial and completely out-of-scope of this > > proposal, as > > > > > > it would mean doubling-down on the existing CI systems like > > > > > > “build-chain” for PR checks and “the Kogito framework” on > > > > > > Apache Jenkins, both inherently wasteful on cross-repo PR > > > > > > checks and > > provenly > > > > > > hard to maintain/operate. > > > > > > > > > > > > Contributions to the proposed new packages would be made > > > > > > following > > the > > > > > > same commands we already use for all other packages on > `kie-tools`. > > > > > > Contributors would need to have `pnpm` and Node.js installed > > > > > > or > > rely > > > > > > on Devbox to install everything automatically. After setting > > > > > > up the environment, cloning the repo, and running `pnpm > > > > > > bootstrap -F '@kie-docs/drools’ && pnpm -F @kie-docs/drools > > > > > > start` contributors would be able to develop the Drools > > > > > > documentation with live-reload capabilities. The very same would > be true for Website(s). > > > > > > > > > > > > For the Examples applications, the process after cloning would > be: > > > > > > > > > > > > 1. `pnpm bootstrap -F @kie-examples/some-example-app...` to > > > > > > install 3rd party dependencies and link packages. > > > > > > 2. `pnpm -F @kie-examples/some-example-app^... build:dev` to > > > > > > build this example’s 1st party dependencies. > > > > > > 3. `pnpm -F @kie-examples/some-example-app start` to start the > > example > > > > > > in its default mode. > > > > > > > > > > > > Each example is also encouraged to define in their README how > > > > > > to > > start > > > > > > itself using technology-specific commands, like `mvn > > > > > > quarkus:dev -Psome-profile` or even `docker-compose up`. The > > > > > > `start` script is only the default way to run an example, a > > > > > > convention already > > cohesive > > > > > > with all other existing packages on `kie-tools`. > > > > > > > > > > > > It’s important to note again that the `pnpm` commands would > > > > > > not be necessary in the `incubator-kie-examples` repository > > > > > > and users > > would > > > > > > be able to run plain `mvn` commands directly inside the > > > > > > example’s directory, for Maven-based examples at least. > > > > > > > > > > > > > > > > > > #### Impact on PR checks (CI) > > > > > > > > > > > > Each new package will contain a `package.json` file describing > > scripts > > > > > > and their dependency relationship with other packages in the > > > > > > repository. The important scripts are `build:dev`, > > > > > > `build:prod`, > > and > > > > > > `start`, to integrate them with the existing build system. > > > > > > > > > > > > PRs changing exclusively Docs packages will NOT build anything > > else in > > > > > > the repository, making PR checks efficient. The same is true > > > > > > for websites. > > > > > > > > > > > > PRs changing example applications will only build their > > > > > > dependency tree and the modified example applications, > > > > > > excluding unrelated packages from the checks while enforcing > > > > > > correctness. This is > > already > > > > > > how `kie-tools` works. > > > > > > > > > > > > If you need more information on how this mechanism works in > > practice, > > > > > > please come forward with clarifying questions. We’d love to > > > > > > show > > you > > > > > > how the partial and partitioned PR checks work on `kie-tools`. > > > > > > > > > > > > > > > > > > #### Impact on Release Process (“CI”) > > > > > > > > > > > > Docs and Websites wouldn’t be release artifacts and would be > > handled > > > > > > by a new “Continuous deployment (CD)” mechanism. This is > > > > > > covered in the next section of this proposal. > > > > > > > > > > > > With that said, only one new release artifact is being > > > > > > proposed > > here. > > > > > > A ZIP file containing all examples, stripping out all > > > > > > `kie-tools`-related boilerplate used to develop them. This new > > > > > > artifact (Examples ZIP) would be published to a new read-only > > > > > > repository (incubator-kie-examples) for all release candidates > > > > > > and releases we do. This repository would be THE consumable > > > > > > repository > > for > > > > > > users who wish to go straight to the point when testing out > > > > > > Apache > > KIE > > > > > > and who are not necessarily committers/developers. > > > > > > > > > > > > To do that, changing some release automations to add a new job > > > > > > that builds and publishes the Examples ZIP would be necessary: > > > > > > > > > > > > 1. Daily-dev ( > > > > > > > > > > > > > https://ci-builds.apache.org/job/KIE/job/kie-tools/job/main/job/kie-to > > ols-daily-dev-publish/ > > > > > > ) > > > > > > 2. Release Candidates > > > > > > ( > > > > > > > > > > > > > https://ci-builds.apache.org/job/KIE/job/kie-tools/job/main/job/releas > > e/job/kie-tools-release-candidate/ > > > > > > ) > > > > > > 3. Release Promote > > > > > > ( > > > > > > > > > > > > > https://ci-builds.apache.org/job/KIE/job/kie-tools/job/main/job/releas > > e/job/kie-tools-release-promote/ > > > > > > ) > > > > > > > > > > > > > > > > > > #### New Continuous Deployment (CD) > > > > > > > > > > > > Websites and Docs would be treated as continually deployed > > > > > > packages for the development stream we declare as “live”. > > > > > > Also, old > > development > > > > > > streams would be continually built and published to make > > documentation > > > > > > updates possible for older versions. Referring to Toshiya's > > > > > > thread > > on > > > > > > documentation [1], our Docs would be “mutable”, and the > > > > > > granularity would be “per stream”. > > > > > > > > > > > > Two new automations would be created for making sure our > > > > > > content is up-to-date at all times: > > > > > > > > > > > > `publish-kie-apache-org-website`: Responsible for updating > > > > > > live > > Apache > > > > > > KIE’s website (https://kie.apache.org) and also capable of > > generating > > > > > > ad-hoc immutable copies of the website for Release Candidates. > > > > > > It would be able to also be executed for the `main` stream for > > > > > > development purposes. > > > > > > > > > > > > `publish-older-docs-to-live-website` ensures that older > > > > > > development streams can have their content updated on the live > > > > > > website. For example, if we find a seriously incorrect > > > > > > statement in an older version, this job would ensure it’s > > > > > > published after being fixed. We can eventually remove this for > versions that become too old. > > > > > > > > > > > > > > > > > > # Actionable items (plan) > > > > > > > > > > > > Phase 1 (Development and contributions): > > > > > > > > > > > > - Move all examples to `kie-tools`, already in the new > > > > > > structure/organization, and integrate them as part of the > > > > > > build > > system > > > > > > - Move all documentation content to `kie-tools` and > > > > > > integrate the new packages as part of the build system > > > > > > - Move `incubator-kie-website` content to `kie-tools` and > > integrate > > > > > > it as part of the build system > > > > > > - Create new landing pages for each major component: Drools, > > > > > > jBPM, SonataFlow, Optaplanner, and Kogito. > > > > > > > > > > > > Phase 2 (Release Process) > > > > > > > > > > > > - Create the `incubator-kie-examples` repository > > > > > > - Update `kie-tools` daily-dev job to publish the Examples > > > > > > ZIP to `incubator-kie-examples@main` > > > > > > - Update `kie-tools` Release Candidate job to publish the > > > > > > Examples ZIP to > > > > > > https://dist.apache.org/repos/dist/dev/incubator/kie/ > > > > > > - Update `kie-tools` Release Promote job to publish the > > > > > > Examples > > ZIP > > > > > > content to incubator-kie-examples@{version}` > > > > > > - Archive the `incubator-kie-kogito-examples` repository > > > > > > (post > > 10.1 > > > > > > release) > > > > > > - Update the Release Procedure for 10.1 to include Docs, > > > > > > Examples, and Websites where necessary > > > > > > > > > > > > Phase 3 (Continuous deployments) > > > > > > > > > > > > - Create and configure new `publish-kie-apache-org-website` > > > > > > job > > > > > > - Create and configure new `publish-docs-to-live-website` job > > > > > > > > > > > > Phase 4 (Cleanup) > > > > > > > > > > > > - Depublish current `drools.org`, `jbpm.org`, > > > > > > `optaplanner.org`, `kie.org`, `sonataflow.org` and all its > subdomains. > > > > > > - Redirect all old domains to the proper landing page of > > > > > > Apache > > KIE’s > > > > > > widely known components (E.g., https://drools.org → > > > > > > https://kie.apache.org/drools) > > > > > > - Preserve Docs from older versions somewhere accessible under > > Apache > > > > > > KIE’s website. > > > > > > > > > > > > > > > > > > # Commitment and rough timeline > > > > > > > > > > > > Alex Porcelli and Tiago Bento are fully committed to executing > > > > > > this plan in collaboration with other members who want to help. > > > > > > > > > > > > We are committed to working on it as soon as the proposal is > > approved. > > > > > > Although it’s hard to predict how long it will take, Phases 1, > > > > > > 2, > > and > > > > > > 3 will take around 6 weeks based on the information we have > today. > > If > > > > > > anything changes or issues are found during execution, we can > > update > > > > > > this thread. > > > > > > > > > > > > Phase 4 is impossible to estimate, as there might be many > > > > > > different people and/or organizations to chase so that these > > > > > > websites can be decommissioned and properly forward users to > > > > > > KIE’s new home in > > Apache. > > > > > > > > > > > > > > > > > > # Observations on `kie-tools` > > > > > > > > > > > > This proposal makes Docs, Examples, and Websites available to > > > > > > all users without touching the complex, unmaintained, > > > > > > black-box systems currently in Apache’s Jenkins, known as > > > > > > “build-chain” and “the > > Kogito > > > > > > CI framework”. > > > > > > > > > > > > We know that `kie-tools` is not unanimously popular among > > > > > > Apache > > KIE > > > > > > committers, but it’s actively maintained and has been proven > > > > > > to > > scale > > > > > > well both in package count and technology diversity. We invite > > > > > > everyone to take a look at the repo’s User Manual at > > > > > > > > https://github.com/apache/incubator-kie-tools/blob/main/repo/MANUAL.md > > > > > > > > > > > > > > > > > > ## Non-obvious positive outcomes > > > > > > > > > > > > - Drools and OptaPlanner repositories won’t have dependencies > > > > > > on JavaScript anymore. > > > > > > - Fonts and other “binary” files won’t need to be committed as > > > > > > a source anymore, as `kie-tools`’s build system can properly > > > > > > depend > > on > > > > > > NPM artifacts. Also helps towards graduation, as Category X > > > > > > dependencies won’t be present. > > > > > > > > > > > > > > > > > > [1] > > https://lists.apache.org/thread/zjt156tk5zjw7463xsfdkojwzndwd1mn > > > > > > [2] > > https://lists.apache.org/thread/qc9vhod96mdoppo5ssj4f0pkqhzt4ghd > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org For > > > > > > additional commands, e-mail: dev-h...@kie.apache.org > > > > > > > > > > > > > > > > > > > > > > > -------------------------------------------------------------------- > > > - To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org For > > > additional commands, e-mail: dev-h...@kie.apache.org > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org For additional > > commands, e-mail: dev-h...@kie.apache.org > > > > >
