Hi Francisco! Just a few remarks regarding what you said:
> First, I would like to remark, after reading some of the comments from > committers who voted +1, that outdated examples are not necessarily easier > to update just because they are moved to a different repository (the > proposal, rightly, does not claim to fix this). I tend to disagree. Currently, the examples are in an isolated repository. If the new repository has more visibility, attracts more contributions, and has more people working on it, I’m inclined to think the examples will also gain more visibility, but that’s just my opinion. As I said, when I had to tweak some of the examples, some process examples were broken or very outdated, which made me feel like no one was looking at or trying them (but that was just my impression). I even created a couple of issues back then [1][2]. One fact that should be remarked, in the direction that Enrique argued, is > that if you want to build your examples using any existing IDE, it is far > easier to do that if the examples are in their own isolated repo (because > you can basically import the whole repo) while, if they are mixed with > other code within the same repo, you need to start navigating to some > subdirectory to start importing them. >From what I understood, we would still have an examples repository, but it would only have the *released *examples, which anyone would be able to clone and use without any additional dependencies/technology. Meanwhile, the examples under development would be in the kie-tools. [1] https://github.com/apache/incubator-kie-issues/issues/1702 [2] https://github.com/apache/incubator-kie-issues/issues/1703 On Mon, 3 Feb 2025 at 16:40, Francisco Javier Tirado Sarti < ftira...@redhat.com> wrote: > I forgot to argue the -1. > First I would like to remark, after reading some of the comments of > comitters that voted +1, that outdated examples are not easier to be > updated because they are moved to a different repository (the proposal, > rightly, does not claim to fix this) > One fact that should be remarked, in the direction that Enrique argued, is > that if you want to build your examples using any existing IDE, it is far > easier to do that if the examples are in their own isolated repo (because > you can basically import the whole repo) while, if they are mixed with > other code within the same repo, you need to start navigating to some > subdirectory to start importing them. > Actually, I think this is a pretty important usability question, to avoid > having multiple potential IDE workspaces over the same repo. I have the > suspicion we are breaking an important implicit isolation design principle > here. > So, in my opinion, if the examples are now outdated, if this proposal goes > through, they will be even more outdated examples after the movement, > because many developers will be tempted to save the extra work of setting > up the IDE to prevent them becoming outdated. > In the same line of thought, please note that currently example repo CI is > integrated with runtimes and apps repo CIs, so if examples are getting > outdated now, it is probably because some integration test might be > enhanced (and this automatic testing enhancements is probably what we > should be working on, if we knew what the problem with the outdated example > was). If this tight CI integration is removed (and I hardly see how, if > lack of CI resources is argued to push this proposal, this is going to be > kept) they will become even more outdated. > So, it is not difficult to predict that, if this proposal goes through, > whenever the problem with examples is right now (again, I would like to see > evidence about what those issues are) it will become worse. > About docs, I honestly cannot understand why they are even mentioned in > this proposal. But my counter proposal is simple, put all docs into a > separate repo, organized in a similar way as Gabriele proposed in a > different thread (by "engine": rules, processes: bpmn and sonataflow, apps: > data index and audit, tools...) > > > > On Mon, Feb 3, 2025 at 6:57 PM Rodrigo Antunes <rantu...@apache.org> > wrote: > > > +1 > > > > On 2025/02/03 17:51:42 Luiz Motta wrote: > > > +1 > > > > > > Thanks, Alex and Tiago, to coming up with this proposal! Recently, I > > updated some examples in the `kogito-examples` repository and noticed > that > > the ones I touched were very outdated, with some even broken. I believe > > this proposal will improve their maintainability! > > > > > > On 2025/02/03 16:30:00 Alex Porcelli wrote: > > > > Given the recent discussions about Docs [1] and Examples [2], Tiago > > > > Bento and I got together to write, in proposal form, the combination > > > > of our understanding of how we can create the baseline for content > > > > (Docs, Examples, and Websites) so that our users can consume Apache > > > > KIE releases in an expected, structured, and cohesive way. This > > > > proposal will take you < 15 min to read. > > > > > > > > - > > > > Alex & Tiago > > > > > > > > # Problem scope > > > > > > > > There has been a significant demand for proper guidance on using > > > > Apache KIE 10.0. Users are unsure how to adopt it, as neither > > > > Documentation nor Examples were available when 10.0.0 was released. > An > > > > attempt to manually adjust the examples to the 10.0.0 release has > > > > happened for a couple of weeks now, showing how time-consuming it is > > > > to defer dealing with “satellite” repositories after releases are > > > > done. Even more complicated is that well-known websites (drools.org, > > > > jbpm.org, kogito.kie.org) are outdated and out of sync with Apache > > > > KIE’s repositories and releases, thus contributing to more confusion > > > > than anything else. > > > > > > > > To avoid being unclear about why Docs, Examples, and Websites are > > > > crucial for the community's success, they are all entry points where > > > > new users can start understanding Apache KIE, how to use it and > > > > whether or not it’s a fit for them. For existing users, they are all > > > > reference material, including code snippets, institutional texts > > > > and/or vision, licenses, migration guides, and legal material. > > > > Examples, in particular, also work as playgrounds, where users can > > > > start with pre-built applications and make minor tweaks to either > > > > check if something they want is supported or simulate a development > > > > workflow on a smaller scale. Our code is worth close to nothing if > > > > there isn’t good supporting material around it. > > > > > > > > This proposal aims to amend all those problems by bringing Docs, > > > > Examples, and Websites to the Release Procedure, PR checks, and a > > > > Continuous deployment (CD) mechanism, making them all be part of our > > > > daily contributions, so they are not treated as afterthoughts. > > > > > > > > > > > > # What is not in scope > > > > > > > > - A complete restructuring of all Apache KIE repositories > > > > - Refactoring the whole CI; this proposal aims to work with the > > > > current established CI already in place > > > > - Refactor content in Docs and Example applications; this proposal > > > > aims to move stuff as-is, except when something is broken, then we’ll > > > > fix it. > > > > - Revisit all Examples to check or validate whether they are fully > > > > functional. This is an important review that must be done > > > > incrementally. > > > > - Retroactively adjust the 10.0.x branch to include Docs, Examples, > > > > and Websites. > > > > - Provide a new blogging framework for Apache KIE in replacement for > > > > https://blog.kie.org/ > > > > > > > > > > > > # The proposal > > > > > > > > - The Apache KIE website gains a new section called "Docs" in its > > > > header, where users are able to select what stream of the Docs they > > > > want to see, for example: "Docs" -> "10.1.x" -> "Drools"; or "Docs" > -> > > > > "10.2.x" -> "SonataFlow" etc. > > > > - A new read-only repository called `incubator-kie-examples` is > > > > created, containing all examples aligned with development stream > > > > branches, and releases candidates and releases tags. > > > > - A new release artifact (Examples ZIP) starts existing, and its > > > > contents then published to the read-only `incubator-kie-examples` > > > > repository. > > > > - The existing `incubator-kogito-examples` repository is archived > > > > after 10.1 is first released. > > > > - A new Continuous Deployment (CD) mechanism is created to ensure > Docs > > > > and Websites are kept current. > > > > > > > > > > > > # The method > > > > > > > > Leverage `kie-tools` flexible build system to reduce complexity > hidden > > > > in Jenkins/GitHub Actions automations, which have proven to become > > > > unmaintained and abandoned, by moving content (preserving Git > history) > > > > from all Docs-, Examples-, and Websites-related repositories into new > > > > packages on `kie-tools`. Here’s the final structure this proposal > aims > > > > to create inside the `kie-tools` repository: > > > > > > > > - kie-tools/docs/drools > > > > - kie-tools/docs/optaplanner > > > > - kie-tools/docs/jbpm > > > > - kie-tools/docs/kogito > > > > - kie-tools/docs/sonataflow > > > > - kie-tools/docs/tools > > > > - kie-tools/websites/kie-apache-org > > > > - kie-tools/examples/rules-* > > > > - kie-tools/examples/decisions-* > > > > - kie-tools/examples/process-* > > > > - kie-tools/examples/sonataflow-* > > > > - kie-tools/examples/pmml-* > > > > - kie-tools/examples/trusty-* > > > > > > > > For each example package, Spring Boot, Quarkus, and plain Java > > > > subdivisions would be done inside each example using different Maven > > > > modules. This would allow users to focus on the domain rather than > the > > > > framework/implementation and avoid content duplication in READMEs. > > > > > > > > Being inside `kie-tools`, Examples would be able to benefit from > > > > graphical Editors, Runtime Consoles, Quarkus Dev UIs, and Container > > > > images, all crucial components for end-to-end solutions built with > > > > Apache KIE. This is currently impossible in the current > > > > `kogito-examples` repository due to the way our PR checks and > releases > > > > are structured/automated, where `kogito-examples` builds before > > > > `kie-tools`. Making `kogito-examples` build after `kie-tools` for PR > > > > checks is non-trivial and completely out-of-scope of this proposal, > as > > > > it would mean doubling-down on the existing CI systems like > > > > “build-chain” for PR checks and “the Kogito framework” on Apache > > > > Jenkins, both inherently wasteful on cross-repo PR checks and > provenly > > > > hard to maintain/operate. > > > > > > > > Contributions to the proposed new packages would be made following > the > > > > same commands we already use for all other packages on `kie-tools`. > > > > Contributors would need to have `pnpm` and Node.js installed or rely > > > > on Devbox to install everything automatically. After setting up the > > > > environment, cloning the repo, and running `pnpm bootstrap -F > > > > '@kie-docs/drools’ && pnpm -F @kie-docs/drools start` contributors > > > > would be able to develop the Drools documentation with live-reload > > > > capabilities. The very same would be true for Website(s). > > > > > > > > For the Examples applications, the process after cloning would be: > > > > > > > > 1. `pnpm bootstrap -F @kie-examples/some-example-app...` to install > > > > 3rd party dependencies and link packages. > > > > 2. `pnpm -F @kie-examples/some-example-app^... build:dev` to build > > > > this example’s 1st party dependencies. > > > > 3. `pnpm -F @kie-examples/some-example-app start` to start the > example > > > > in its default mode. > > > > > > > > Each example is also encouraged to define in their README how to > start > > > > itself using technology-specific commands, like `mvn quarkus:dev > > > > -Psome-profile` or even `docker-compose up`. The `start` script is > > > > only the default way to run an example, a convention already cohesive > > > > with all other existing packages on `kie-tools`. > > > > > > > > It’s important to note again that the `pnpm` commands would not be > > > > necessary in the `incubator-kie-examples` repository and users would > > > > be able to run plain `mvn` commands directly inside the example’s > > > > directory, for Maven-based examples at least. > > > > > > > > > > > > #### Impact on PR checks (CI) > > > > > > > > Each new package will contain a `package.json` file describing > scripts > > > > and their dependency relationship with other packages in the > > > > repository. The important scripts are `build:dev`, `build:prod`, and > > > > `start`, to integrate them with the existing build system. > > > > > > > > PRs changing exclusively Docs packages will NOT build anything else > in > > > > the repository, making PR checks efficient. The same is true for > > > > websites. > > > > > > > > PRs changing example applications will only build their dependency > > > > tree and the modified example applications, excluding unrelated > > > > packages from the checks while enforcing correctness. This is already > > > > how `kie-tools` works. > > > > > > > > If you need more information on how this mechanism works in practice, > > > > please come forward with clarifying questions. We’d love to show you > > > > how the partial and partitioned PR checks work on `kie-tools`. > > > > > > > > > > > > #### Impact on Release Process (“CI”) > > > > > > > > Docs and Websites wouldn’t be release artifacts and would be handled > > > > by a new “Continuous deployment (CD)” mechanism. This is covered in > > > > the next section of this proposal. > > > > > > > > With that said, only one new release artifact is being proposed here. > > > > A ZIP file containing all examples, stripping out all > > > > `kie-tools`-related boilerplate used to develop them. This new > > > > artifact (Examples ZIP) would be published to a new read-only > > > > repository (incubator-kie-examples) for all release candidates and > > > > releases we do. This repository would be THE consumable repository > for > > > > users who wish to go straight to the point when testing out Apache > KIE > > > > and who are not necessarily committers/developers. > > > > > > > > To do that, changing some release automations to add a new job that > > > > builds and publishes the Examples ZIP would be necessary: > > > > > > > > 1. Daily-dev ( > > > https://ci-builds.apache.org/job/KIE/job/kie-tools/job/main/job/kie-tools-daily-dev-publish/ > > ) > > > > 2. Release Candidates > > > > ( > > > https://ci-builds.apache.org/job/KIE/job/kie-tools/job/main/job/release/job/kie-tools-release-candidate/ > > ) > > > > 3. Release Promote > > > > ( > > > https://ci-builds.apache.org/job/KIE/job/kie-tools/job/main/job/release/job/kie-tools-release-promote/ > > > > ) > > > > > > > > > > > > #### New Continuous Deployment (CD) > > > > > > > > Websites and Docs would be treated as continually deployed packages > > > > for the development stream we declare as “live”. Also, old > development > > > > streams would be continually built and published to make > documentation > > > > updates possible for older versions. Referring to Toshiya's thread on > > > > documentation [1], our Docs would be “mutable”, and the granularity > > > > would be “per stream”. > > > > > > > > Two new automations would be created for making sure our content is > > > > up-to-date at all times: > > > > > > > > `publish-kie-apache-org-website`: Responsible for updating live > Apache > > > > KIE’s website (https://kie.apache.org) and also capable of > generating > > > > ad-hoc immutable copies of the website for Release Candidates. It > > > > would be able to also be executed for the `main` stream for > > > > development purposes. > > > > > > > > `publish-older-docs-to-live-website` ensures that older development > > > > streams can have their content updated on the live website. For > > > > example, if we find a seriously incorrect statement in an older > > > > version, this job would ensure it’s published after being fixed. We > > > > can eventually remove this for versions that become too old. > > > > > > > > > > > > # Actionable items (plan) > > > > > > > > Phase 1 (Development and contributions): > > > > > > > > - Move all examples to `kie-tools`, already in the new > > > > structure/organization, and integrate them as part of the build > system > > > > - Move all documentation content to `kie-tools` and integrate the > > > > new packages as part of the build system > > > > - Move `incubator-kie-website` content to `kie-tools` and integrate > > > > it as part of the build system > > > > - Create new landing pages for each major component: Drools, jBPM, > > > > SonataFlow, Optaplanner, and Kogito. > > > > > > > > Phase 2 (Release Process) > > > > > > > > - Create the `incubator-kie-examples` repository > > > > - Update `kie-tools` daily-dev job to publish the Examples ZIP to > > > > `incubator-kie-examples@main` > > > > - Update `kie-tools` Release Candidate job to publish the Examples > > > > ZIP to https://dist.apache.org/repos/dist/dev/incubator/kie/ > > > > - Update `kie-tools` Release Promote job to publish the Examples ZIP > > > > content to incubator-kie-examples@{version}` > > > > - Archive the `incubator-kie-kogito-examples` repository (post 10.1 > > release) > > > > - Update the Release Procedure for 10.1 to include Docs, Examples, > > > > and Websites where necessary > > > > > > > > Phase 3 (Continuous deployments) > > > > > > > > - Create and configure new `publish-kie-apache-org-website` job > > > > - Create and configure new `publish-docs-to-live-website` job > > > > > > > > Phase 4 (Cleanup) > > > > > > > > - Depublish current `drools.org`, `jbpm.org`, `optaplanner.org`, > > > > `kie.org`, `sonataflow.org` and all its subdomains. > > > > - Redirect all old domains to the proper landing page of Apache > KIE’s > > > > widely known components (E.g., https://drools.org → > > > > https://kie.apache.org/drools) > > > > - Preserve Docs from older versions somewhere accessible under Apache > > > > KIE’s website. > > > > > > > > > > > > # Commitment and rough timeline > > > > > > > > Alex Porcelli and Tiago Bento are fully committed to executing this > > > > plan in collaboration with other members who want to help. > > > > > > > > We are committed to working on it as soon as the proposal is > approved. > > > > Although it’s hard to predict how long it will take, Phases 1, 2, and > > > > 3 will take around 6 weeks based on the information we have today. If > > > > anything changes or issues are found during execution, we can update > > > > this thread. > > > > > > > > Phase 4 is impossible to estimate, as there might be many different > > > > people and/or organizations to chase so that these websites can be > > > > decommissioned and properly forward users to KIE’s new home in > Apache. > > > > > > > > > > > > # Observations on `kie-tools` > > > > > > > > This proposal makes Docs, Examples, and Websites available to all > > > > users without touching the complex, unmaintained, black-box systems > > > > currently in Apache’s Jenkins, known as “build-chain” and “the Kogito > > > > CI framework”. > > > > > > > > We know that `kie-tools` is not unanimously popular among Apache KIE > > > > committers, but it’s actively maintained and has been proven to scale > > > > well both in package count and technology diversity. We invite > > > > everyone to take a look at the repo’s User Manual at > > > > > https://github.com/apache/incubator-kie-tools/blob/main/repo/MANUAL.md > > > > > > > > > > > > ## Non-obvious positive outcomes > > > > > > > > - Drools and OptaPlanner repositories won’t have dependencies on > > > > JavaScript anymore. > > > > - Fonts and other “binary” files won’t need to be committed as a > > > > source anymore, as `kie-tools`’s build system can properly depend on > > > > NPM artifacts. Also helps towards graduation, as Category X > > > > dependencies won’t be present. > > > > > > > > > > > > [1] https://lists.apache.org/thread/zjt156tk5zjw7463xsfdkojwzndwd1mn > > > > [2] https://lists.apache.org/thread/qc9vhod96mdoppo5ssj4f0pkqhzt4ghd > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > > > > For additional commands, e-mail: dev-h...@kie.apache.org > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > > > For additional commands, e-mail: dev-h...@kie.apache.org > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > > For additional commands, e-mail: dev-h...@kie.apache.org > > > > >
