Re: [Website] Ongoing problem with broken links in blog posts
I opened some issues... > On Feb 8, 2022, at 11:25 AM, Zoran Regvart wrote: > > Hi David & Cameleers, > > On Tue, Feb 8, 2022 at 7:57 PM David Jencks <mailto:david.a.jen...@gmail.com>> wrote: >> We have a problem in a lot of blog posts that they link into the Antora >> generated docs. > > Yes this happens with increasing frequency and it is problematic > >> Many link to the “next” version of the docs which has two problems: >> >> - as release announcements, they should link to the doc version that’s been >> released. >> - there’s no reason to suppose any particular page in ‘next’ is going to >> stay there. > > We should agree never to link to `next` or `latest`, but to concrete > versions. A validation rule to that effect can be implemented to > enforce this. https://github.com/apache/camel-website/issues/779 <https://github.com/apache/camel-website/issues/779> deals with fixing existing blog posts. https://github.com/apache/camel-website/issues/780 <https://github.com/apache/camel-website/issues/780> suggests adding a rule to keep us out of trouble in the future. > >> From this point of view we should change all the release announcement links >> to point to the docs in the released version. That’s an improvement, but it >> will cause another problem: >> >> - At the moment our plan is to drop the docs for no-longer-supported >> versions when they aren’t used by supported versions of other subprojects. >> >> So, eventually all the links to docs in blog posts will break. >> >> I think we should: >> >> - change all blog post links to point to explicit appropriate doc versions. >> I’m somewhat afraid about how big a job this will be, but the current links >> to “next” are clearly wrong in all cases. >> >> - Decide what to do about old versions. I can think of three possibilities: >> >> — Keep all the old docs versions in the docs, perhaps marking them >> “unsupported”. There might be ways to do this without building those >> portions of the website every time but this would take some time and thought >> and wouldn’t be my highest priority for some time. >> — Delink broken links in the blog, marking them something “no longer >> documented” or “obsolete” or something. >> — Remove the blog posts about no-longer-documented versions. >> >> I really don’t know what to do here, so I’m hoping discussion will result in >> a “best path forward”. > > I'd like to be in a place where we could keep the old versions of the > documentation, given that we've split off the website source and > publish repositories. Perhaps there is a way forward by keeping more > content in the publish repositories, even if it is not built/linked > from the site. As a form of archived documentation. For that we would > need to keep everything within the "/_" (CSS, images...) and when > publishing instead of deleting everything[1] we delete only the built > versions. I also would really like to be able to keep all the old doc versions somehow. Around some details, either I don’t quite understand you or slightly disagree on what is needed or will work. I opened https://github.com/apache/camel-website/issues/781 <https://github.com/apache/camel-website/issues/781> to try to explain my thinking on what we could do. > > The drawback of this approach is that it might overwhelm whatever is > done outside of our purview to publish the site, but we can try and > see if it really does and engage INFRA to explore options. It seems implausible to me that even a few GB of infrequently accessed static content would be a problem, and I think we should start by solving the problems of how to avoid generating the entire site on every build. thanks! David Jencks > > zoran > [1] > https://github.com/apache/camel-website/blob/b978a3e6af3fcc97808515bc01a004ddd28426a6/Jenkinsfile#L99 > > <https://github.com/apache/camel-website/blob/b978a3e6af3fcc97808515bc01a004ddd28426a6/Jenkinsfile#L99> > -- > Zoran Regvart
[Website] Ongoing problem with broken links in blog posts
We have a problem in a lot of blog posts that they link into the Antora generated docs. Many link to the “next” version of the docs which has two problems: - as release announcements, they should link to the doc version that’s been released. - there’s no reason to suppose any particular page in ‘next’ is going to stay there. From this point of view we should change all the release announcement links to point to the docs in the released version. That’s an improvement, but it will cause another problem: - At the moment our plan is to drop the docs for no-longer-supported versions when they aren’t used by supported versions of other subprojects. So, eventually all the links to docs in blog posts will break. I think we should: - change all blog post links to point to explicit appropriate doc versions. I’m somewhat afraid about how big a job this will be, but the current links to “next” are clearly wrong in all cases. - Decide what to do about old versions. I can think of three possibilities: — Keep all the old docs versions in the docs, perhaps marking them “unsupported”. There might be ways to do this without building those portions of the website every time but this would take some time and thought and wouldn’t be my highest priority for some time. — Delink broken links in the blog, marking them something “no longer documented” or “obsolete” or something. — Remove the blog posts about no-longer-documented versions. I really don’t know what to do here, so I’m hoping discussion will result in a “best path forward”. This problem has been appearing with some regularity, and appeared today with camel-quarkus. I propose https://github.com/apache/camel-website/pull/778 <https://github.com/apache/camel-website/pull/778> to fix todays problem, which follows none of these ideas and is not very satisfactory, but allows the site to build. David Jencks
Re: Dependabot alerts
I don’t see an obvious upgrade path to ameliorate these docs/* “security” problems (I upped all the versions I could in package.json with no useful effect), but: - I’ve been planning on completely eliminating all the copying with the gulp file (where the dependencies come from) and having Antora find the originals instead. This is going to require a camel-specific Antora extension (first one!) but should not be terribly difficult. We then wouldn’t have a yarn.lock :-) This may take a while. - I don’t think there’s actually any security risk to running a script to symlink some files in the git repo, and committing the result. I’d appreciate Zoran’s perspective, I am by no means a security expert. David Jencks > On Feb 3, 2022, at 1:23 PM, Claus Ibsen wrote: > > Hi > > The most of the remainder alerts are in the docs folder about yarn. > > Wonder if David or Zoran would take a look? > > On Thu, Feb 3, 2022 at 1:02 PM Colm O hEigeartaigh > wrote: >> >> Hi, >> >> I've worked with INFRA to enable GitHub dependabot alerts for various >> Apache projects. The idea is that the GitHub committers for a given >> project can have access to the page on GitHub (for example for CXF: >> https://github.com/apache/cxf/security/dependabot) which shows the >> list of dependencies for the project with known CVEs. >> >> I plan to do the same for Camel on these repos: >> >> https://github.com/apache/camel >> https://github.com/apache/camel-karaf >> https://github.com/apache/camel-quarkus >> https://github.com/apache/camel-spring-boot >> >> Any objections or anything I'm missing? If not I'll proceed with enabling it. >> >> Colm. > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2
Re: comments on c-k-c docs state && RI preview
There’s the preview PR which shouldn’t be merged because it refers to all my repos, but there’s also https://github.com/apache/camel-website/pull/763 <https://github.com/apache/camel-website/pull/763> which does need to be merged but I haven’t been able to attract anyone’s attention to it. That will bring in the contents to all these new tables plus the newly released camel-k/kamelets branches. David Jencks > On Jan 24, 2022, at 4:36 PM, Andrea wrote: > > Hello, > > On Sun, Jan 23, 2022, at 07:18, David Jencks wrote: >> With 18 PRs, there are compatibility tables for everything except camel >> base, camel-karaf, and camel-spring boot: see preview at e.g. >> https://pr-747--camel.netlify.app/camel-quarkus/next/index.html#_compatibility_matrix >> >> <https://pr-747--camel.netlify.app/camel-quarkus/next/index.html#_compatibility_matrix> >> >> <https://pr-747--camel.netlify.app/camel-quarkus/next/index.html#_compatibility_matrix >> >> <https://pr-747--camel.netlify.app/camel-quarkus/next/index.html#_compatibility_matrix>>. > > Thanks for the work! Preview looks awesome! > > Published site have some issues though? > > > the PR of the preview is marked as don' merge, what is holding us off? > >> >> These PRs also add the branches for camel-k 1.8.x,camel-k-runtime 1.11.x, >> and kamelets 0.7.x releases, I expect the vote can be concluded at any time. >> >> I think many of the PR builds are failing: the few I’ve looked at appear to >> be for reasons unrelated to these PRs. If these PRs cause actual problems >> I’m happy to fix them but it would be great if others could determine >> whether they cause problems. >> >> I think it would be a good idea to eventually add this info for the >> remaining projects, but I’m planning to take a break and work on something >> more interesting for a while, such as generating camel-quarkus pages. >> >> David Jencks >> >> > On Jan 18, 2022, at 10:29 PM, Andrea Cosentino > > <mailto:anco...@gmail.com>> wrote: >> > >> > I think it would be really great to have it for camel-k and all the other >> > subprojects. >> > >> > Il giorno mer 19 gen 2022 alle ore 07:27 Claus Ibsen >> > mailto:claus.ib...@gmail.com>> >> > ha scritto: >> > >> >> On Wed, Jan 19, 2022 at 3:08 AM David Jencks > >> <mailto:david.a.jen...@gmail.com>> >> >> wrote: >> >>> >> >>> Putting the conclusion at the top…. >> >>> >> >>> https://pr-747--camel.netlify.app/camel-kafka-connector/next/index.html >> >>> <https://pr-747--camel.netlify.app/camel-kafka-connector/next/index.html> >> >> <https://pr-747--camel.netlify.app/camel-kafka-connector/next/index.html >> >> <https://pr-747--camel.netlify.app/camel-kafka-connector/next/index.html>> >> >>> >> >>> I put the compatibility table at the bottom of the index page, added a >> >> kamelets column, and turned entries into links where plausible. >> >>> >> >>> The table is also on the compatibility matrix page. >> >>> >> >>> If the index page looks good, I’ll remove the compatibility page and >> >> squash everything. >> >>> >> >>> No one has commented on whether such a table would be useful or >> >> desirable for other camel subprojects. >> >>> >> >> >> >> Whoa that matrix looks really good. >> >> >> >> Yes I think its a good idea for other sub projects to have this page too. >> >> >> >> >> >> >> >> >> >> >> >>> David Jencks >> >>> >> >>>> On Jan 18, 2022, at 6:19 AM, Andrea > >>>> <mailto:and...@tarocch.it>> wrote: >> >>>> >> >>>> >> >>>> >> >>>> On Tue, Jan 18, 2022, at 05:55, David Jencks wrote: >> >>>>> >> >>>>> >> >>>>>> On Jan 17, 2022, at 1:50 PM, Andrea > >>>>>> <mailto:and...@tarocch.it>> wrote: >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> On Mon, Jan 17, 2022, at 18:00, David Jencks wrote: >> >>>>>>> >> >>>>>>> >> >>>>>>>> On Jan 17, 2022, at 3:23
Re: comments on c-k-c docs state && RI preview
With 18 PRs, there are compatibility tables for everything except camel base, camel-karaf, and camel-spring boot: see preview at e.g. https://pr-747--camel.netlify.app/camel-quarkus/next/index.html#_compatibility_matrix <https://pr-747--camel.netlify.app/camel-quarkus/next/index.html#_compatibility_matrix>. These PRs also add the branches for camel-k 1.8.x,camel-k-runtime 1.11.x, and kamelets 0.7.x releases, I expect the vote can be concluded at any time. I think many of the PR builds are failing: the few I’ve looked at appear to be for reasons unrelated to these PRs. If these PRs cause actual problems I’m happy to fix them but it would be great if others could determine whether they cause problems. I think it would be a good idea to eventually add this info for the remaining projects, but I’m planning to take a break and work on something more interesting for a while, such as generating camel-quarkus pages. David Jencks > On Jan 18, 2022, at 10:29 PM, Andrea Cosentino wrote: > > I think it would be really great to have it for camel-k and all the other > subprojects. > > Il giorno mer 19 gen 2022 alle ore 07:27 Claus Ibsen > ha scritto: > >> On Wed, Jan 19, 2022 at 3:08 AM David Jencks >> wrote: >>> >>> Putting the conclusion at the top…. >>> >>> https://pr-747--camel.netlify.app/camel-kafka-connector/next/index.html >> <https://pr-747--camel.netlify.app/camel-kafka-connector/next/index.html> >>> >>> I put the compatibility table at the bottom of the index page, added a >> kamelets column, and turned entries into links where plausible. >>> >>> The table is also on the compatibility matrix page. >>> >>> If the index page looks good, I’ll remove the compatibility page and >> squash everything. >>> >>> No one has commented on whether such a table would be useful or >> desirable for other camel subprojects. >>> >> >> Whoa that matrix looks really good. >> >> Yes I think its a good idea for other sub projects to have this page too. >> >> >> >> >> >>> David Jencks >>> >>>> On Jan 18, 2022, at 6:19 AM, Andrea wrote: >>>> >>>> >>>> >>>> On Tue, Jan 18, 2022, at 05:55, David Jencks wrote: >>>>> >>>>> >>>>>> On Jan 17, 2022, at 1:50 PM, Andrea wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Jan 17, 2022, at 18:00, David Jencks wrote: >>>>>>> >>>>>>> >>>>>>>> On Jan 17, 2022, at 3:23 AM, Andrea > and...@tarocch.it>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Jan 17, 2022, at 07:24, David Jencks wrote: >>>>>>>>> Thanks, inline... >>>>>>>>> >>>>>>>>>> On Jan 16, 2022, at 1:04 AM, Andrea > and...@tarocch.it>> wrote: >>>>>>>>>> >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> comments inline: >>>>>>>>>> >>>>>>>>>> On Sat, Jan 15, 2022, at 06:37, David Jencks wrote: >>>>>>>>>>> I noticed a few things working on the RI info for >> camel-kafka-connector. >>>>>>>>>>> >>>>>>>>>>> - the compatibility matrices are thoroughly out of date, e.g. >> https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html >> < >> https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html> >> < >> https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html >> < >> https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html >>>> >>>>>> >>>>>> Awesome, basically a collection of the content of all these notes >>>>>> >>>>>> could be the generated compatibility matrix page? Or is there >> already a way where that content is sown other that for the last release? >>>>> >>>>> Here’s a basic generated table for the compatibility matrix: >> https://pr-747--camel.netlify.app/camel-kafka-connector/1.0.x/user-guide/camel-compatibility-matrix.html >>>>> >>>>> Note: >>>&g
Re: [VOTE] Release Apache Camel K 1.8.0 and related libraries
Thanks! > On Jan 20, 2022, at 11:23 PM, Andrea Cosentino wrote: > > Done. > > Il giorno ven 21 gen 2022 alle ore 08:20 Andrea Cosentino > ha scritto: > >> I can create them from the tag. Let me do it now. >> >> Il giorno ven 21 gen 2022 alle ore 08:19 David Jencks < >> david.a.jen...@gmail.com> ha scritto: >> >>> The release branches for camel-k-runtime 1.11.0 and kamelets 0.7.x >>> haven’t been created yet, although obviously they can be at any time. >>> >>> The lack of the kamelets branch is blocking producing RI-tables for >>> kamelets and camel-k. >>> >>> David Jencks >>> >>>> On Jan 19, 2022, at 9:17 AM, Andrea Cosentino >>> wrote: >>>> >>>> Hello all: >>>> >>>> This is a combined vote to release Apache Camel K 1.8.0, Camel K Runtime >>>> 1.11.0 and Kamelets 0.7.0. >>>> >>>> This is a major release based on Camel-Quarkus 2.6.0 and Camel 3.14.0 >>> and >>>> contains several changes in both the operator and the runtime. >>>> >>>> Runtime source files: >>>> https://dist.apache.org/repos/dist/dev/camel/camel-k-runtime/1.11.0/ >>>> Runtime staging repository: >>>> https://repository.apache.org/content/repositories/orgapachecamel-1401 >>>> Runtime Tag: >>>> >>> https://gitbox.apache.org/repos/asf?p=camel-k-runtime.git;a=shortlog;h=refs/tags/camel-k-runtime-project-1.11.0 >>>> >>>> Kamelets release files: >>>> https://dist.apache.org/repos/dist/dev/camel/camel-kamelets/0.7.0/ >>>> Kamelets staging repository: >>>> https://repository.apache.org/content/repositories/orgapachecamel-1400 >>>> Kamelets Tag: >>>> >>> https://gitbox.apache.org/repos/asf?p=camel-kamelets.git;a=shortlog;h=refs/tags/v0.7.0 >>>> >>>> Camel K release files: >>>> https://dist.apache.org/repos/dist/dev/camel/camel-k/1.8.0/ >>>> Camel K Tag: >>>> >>> https://gitbox.apache.org/repos/asf?p=camel-k.git;a=shortlog;h=refs/tags/v1.8.0 >>>> >>>> Staging container image repository: >>> https://hub.docker.com/r/camelk/camel-k >>>> /tags >>>> >>>> It's possible to install all staging artifacts with a single command: >>>> kamel install --operator-image=camelk/camel-k:1.8.0 --maven-repository= >>>> https://repository.apache.org/content/repositories/orgapachecamel-1400 >>>> --maven-repository= >>>> <https://repository.apache.org/content/repositories/orgapachecamel-1400 >>>> >>>> >>> https://repository.apache.org/content/repositories/orgapachecamel-1401--olm=false >>>> <https://repository.apache.org/content/repositories/orgapachecamel-1401 >>>> >>>> >>>> Please test this release candidate and cast your vote. >>>> >>>> [ ] +1 Release the binary as Apache Camel K 1.8.0, Apache Camel K >>> Runtime >>>> 1.11.0 and Apache Camel Kamelets 0.7.0 >>>> [ ] -1 Veto the release (provide specific comments) >>>> >>>> The vote is open for at least 72 hours. >>>> >>>> Here's my +1. >>>> >>>> Thanks, >>>> Andrea Cosentino >>> >>>
Re: [VOTE] Release Apache Camel K 1.8.0 and related libraries
The release branches for camel-k-runtime 1.11.0 and kamelets 0.7.x haven’t been created yet, although obviously they can be at any time. The lack of the kamelets branch is blocking producing RI-tables for kamelets and camel-k. David Jencks > On Jan 19, 2022, at 9:17 AM, Andrea Cosentino wrote: > > Hello all: > > This is a combined vote to release Apache Camel K 1.8.0, Camel K Runtime > 1.11.0 and Kamelets 0.7.0. > > This is a major release based on Camel-Quarkus 2.6.0 and Camel 3.14.0 and > contains several changes in both the operator and the runtime. > > Runtime source files: > https://dist.apache.org/repos/dist/dev/camel/camel-k-runtime/1.11.0/ > Runtime staging repository: > https://repository.apache.org/content/repositories/orgapachecamel-1401 > Runtime Tag: > https://gitbox.apache.org/repos/asf?p=camel-k-runtime.git;a=shortlog;h=refs/tags/camel-k-runtime-project-1.11.0 > > Kamelets release files: > https://dist.apache.org/repos/dist/dev/camel/camel-kamelets/0.7.0/ > Kamelets staging repository: > https://repository.apache.org/content/repositories/orgapachecamel-1400 > Kamelets Tag: > https://gitbox.apache.org/repos/asf?p=camel-kamelets.git;a=shortlog;h=refs/tags/v0.7.0 > > Camel K release files: > https://dist.apache.org/repos/dist/dev/camel/camel-k/1.8.0/ > Camel K Tag: > https://gitbox.apache.org/repos/asf?p=camel-k.git;a=shortlog;h=refs/tags/v1.8.0 > > Staging container image repository: https://hub.docker.com/r/camelk/camel-k > /tags > > It's possible to install all staging artifacts with a single command: > kamel install --operator-image=camelk/camel-k:1.8.0 --maven-repository= > https://repository.apache.org/content/repositories/orgapachecamel-1400 > --maven-repository= > <https://repository.apache.org/content/repositories/orgapachecamel-1400> > https://repository.apache.org/content/repositories/orgapachecamel-1401--olm=false > <https://repository.apache.org/content/repositories/orgapachecamel-1401> > > Please test this release candidate and cast your vote. > > [ ] +1 Release the binary as Apache Camel K 1.8.0, Apache Camel K Runtime > 1.11.0 and Apache Camel Kamelets 0.7.0 > [ ] -1 Veto the release (provide specific comments) > > The vote is open for at least 72 hours. > > Here's my +1. > > Thanks, > Andrea Cosentino
Re: comments on c-k-c docs state && RI preview
Putting the conclusion at the top…. https://pr-747--camel.netlify.app/camel-kafka-connector/next/index.html <https://pr-747--camel.netlify.app/camel-kafka-connector/next/index.html> I put the compatibility table at the bottom of the index page, added a kamelets column, and turned entries into links where plausible. The table is also on the compatibility matrix page. If the index page looks good, I’ll remove the compatibility page and squash everything. No one has commented on whether such a table would be useful or desirable for other camel subprojects. David Jencks > On Jan 18, 2022, at 6:19 AM, Andrea wrote: > > > > On Tue, Jan 18, 2022, at 05:55, David Jencks wrote: >> >> >>> On Jan 17, 2022, at 1:50 PM, Andrea wrote: >>> >>> >>> >>> On Mon, Jan 17, 2022, at 18:00, David Jencks wrote: >>>> >>>> >>>>> On Jan 17, 2022, at 3:23 AM, Andrea >>>> <mailto:and...@tarocch.it>> wrote: >>>>> >>>>> >>>>> >>>>> On Mon, Jan 17, 2022, at 07:24, David Jencks wrote: >>>>>> Thanks, inline... >>>>>> >>>>>>> On Jan 16, 2022, at 1:04 AM, Andrea >>>>>> <mailto:and...@tarocch.it>> wrote: >>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> comments inline: >>>>>>> >>>>>>> On Sat, Jan 15, 2022, at 06:37, David Jencks wrote: >>>>>>>> I noticed a few things working on the RI info for >>>>>>>> camel-kafka-connector. >>>>>>>> >>>>>>>> - the compatibility matrices are thoroughly out of date, e.g. >>>>>>>> https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html >>>>>>>> >>>>>>>> <https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html> >>>>>>>> >>>>>>>> <https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html >>>>>>>> >>>>>>>> <https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html>> >>> >>> Awesome, basically a collection of the content of all these notes >>> >>> could be the generated compatibility matrix page? Or is there already a >>> way where that content is sown other that for the last release? >> >> Here’s a basic generated table for the compatibility matrix: >> https://pr-747--camel.netlify.app/camel-kafka-connector/1.0.x/user-guide/camel-compatibility-matrix.html >> >> Note: >> - it will only ever have one line per branch: there’s no way to have >> separate lines for e.g. 0.11.0, 0.11.1. 0.11.2, etc, since there is only one >> doc version that covers all of these. > > well in case it is needed we can always do a version from a tag >> - It only has currently documented versions listed. This makes it >> self-maintaining: adding anything else will require periodic manual updates, >> which will gradually decay into wrong or outdated information. >> >> Are these acceptable limitations? > > fine for me >> >> - This version of the table isn’t quite finished: for instance the “branch” >> for next is wrong. If we like this generated table in principle, I can fix >> that and turn most entries into links. > > can you also add the version of the kamelet catalog? >> >> However, I’m not entirely sure that having this information >> duplicated/aggregated from the index pages is useful. I think we should just >> delete the compatibility matrix page. WDYT? > > as long as the table is somewhere I don't particularly care; it is fine to > put it in the index page and remove the compatibility matrix page >> >> On the other hand, if this table is popular, maybe we should have one for >> each subproject. >> >> David Jencks >> >>> >>> >>>>>>> Yep the compatibility matrix page needs some love... a column >>>>>>> mentioning kamelet catalog version needs to be added and probably we >>>>>>> can remove some old rows? >>>>>>> willing to help on this on too? :) >>>>>> >>>>>> I think the entire existing matrix is out of date and should be removed? >>>>>>
Re: comments on c-k-c docs state && RI preview
> On Jan 17, 2022, at 1:50 PM, Andrea wrote: > > > > On Mon, Jan 17, 2022, at 18:00, David Jencks wrote: >> >> >> > On Jan 17, 2022, at 3:23 AM, Andrea > > <mailto:and...@tarocch.it>> wrote: >> > >> > >> > >> > On Mon, Jan 17, 2022, at 07:24, David Jencks wrote: >> >> Thanks, inline... >> >> >> >>> On Jan 16, 2022, at 1:04 AM, Andrea > >>> <mailto:and...@tarocch.it>> wrote: >> >>> >> >>> Hello, >> >>> >> >>> comments inline: >> >>> >> >>> On Sat, Jan 15, 2022, at 06:37, David Jencks wrote: >> >>>> I noticed a few things working on the RI info for camel-kafka-connector. >> >>>> >> >>>> - the compatibility matrices are thoroughly out of date, e.g. >> >>>> https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html >> >>>> >> >>>> <https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html> >> >>>> >> >>>> <https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html >> >>>> >> >>>> <https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html>> > > Awesome, basically a collection of the content of all these notes > > could be the generated compatibility matrix page? Or is there already a way > where that content is sown other that for the last release? Here’s a basic generated table for the compatibility matrix: https://pr-747--camel.netlify.app/camel-kafka-connector/1.0.x/user-guide/camel-compatibility-matrix.html Note: - it will only ever have one line per branch: there’s no way to have separate lines for e.g. 0.11.0, 0.11.1. 0.11.2, etc, since there is only one doc version that covers all of these. - It only has currently documented versions listed. This makes it self-maintaining: adding anything else will require periodic manual updates, which will gradually decay into wrong or outdated information. Are these acceptable limitations? - This version of the table isn’t quite finished: for instance the “branch” for next is wrong. If we like this generated table in principle, I can fix that and turn most entries into links. However, I’m not entirely sure that having this information duplicated/aggregated from the index pages is useful. I think we should just delete the compatibility matrix page. WDYT? On the other hand, if this table is popular, maybe we should have one for each subproject. David Jencks > > >> >>> Yep the compatibility matrix page needs some love... a column mentioning >> >>> kamelet catalog version needs to be added and probably we can remove >> >>> some old rows? >> >>> willing to help on this on too? :) >> >> >> >> I think the entire existing matrix is out of date and should be removed? >> >> Or are there usable versions of c-k-c that aren’t documented? >> >> >> >> I wonder if the RI information is sufficient, WDYT? >> > >> > Where can I see a preview of the site with the IR information? >> >> https://pr-747--camel.netlify.app/camel-kafka-connector/1.0.x/index.html >> <https://pr-747--camel.netlify.app/camel-kafka-connector/1.0.x/index.html> >> <https://pr-747--camel.netlify.app/camel-kafka-connector/1.0.x/index.html >> <https://pr-747--camel.netlify.app/camel-kafka-connector/1.0.x/index.html>> >> shows all three branches (including not-quite-released 1.0). >> >> >> >> >> If you’re sure we need the matrix, let me know where it should start and >> >> I’ll make some PRs. I’d suggest having only one copy, perhaps in next, >> >> and referring to it from the other branches. >> >>> >> >>>> >> >>>> - All other camel subprojects use e.g. 2.5.x as the Antora component >> >>>> version, but c-k-c is using 0.11.0. Especially since it’s LTS I think >> >>>> we should change it to 0.11.x so when 0.11.1 comes out the version >> >>>> still makes sense, as well as being consistent with the rest of the >> >>>> site. I’m setting the 1.0.x branch up to say 1.0.x as the Antora >> >>>> version. >> >>> +1 I agree >> >>> >> >> >> >> I’ve changed to 0.11.x in my RI PR. >> >> >>
Re: comments on c-k-c docs state && RI preview
> On Jan 17, 2022, at 3:23 AM, Andrea wrote: > > > > On Mon, Jan 17, 2022, at 07:24, David Jencks wrote: >> Thanks, inline... >> >>> On Jan 16, 2022, at 1:04 AM, Andrea wrote: >>> >>> Hello, >>> >>> comments inline: >>> >>> On Sat, Jan 15, 2022, at 06:37, David Jencks wrote: >>>> I noticed a few things working on the RI info for camel-kafka-connector. >>>> >>>> - the compatibility matrices are thoroughly out of date, e.g. >>>> https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html >>>> >>>> <https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html> >>> Yep the compatibility matrix page needs some love... a column mentioning >>> kamelet catalog version needs to be added and probably we can remove some >>> old rows? >>> willing to help on this on too? :) >> >> I think the entire existing matrix is out of date and should be removed? Or >> are there usable versions of c-k-c that aren’t documented? >> >> I wonder if the RI information is sufficient, WDYT? > > Where can I see a preview of the site with the IR information? https://pr-747--camel.netlify.app/camel-kafka-connector/1.0.x/index.html <https://pr-747--camel.netlify.app/camel-kafka-connector/1.0.x/index.html> shows all three branches (including not-quite-released 1.0). >> >> If you’re sure we need the matrix, let me know where it should start and >> I’ll make some PRs. I’d suggest having only one copy, perhaps in next, and >> referring to it from the other branches. >>> >>>> >>>> - All other camel subprojects use e.g. 2.5.x as the Antora component >>>> version, but c-k-c is using 0.11.0. Especially since it’s LTS I think we >>>> should change it to 0.11.x so when 0.11.1 comes out the version still >>>> makes sense, as well as being consistent with the rest of the site. I’m >>>> setting the 1.0.x branch up to say 1.0.x as the Antora version. >>> +1 I agree >>> >> >> I’ve changed to 0.11.x in my RI PR. >> >>>> >>>> - archetype-dataformat-connector has camel-version 3.6.0, rather out of >>>> date. >>> What do you mean here? >> >> I should have looked harder and explained better. The example output shown >> in archetype-dataformat-connector.adoc shows using camel 3.6.0. This page >> should probably be updated, and I wonder if it is even relevant for the >> kamelet-based c-k-c. > > Yep for sure that part needs to be revisited and re-evaluated... >>> >>>> >>>> - the maven versions in 1.0.x branch is 1.1.0-SNAPSHOT and for 0.11.x >>>> 0.12.0-SNAPSHOT. I think these should increment the micro version, not >>>> minor version? >>> In theory you are right, in practice for convenience and based on how maven >>> release plugin works and how we do releases, is more convenient to leave >>> the version set as next release version even in the single releases >>> branches... hoping to remember that once a new minor release of a release >>> branch is needed. >>> >>> I admit that might be we are just being lazy and there is a reasonably >>> hassle free way of handling this better? >> >> Perhaps I didn’t explain very well. Both 0.11.0 and 1.0.0 are LTS so we can >> expect releases on these branches. The next versions will be 0.11.1 and >> 1.0.1, so the current maven versions should be 0.11.1-SNAPSHOT and >> 1.0.1-SNAPSHOT, with the micro version incremented: the current >> 0.12.0-SNAPSHOT and 1.1.0-SNAPSHOT are extremely misleading. > > I understand that is misleading, but at the same time it is convenient from a > release pov, because we mass change the version only during release and it is > set to the next "major" release that will happen from main branch... doing > what you suggest would require to add some steps to the already long and > tedious release process... I am not sure it is worth the effort but I admit I > am biased being the one who does the releases 90% of the times :) It’s been years since I did a release, and I’m not sure the maven tools have gotten much better. However, I think the other camel subprojects have found a way to have the branch maven versions more correct. I think that “LTS” means to expect more releases on the branch, and changing the branch version from 0.12.0-SNAPSHOT to 0.11.1-SNAPSHOT before release seems extremel
Re: comments on c-k-c docs state && RI preview
Thanks, inline... > On Jan 16, 2022, at 1:04 AM, Andrea wrote: > > Hello, > > comments inline: > > On Sat, Jan 15, 2022, at 06:37, David Jencks wrote: >> I noticed a few things working on the RI info for camel-kafka-connector. >> >> - the compatibility matrices are thoroughly out of date, e.g. >> https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html >> >> <https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html> > Yep the compatibility matrix page needs some love... a column mentioning > kamelet catalog version needs to be added and probably we can remove some old > rows? > willing to help on this on too? :) I think the entire existing matrix is out of date and should be removed? Or are there usable versions of c-k-c that aren’t documented? I wonder if the RI information is sufficient, WDYT? If you’re sure we need the matrix, let me know where it should start and I’ll make some PRs. I’d suggest having only one copy, perhaps in next, and referring to it from the other branches. > >> >> - All other camel subprojects use e.g. 2.5.x as the Antora component >> version, but c-k-c is using 0.11.0. Especially since it’s LTS I think we >> should change it to 0.11.x so when 0.11.1 comes out the version still makes >> sense, as well as being consistent with the rest of the site. I’m setting >> the 1.0.x branch up to say 1.0.x as the Antora version. > +1 I agree > I’ve changed to 0.11.x in my RI PR. >> >> - archetype-dataformat-connector has camel-version 3.6.0, rather out of date. > What do you mean here? I should have looked harder and explained better. The example output shown in archetype-dataformat-connector.adoc shows using camel 3.6.0. This page should probably be updated, and I wonder if it is even relevant for the kamelet-based c-k-c. > >> >> - the maven versions in 1.0.x branch is 1.1.0-SNAPSHOT and for 0.11.x >> 0.12.0-SNAPSHOT. I think these should increment the micro version, not >> minor version? > In theory you are right, in practice for convenience and based on how maven > release plugin works and how we do releases, is more convenient to leave the > version set as next release version even in the single releases branches... > hoping to remember that once a new minor release of a release branch is > needed. > > I admit that might be we are just being lazy and there is a reasonably hassle > free way of handling this better? Perhaps I didn’t explain very well. Both 0.11.0 and 1.0.0 are LTS so we can expect releases on these branches. The next versions will be 0.11.1 and 1.0.1, so the current maven versions should be 0.11.1-SNAPSHOT and 1.0.1-SNAPSHOT, with the micro version incremented: the current 0.12.0-SNAPSHOT and 1.1.0-SNAPSHOT are extremely misleading. > >> >> - Is 1.0.x LTS? > Yes it is, as it is 0.11.0 I changed to this in my RI PR. >> >> - I guess it would make sense to change >> >> Camel Kafka Connector allows you to use all Camel components >> >> as Kafka Connect <http://kafka.apache.org/documentation/#connect> >> connectors. >> to >> Camel Kafka Connector allows you to use all Kamelets as Kafka Connect >> <http://kafka.apache.org/documentation/#connect> connectors.* > +1 I agree Lets do that in another PR :-). > >> >> As part of the RI effort there’s a preview for c-k-c at >> https://pr-747--camel.netlify.app/camel-kafka-connector/next/index.html >> >> David Jencks The RI PRs are merged and the next and 0,11.x should be visible shortly. Thanks David Jencks
Re: [VOTE] Release Apache Camel-kafka-connector 1.0.0
I’m glad it’s not a problem, thanks! FWIW, I’d appreciate it if when there’s something that doesn’t follow the “SHOULD” of an apache policy there’s an explanation of why: in this case the 48 hr rather than 72 hour voting window. Thanks, David Jencks > On Jan 14, 2022, at 4:11 PM, Andrea wrote: > > > > On Fri, Jan 14, 2022, at 22:00, David Jencks wrote: >> I don’t know if this makes any difference to anything, but I see that the >> camel-k-runtime version in ckc is 1.8.0, using kamelets 0.6.0, whereas in >> camel-k main it’s 1.10.0 using kamelets 0.5.0. >> (I’m looking at main rather than the tag) >> >> David Jencks > > The dependency can be removed, it was used just in a disabled itest and it is > not needed anymore. So I think that is not affecting what we are releasing. > > Again thanks for the good catch, pr opened to fix it in main: > https://github.com/apache/camel-kafka-connector/pull/1314 > >> >>> On Jan 14, 2022, at 7:44 AM, Andrea wrote: >>> >>> Hello all, >>> >>> This is a vote to release Apache Camel-kafka-connector 1.0.0 >>> >>> Staging repository: >>> https://repository.apache.org/content/repositories/orgapachecamel-1399 >>> >>> Tag: >>> https://gitbox.apache.org/repos/asf?p=camel-kafka-connector.git;a=tag;h=refs/tags/camel-kafka-connector-1.0.0 >>> >>> based on camel 3.14.0 and kamelet catalog 0.6.0 is the first release where >>> connectors are generated mainly from kamelets! >>> >>> Please test this release candidate and cast your vote. >>> [ ] +1 Release the binary as Apache Camel-kafka-connector 1.0.0 >>> [ ] -1 Veto the release (provide specific comments) >>> >>> The vote is open for at least 48 hours. >>> >>> Thanks, >>> Andrea. >> >>
comments on c-k-c docs state && RI preview
I noticed a few things working on the RI info for camel-kafka-connector. - the compatibility matrices are thoroughly out of date, e.g. https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html <https://camel.apache.org/camel-kafka-connector/0.11.0/user-guide/camel-compatibility-matrix.html> - All other camel subprojects use e.g. 2.5.x as the Antora component version, but c-k-c is using 0.11.0. Especially since it’s LTS I think we should change it to 0.11.x so when 0.11.1 comes out the version still makes sense, as well as being consistent with the rest of the site. I’m setting the 1.0.x branch up to say 1.0.x as the Antora version. - archetype-dataformat-connector has camel-version 3.6.0, rather out of date. - the maven versions in 1.0.x branch is 1.1.0-SNAPSHOT and for 0.11.x 0.12.0-SNAPSHOT. I think these should increment the micro version, not minor version? - Is 1.0.x LTS? - I guess it would make sense to change Camel Kafka Connector allows you to use all Camel components as Kafka Connect <http://kafka.apache.org/documentation/#connect> connectors. to Camel Kafka Connector allows you to use all Kamelets as Kafka Connect <http://kafka.apache.org/documentation/#connect> connectors. As part of the RI effort there’s a preview for c-k-c at https://pr-747--camel.netlify.app/camel-kafka-connector/next/index.html David Jencks
Re: [VOTE] Release Apache Camel-kafka-connector 1.0.0
I don’t know if this makes any difference to anything, but I see that the camel-k-runtime version in ckc is 1.8.0, using kamelets 0.6.0, whereas in camel-k main it’s 1.10.0 using kamelets 0.5.0. (I’m looking at main rather than the tag) David Jencks > On Jan 14, 2022, at 7:44 AM, Andrea wrote: > > Hello all, > > This is a vote to release Apache Camel-kafka-connector 1.0.0 > > Staging repository: > https://repository.apache.org/content/repositories/orgapachecamel-1399 > > Tag: > https://gitbox.apache.org/repos/asf?p=camel-kafka-connector.git;a=tag;h=refs/tags/camel-kafka-connector-1.0.0 > > based on camel 3.14.0 and kamelet catalog 0.6.0 is the first release where > connectors are generated mainly from kamelets! > > Please test this release candidate and cast your vote. > [ ] +1 Release the binary as Apache Camel-kafka-connector 1.0.0 > [ ] -1 Veto the release (provide specific comments) > > The vote is open for at least 48 hours. > > Thanks, > Andrea.
Re: [VOTE] Release Apache Camel-kafka-connector 1.0.0
I see the tag, but no branch. Do you usually create the branch after the vote succeeds? Thanks David Jencks > On Jan 14, 2022, at 7:44 AM, Andrea wrote: > > Hello all, > > This is a vote to release Apache Camel-kafka-connector 1.0.0 > > Staging repository: > https://repository.apache.org/content/repositories/orgapachecamel-1399 > > Tag: > https://gitbox.apache.org/repos/asf?p=camel-kafka-connector.git;a=tag;h=refs/tags/camel-kafka-connector-1.0.0 > > based on camel 3.14.0 and kamelet catalog 0.6.0 is the first release where > connectors are generated mainly from kamelets! > > Please test this release candidate and cast your vote. > [ ] +1 Release the binary as Apache Camel-kafka-connector 1.0.0 > [ ] -1 Veto the release (provide specific comments) > > The vote is open for at least 48 hours. > > Thanks, > Andrea.
Re: Camel Quarkus 2.2.1 LTS in preparation
What’s going to happen about docs? There are no 2.2.x docs at the moment…. I assumed 2.2.x was obsolete and unsupported. Given the length of time they have been missing, making them work will take some effort. David Jencks > On Jan 13, 2022, at 3:46 AM, Claus Ibsen wrote: > > On Thu, Jan 13, 2022 at 10:32 AM Peter Palaga wrote: >> >> Hi, >> >> this is a heads up that we are preparing the release of Camel Quarkus >> 2.2.1 these days. >> >> The main motivation was to fix the recent Log4j vulnerabilities in the >> Camel Quarkus branch based on Camel 3.14 LTS. Backporting those fixes, >> we told ourselves to backport a bit more fixes to have a proper 2.2.1 >> LTS release. >> >> This is the current state: >> https://github.com/apache/camel-quarkus/commits/2.2.x >> This PR is also targeted at 2.2.x: >> https://github.com/apache/camel-quarkus/pull/3456 >> >> Please feel free to raise your hand if you see some important fix to be >> missing in 2.2.x. >> >> Thanks, >> >> -- Peter >> > > +1 to get a LTS release for camel-quarkus > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2
Re: Website build and CI error - overall improved a lot
The spring-boot completeness check is a complicated thing…. Antora isn’t really designed to run IT tests :-) Do you have any suggestions about how to make fixing such problems easier or better documented? IIRC… — you get a build error that hints you should look at the spring boot index page. We can change the text of the broken link but can’t really produce any more details here. — you have to build the site locally to see the built page that shows the problem. Maybe Zoran knows a way, but I don’t know how to see the CI built site if the build broke. — Hopefully the locally built page shows the problem in enough detail so what to fix is clear. Thanks! David Jencks > On Jan 12, 2022, at 9:18 AM, Claus Ibsen wrote: > > Hi > > Today we had a CI build error of the website, and when you browse > there you get limited information about the problem (logs with ERROR). > > However all the hard work thanks to David Jenks, has improved this > experience a lot. > > There are good guides on how to install and build the website locally > (even per sub project). > And that really helps, where you can see more information about what > is the problem. > > So after some more attempts today we were able to fix a few things > that resolved the website build. > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2
Re: Please check that any documentation changes build!!
Even worse,
commit 8101887dc03352442f308358246409e425a9e441
Author: nicolaferraro
Date: Wed Jan 12 13:50:25 2022 +0100
Add KEDA markers to some Kamelets
docs/modules/ROOT/assets/images/kamelets/delay-action.svg | 1 -
docs/modules/ROOT/nav.adoc| 157
+
docs/modules/ROOT/pages/delay-action.adoc | 160
docs/{modules/ROOT => }/pages/index.adoc | 0
4 files changed, 1 insertion(+), 317 deletions(-)
completely removed the next kamelet docs.
Please pay attention to what you are doing to the docs!
David Jencks
> On Jan 12, 2022, at 6:56 PM, David Jencks wrote:
>
> Today the website build is broken due to:
>
> commit 4e15b691d47ed8c2aff86c4a08eb0b783f50d636
> Author: Antonin Stefanutti <mailto:anto...@stefanutti.fr>>
> Date: Wed Jan 12 11:33:37 2022 +0100
>
> feat: Add HTTP proxy documentation
>
>
> As detailed here:
>
> https://camel.apache.org/manual/improving-the-documentation.html#_quick_partial_build
>
> <https://camel.apache.org/manual/improving-the-documentation.html#_quick_partial_build>
>
> once you are set up, it is quick and easy to do a partial local build of your
> changes.
>
> Admittedly I’m familiar with this, but it took me about 2 minutes to run the
> local build, fix the problem, rerun the local build, and make
> https://github.com/apache/camel-k/pull/2874
> <https://github.com/apache/camel-k/pull/2874>.
>
> Please don’t break everything for everyone else by committing doc changes
> without verifying that they work.
>
> If you have problems setting this up, please let me know what they are rather
> than ignoring the possibility of stopping everyone else from working on docs.
>
> It would be great to eventually have a PR check for doc changes, and it might
> be within reach, but it involves some communication between repos and GitHub
> workflows, so it’s not trivial to set up.
>
> Would you commit java or go code that doesn’t compile? Don’t commit AsciiDoc
> that doesn’t compile either.
>
> Thanks
> David Jencks
Please check that any documentation changes build!!
Today the website build is broken due to: commit 4e15b691d47ed8c2aff86c4a08eb0b783f50d636 Author: Antonin Stefanutti Date: Wed Jan 12 11:33:37 2022 +0100 feat: Add HTTP proxy documentation As detailed here: https://camel.apache.org/manual/improving-the-documentation.html#_quick_partial_build <https://camel.apache.org/manual/improving-the-documentation.html#_quick_partial_build> once you are set up, it is quick and easy to do a partial local build of your changes. Admittedly I’m familiar with this, but it took me about 2 minutes to run the local build, fix the problem, rerun the local build, and make https://github.com/apache/camel-k/pull/2874 <https://github.com/apache/camel-k/pull/2874>. Please don’t break everything for everyone else by committing doc changes without verifying that they work. If you have problems setting this up, please let me know what they are rather than ignoring the possibility of stopping everyone else from working on docs. It would be great to eventually have a PR check for doc changes, and it might be within reach, but it involves some communication between repos and GitHub workflows, so it’s not trivial to set up. Would you commit java or go code that doesn’t compile? Don’t commit AsciiDoc that doesn’t compile either. Thanks David Jencks
Re: Is camel-kamelets 0.6.0 release incomplete?
This is good but i realized eventually there may be a LTS kamelets release where we can’t just point to the branch. I suggest pointing to the tag instead, where the version is going to be always correct. PR soon (I hope). I think it would be possible to automate figuring out the branch/tag but this would require extracting the version from the pom and analyzing it: I’m not sure if it’s worth it, although having to set it during release seems like a big annoyance. Any thoughts? Thanks!! David Jencks > On Jan 7, 2022, at 12:54 AM, Andrea Cosentino wrote: > > To harmonize with 0.5.x and since there won't be any 0.6.x release anymore > I'll change to 0.6.0 in the end. > > Il giorno ven 7 gen 2022 alle ore 09:18 Andrea Cosentino > ha scritto: > >> About the kamelet.yaml: >> >> In the tag released the version is 0.6.0 >> >> >> https://github.com/apache/camel-kamelets/blob/v0.6.0/kamelets/aws-redshift-sink.kamelet.yaml >> >> And even the library are pointing to a released version >> >> >> https://github.com/apache/camel-kamelets/blob/v0.6.0/kamelets/extract-field-action.kamelet.yaml#L42 >> >> I could change the SNAPSHOT version to 0.6.0, but I don't see why, in the >> end the 0.6.x may be a development branch pointing to something still to be >> release, the important part is the tag and the released code which it's >> still pointing to 0.6.0 >> >> The released version is pointing to released code and you could see this >> by looking at the uploaded jars or into the tag. >> >> Il giorno ven 7 gen 2022 alle ore 08:52 David Jencks < >> david.a.jen...@gmail.com> ha scritto: >> >>> Thanks! >>> >>> I think there are still a couple of problems, though: >>> >>> - the versions in the kamelet yaml are snapshots >>> - the source links are pointing to main >>> >>> As a result, we’re pointing to unreleased code from a released version, >>> and even fixing the 2nd point won’t help. The dependencies in 0.5.x look >>> like `- "mvn:org.apache.camel.kamelets:camel-kamelets-utils:0.5.0”` which >>> seems more correct to me. >>> >>> David Jencks >>> >>>> On Jan 6, 2022, at 11:08 PM, Andrea Cosentino >>> wrote: >>>> >>>> Should be ok, now. >>>> >>>> Il giorno ven 7 gen 2022 alle ore 08:03 Andrea Cosentino < >>> anco...@gmail.com> >>>> ha scritto: >>>> >>>>> There won't be any new release of 0.6.x by the way. >>>>> >>>>> Il giorno ven 7 gen 2022 alle ore 08:02 Andrea Cosentino < >>>>> anco...@gmail.com> ha scritto: >>>>> >>>>>> I still need to align the branch, but the release is complete. Using >>>>>> main-SNAPSHOT is not a problem, it's just a descriptor, by the way to >>> avoid >>>>>> confusion I'll align there. >>>>>> >>>>>> Il giorno ven 7 gen 2022 alle ore 07:58 David Jencks < >>>>>> david.a.jen...@gmail.com> ha scritto: >>>>>> >>>>>>> The source tar.gz download looks fine to me, but the GitHub branch >>> poms >>>>>>> (e.g. https://github.com/apache/camel-kamelets/blob/0.6.x/pom.xml < >>>>>>> https://github.com/apache/camel-kamelets/blob/0.6.x/pom.xml>) have >>>>>>> version main-SNAPSHOT instead of 0.6.1-SNAPSHOT (or whatever is >>> correct), >>>>>>> and the dependencies in the kamelet yamls also have main-SNAPSHOT >>> for the >>>>>>> util version. >>>>>>> >>>>>>> David Jencks >>>>>> >>>>>> >>> >>>
Re: Is camel-kamelets 0.6.0 release incomplete?
Thanks! I think there are still a couple of problems, though: - the versions in the kamelet yaml are snapshots - the source links are pointing to main As a result, we’re pointing to unreleased code from a released version, and even fixing the 2nd point won’t help. The dependencies in 0.5.x look like `- "mvn:org.apache.camel.kamelets:camel-kamelets-utils:0.5.0”` which seems more correct to me. David Jencks > On Jan 6, 2022, at 11:08 PM, Andrea Cosentino wrote: > > Should be ok, now. > > Il giorno ven 7 gen 2022 alle ore 08:03 Andrea Cosentino > ha scritto: > >> There won't be any new release of 0.6.x by the way. >> >> Il giorno ven 7 gen 2022 alle ore 08:02 Andrea Cosentino < >> anco...@gmail.com> ha scritto: >> >>> I still need to align the branch, but the release is complete. Using >>> main-SNAPSHOT is not a problem, it's just a descriptor, by the way to avoid >>> confusion I'll align there. >>> >>> Il giorno ven 7 gen 2022 alle ore 07:58 David Jencks < >>> david.a.jen...@gmail.com> ha scritto: >>> >>>> The source tar.gz download looks fine to me, but the GitHub branch poms >>>> (e.g. https://github.com/apache/camel-kamelets/blob/0.6.x/pom.xml < >>>> https://github.com/apache/camel-kamelets/blob/0.6.x/pom.xml>) have >>>> version main-SNAPSHOT instead of 0.6.1-SNAPSHOT (or whatever is correct), >>>> and the dependencies in the kamelet yamls also have main-SNAPSHOT for the >>>> util version. >>>> >>>> David Jencks >>> >>>
Is camel-kamelets 0.6.0 release incomplete?
The source tar.gz download looks fine to me, but the GitHub branch poms (e.g. https://github.com/apache/camel-kamelets/blob/0.6.x/pom.xml <https://github.com/apache/camel-kamelets/blob/0.6.x/pom.xml>) have version main-SNAPSHOT instead of 0.6.1-SNAPSHOT (or whatever is correct), and the dependencies in the kamelet yamls also have main-SNAPSHOT for the util version. David Jencks
Re: camel-kamelets release 0.6.0
I agree, and I would want to know the specific reason. For a security problem I’d expect the discussion and vote to be on the PMC list. David Jencks > On Dec 23, 2021, at 9:54 AM, Andrea Cosentino wrote: > > Sorry, pressed enter too fast > > https://lists.apache.org/thread/8rn1468ky9n1mnn4h4dkgvbpjjlzr6jf > > So I think, in particular situation we could cut down the time window for > specific reason, well explained. > > > > Il giorno gio 23 dic 2021 alle ore 18:54 Andrea Cosentino > ha scritto: > >> About the time window for a release, in particular in case of CVE or >> security issues, we could even release in hours. >> >> There is this discussion in member mailing list >> >> >> Il giorno lun 20 dic 2021 alle ore 04:01 David Jencks < >> david.a.jen...@gmail.com> ha scritto: >> >>> I have no problem whatsoever with you. Possibly since you have so much >>> to say about the project, much more than any other PMC chair I’ve >>> encountered, you run into my concerns more often. >>> >>> I have several motivations. >>> >>> Back when I was first involved in an Apache project, around 2004, the >>> board stepped in because they thought we weren’t following apache >>> policies. They could well have been correct, but the oversight was not >>> pleasant. I might be hypervigilant, but I really really don’t want that to >>> happen to Camel. I also follow the incubator list and have some idea what >>> new projects are expected to do. When I see something that looks to me >>> like it’s inconsistent with a policy or likely to provoke a correction if >>> Camel were in the incubator, I try to ask what’s going on. >>> >>> I’ve also found that at least for me having behavior consistent across >>> projects is much more welcoming to new contributors than having unusual >>> practices. Even though I’m a founding PMC member of Camel, I’ve only >>> started contributing quite recently when I realized I might be able to help >>> with the website. I definitely feel like a newcomer. When I find >>> something that seems confusing or unwelcoming, I try to ask about it: I >>> suspect that if I find it unwelcoming others might also. >>> >>> Finally, camel is pretty complex, and there are a lot of subprojects, and >>> I haven’t found documentation about how they are related. I’ve been slowly >>> trying to understand how they are related and find ways to make the >>> documentation reflect that more clearly. Sometimes I get confused, can’t >>> find the right place to look, or don’t look far enough, and ask…. perhaps >>> not always very politely. However, I’ve always already spent some time >>> investigating and am genuinely puzzled before I ask. >>> >>> Let’s look at an example… >>> >>> It’s an absolute policy that projects make it really clear that >>> non-released code is only for project development purposes. I don’t know to >>> what extent this is a firm policy for websites, and I’ve never seen it >>> discussed, perhaps because AFAIK no other projects have a multi-version >>> website capable of showing the docs for unreleased versions under >>> development, but I think it’s pretty obvious that having the default view >>> be of a released version is more consistent with this policy that having >>> the default be an unreleased dev version. I also think it’s much more >>> friendly to users to have the docs by default show you something it’s >>> appropriate for you to use. After several steps, I think the website is >>> now by default showing the docs for the latest released version of every >>> subproject rather than the unreleased dev version, and it displays some >>> indication that you shouldn’t be using the unreleased version. >>> >>> My recollection of the history here is… >>> - I noticed that the default was “latest” which was unreleased, and >>> thought that wasn’t very consistent with the “direct people to released >>> code only” policy, especially since there was no clear indication that the >>> docs referred to something unreleased. It was pretty easy to use Antora >>> facilities to mark it prerelease and to have the display version show >>> “prerelease” so at least there’s some indication of the status and Antora >>> would regard the latest released version as “latest" >>> - Eventually I realized we could use redirects to get the hugo portion of >>> the site to point by default to relea
Re: [Website] POC local/partial builds
Hi Zoran, inline after extracting a couple of concrete proposals. 1. I’m going to open a website issue and PR for publishing the site manifest with the site, to enable immediate quick local “xref check” builds without needing to build the full site. 2. I’m going to open a website issue proposing checking in the build UI bundle. This is a different solution to the problem than PR#537[1], and is certainly open to debate. 2. I’m going to open a website issue to move the site publishing to a separate repo 3. I’m going to start on a PR for the user manual to document how to do local builds. My comments on why below... > On Dec 21, 2021, at 1:55 AM, Zoran Regvart wrote: > > Hi David, > > On Mon, Dec 20, 2021 at 10:23 PM David Jencks > wrote: >> >> (Mostly general info, everyone please read) >> >> Hi Antonin, >> >> Sorry, somehow I missed this earlier! >> >> This work should help with that issue. I’m finding it really great for >> stuff I do, it’s also really easy to adapt to create a website PR for >> integrating lots of branch changes, cf. >> https://github.com/apache/camel-website/pull/729 >> <https://github.com/apache/camel-website/pull/729>. >> >> I’m going to be working on my “referential integrity” idea next (unless more >> documentation fires break out :-). This will touch every active branch of >> every subproject, and unless someone objects I’m going to install the >> local-build stuff everywhere to make it easier for myself :-), so it should >> soon be pretty easy for everyone to try it on their favorite parts of camel. > > +1 this is great work and it'll help us in the long run. I think > you're doing it carefully and conscientiously taking into account not > to be disruptive. Did we already update the README's, so folk > understand how to use `local-build.sh`? Not yet, but I think this is going well enough to assume we'll keep it. I think the best will be if I have detailed instructions in the “working on the website” page in the user manual and very short directions in each subproject. Would it be a good idea to make the local instructions named so they are obviously instructions, maybe README_local_build.adoc? I never know what to expect from a README :-) > >> It’s not a problem for me since I basically only work on the website, but I >> would imagine two major annoyances of the current system for everyone else >> are: >> >> (1) You have to build the full site at least once, which takes annoyingly >> long (7 minutes?). >> (2) You have to build the UI locally, which is tricky on non-linux systems. >> >> For (1), building the site manifest as part of the regular website build so >> the latest can always be downloaded would make it so you can always do a >> quick local partial build, although you won’t get a very usable local >> website: just the local stuff will be there locally. It would detect the >> same errors as a the current local build, but not xref problems into the >> local changes: this (currently) requires a full build. > > Still, I think this is a great step forward, we can detect some issues > early and quickly, I'm sure we can improve to detect most issues in > subsequent refinements I think there are 3 cases for local builds: 1. You just want to check for errors as quickly as possible. I think this is what most people want to do most of the time. We might even be able to get maven to run this. Having the site manifest served from the live site (or some other location) is needed for this. 2. You are actually working on the docs and want a quick continuous build of a subproject, and don’t mind having to do a full build first (maybe 7-8 minutes). This is what the current `./local-build.sh` does, after a `./local-build.sh full`. This detects xref problems out of the subproject but not into the subproject. 3. You want to check for all xref and other problems with your local changes. This is what the current `.local-build.sh full` does. We don’t support (1) yet, likely by far the most common case. So, I propose we start by publishing the site manifest. I can’t see any downside…. we can always stop. > >> For (2), one simple option is just checking the built UI bundle into git. >> >> A further annoyance that I experience is that the built website is checked >> into a branch of camel-website. This means whenever you pull camel-website, >> you get the latest published website which is a lot of changes, along with >> the tiny changes that might or might not have occurred since your last >> update. > > Nicola touched upon this in PR#537[1], would that be a better option? #537 is about publishing the UI bundle,
Re: [Website] POC local/partial builds
(Mostly general info, everyone please read) Hi Antonin, Sorry, somehow I missed this earlier! This work should help with that issue. I’m finding it really great for stuff I do, it’s also really easy to adapt to create a website PR for integrating lots of branch changes, cf. https://github.com/apache/camel-website/pull/729 <https://github.com/apache/camel-website/pull/729>. I’m going to be working on my “referential integrity” idea next (unless more documentation fires break out :-). This will touch every active branch of every subproject, and unless someone objects I’m going to install the local-build stuff everywhere to make it easier for myself :-), so it should soon be pretty easy for everyone to try it on their favorite parts of camel. It’s not a problem for me since I basically only work on the website, but I would imagine two major annoyances of the current system for everyone else are: (1) You have to build the full site at least once, which takes annoyingly long (7 minutes?). (2) You have to build the UI locally, which is tricky on non-linux systems. For (1), building the site manifest as part of the regular website build so the latest can always be downloaded would make it so you can always do a quick local partial build, although you won’t get a very usable local website: just the local stuff will be there locally. It would detect the same errors as a the current local build, but not xref problems into the local changes: this (currently) requires a full build. For (2), one simple option is just checking the built UI bundle into git. A further annoyance that I experience is that the built website is checked into a branch of camel-website. This means whenever you pull camel-website, you get the latest published website which is a lot of changes, along with the tiny changes that might or might not have occurred since your last update. I would like us to move the published website to a different repo. This would also allow us to keep the website history, which would partly solve the problem of docs for outdated component versions that we’ve dropped from the active website. I also think from some comments about the old svn site publishing system that infra expects website history to be preserved, although I haven’t asked explicitly. I’ve set this up in a couple other projects, and it’s pretty easy to do. David Jencks > On Dec 17, 2021, at 1:44 AM, Antonin Stefanutti > wrote: > > Hi David, > > Thanks a lot for your initiative. I hesitated to comment, both because I'm > definitely foreign to the doc building process, and also as you did not > mention Camel K. However, this, if I understand it correctly would help > solving the following issue: > > https://github.com/apache/camel-k/issues/2278 > <https://github.com/apache/camel-k/issues/2278> > > I think this is critical to have a way to validate doc updates made to > sub-projects before merging so as to make sure they do not fail the main > website build. > > As I said, I'm not familiar with the doc build process, but I'd be happy to > help you on this if there is anything I can do. > > On 17 Dec 2021, at 00:46, David Jencks <mailto:david.a.jen...@gmail.com><mailto:david.a.jen...@gmail.com > <mailto:david.a.jen...@gmail.com>>> wrote: > > Although there have been some comments on the associated PRs, and I’ve merged > some of them, I’m slightly disappointed that there has been no discussion > here, although there have been complaints about one of the problems this is > intended to help with (broken website builds). > > I mention in future work (3) that it might be possible to adapt some of this > for docs PR builds. Here’s what I have in mind: > > 1. Something in the CI detects that there’s a change impacting the docs (or > we just run this for every PR). > > 2. CI for the PR extracts the main repo, the PR repo, the original branch, > and the PR branch and sends them (how?) to camel-website CI. > > 3. CI for camel-website takes this information and constructs a snippet like > this: > > ``` > - require: '@djencks/antora-source-map' > source-map: > - url: ‘$ORIGINAL_REPO' > mapped-url: ‘$PR_REPO' > branches: > - branch: $ORIGINAL_BRANCH > mapped-branch: $PR_BRANCH > ``` > > This gets appended to the antora playbook and the antora build runs. (or a > full build) > > To make it quicker, if a current site-manifest is available, we could also > append something for @djencks/antora-source-watch and > @djencks/antora-site-manifest to only build the stuff from the changed repo > (although this can often be tricky due to interrelationships between > projects, e.g. camel and camel-spring-boot). > > Conceptua
Re: camel-kamelets release 0.6.0
I have no problem whatsoever with you. Possibly since you have so much to say about the project, much more than any other PMC chair I’ve encountered, you run into my concerns more often. I have several motivations. Back when I was first involved in an Apache project, around 2004, the board stepped in because they thought we weren’t following apache policies. They could well have been correct, but the oversight was not pleasant. I might be hypervigilant, but I really really don’t want that to happen to Camel. I also follow the incubator list and have some idea what new projects are expected to do. When I see something that looks to me like it’s inconsistent with a policy or likely to provoke a correction if Camel were in the incubator, I try to ask what’s going on. I’ve also found that at least for me having behavior consistent across projects is much more welcoming to new contributors than having unusual practices. Even though I’m a founding PMC member of Camel, I’ve only started contributing quite recently when I realized I might be able to help with the website. I definitely feel like a newcomer. When I find something that seems confusing or unwelcoming, I try to ask about it: I suspect that if I find it unwelcoming others might also. Finally, camel is pretty complex, and there are a lot of subprojects, and I haven’t found documentation about how they are related. I’ve been slowly trying to understand how they are related and find ways to make the documentation reflect that more clearly. Sometimes I get confused, can’t find the right place to look, or don’t look far enough, and ask…. perhaps not always very politely. However, I’ve always already spent some time investigating and am genuinely puzzled before I ask. Let’s look at an example… It’s an absolute policy that projects make it really clear that non-released code is only for project development purposes. I don’t know to what extent this is a firm policy for websites, and I’ve never seen it discussed, perhaps because AFAIK no other projects have a multi-version website capable of showing the docs for unreleased versions under development, but I think it’s pretty obvious that having the default view be of a released version is more consistent with this policy that having the default be an unreleased dev version. I also think it’s much more friendly to users to have the docs by default show you something it’s appropriate for you to use. After several steps, I think the website is now by default showing the docs for the latest released version of every subproject rather than the unreleased dev version, and it displays some indication that you shouldn’t be using the unreleased version. My recollection of the history here is… - I noticed that the default was “latest” which was unreleased, and thought that wasn’t very consistent with the “direct people to released code only” policy, especially since there was no clear indication that the docs referred to something unreleased. It was pretty easy to use Antora facilities to mark it prerelease and to have the display version show “prerelease” so at least there’s some indication of the status and Antora would regard the latest released version as “latest" - Eventually I realized we could use redirects to get the hugo portion of the site to point by default to released versions. - I noticed there was no released version of kamelets in the docs and tried to find out why…. should no one be using kamelets???… there’s no documentation that they have ever been released!!! I wasn’t very successful at deciphering the rather inconsistent state of the website or finding the vote emails, and asked, perhaps with a bit too much panic. I think we’ve mostly straightened this out now. Thanks David Jencks > On Dec 19, 2021, at 11:33 AM, Andrea Cosentino wrote: > > It looks to me you have something personal with me, because each time I > write something you suddenly appear and ask why, reporting rules.. what is > exactly your problem with me? > > Il dom 19 dic 2021, 20:26 Andrea Cosentino ha scritto: > >> And btw i know the ASF rules better than you. To me 72 hours doesn't make >> any sense and I'm trying to use my brain instead of reporting a well known >> rule. >> >> Il dom 19 dic 2021, 20:23 Andrea Cosentino ha scritto: >> >>> For any other problem you can write to the board. >>> >>> Il dom 19 dic 2021, 20:21 Andrea Cosentino ha >>> scritto: >>> >>>> What's the reason for having 72 hours of release window if this is not >>>> an important release but just a middle release? I don't see good reason, >>>> just a rule 20 years old, that should be revisited. >>>> >>>> Btw, David, it looks to me you're trying to create problems here. The >>>> vote will be for 72
Re: camel-kamelets release 0.6.0
https://www.apache.org/legal/release-policy.html#release-approval <https://www.apache.org/legal/release-policy.html#release-approval> I think it’s entirely appropriate that I ask why you are not following the well known: `Release votes SHOULD remain open for at least 72 hours.` To me this means that any shorter release vote needs a good justification. What does it mean to you? You say it’s not a critical release, so what’s the hurry? David Jencks > On Dec 19, 2021, at 9:40 AM, Andrea Cosentino wrote: > > It's not a critical release i meant to say > > Il dom 19 dic 2021, 18:36 Andrea Cosentino ha scritto: > >> It's a critical release and we already done it in the past. If you any >> troubles we could extend to 72 hours. But i don't see why. For some sub >> projects we use 48 hours. >> >> It looks to me you're looking for problems where they don't exist. >> >> Let's do 72 hours. I don't want complaints. >> >> Il dom 19 dic 2021, 18:29 David Jencks ha >> scritto: >> >>> What justifies the shorter-than-standard-72-hours voting window? >>> >>> David Jencks >>> >>>> On Dec 19, 2021, at 1:27 AM, Andrea Cosentino >>> wrote: >>>> >>>> I'll release tomorrow morning and open a vote for 48 hours, since this >>> will >>>> be outside camel-k. >>>> >>>> Il sab 18 dic 2021, 10:05 Claus Ibsen ha >>> scritto: >>>> >>>>> Hi >>>>> >>>>> I think it would be good to get a new release of kamelets that works >>>>> well with the new Camel 3.14.0 release. >>>>> >>>>> We did some updates to the yaml-dsl in 3.14 that requires a new >>>>> release of kamelets to work. >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Claus Ibsen >>>>> - >>>>> http://davsclaus.com @davsclaus >>>>> Camel in Action 2: https://www.manning.com/ibsen2 >>>>> >>> >>>
Re: camel-kamelets release 0.6.0
What justifies the shorter-than-standard-72-hours voting window? David Jencks > On Dec 19, 2021, at 1:27 AM, Andrea Cosentino wrote: > > I'll release tomorrow morning and open a vote for 48 hours, since this will > be outside camel-k. > > Il sab 18 dic 2021, 10:05 Claus Ibsen ha scritto: > >> Hi >> >> I think it would be good to get a new release of kamelets that works >> well with the new Camel 3.14.0 release. >> >> We did some updates to the yaml-dsl in 3.14 that requires a new >> release of kamelets to work. >> >> >> >> >> -- >> Claus Ibsen >> - >> http://davsclaus.com @davsclaus >> Camel in Action 2: https://www.manning.com/ibsen2 >>
Re: Issue preparing Camel 3.7.7 release
I don’t recall which subprojects or branches I’ve had problems with, but I’ve had enough problems that I now expect that a full maven build (skipping tests) of a non-main camel subproject branch will fail. Should I complain when this happens? David Jencks > On Dec 17, 2021, at 7:48 AM, Andrea Cosentino wrote: > > You need to fully build the branch each time. To be sure everything it's > installed, probably you're building with fastinstall profile > > Il ven 17 dic 2021, 16:44 David Jencks ha > scritto: > >> Yikes, we should not be trying to use the xref validator in any >> circumstances. I’ll look into this shortly unless someone beats me to it. >> >> Has anyone built the 3.7.x branch successfully recently? I have often >> found that I cannot (mvn) build non-main branches of camel subprojects >> successfull, often it seems that snapshots have not been deployed and all >> sorts of other problems have occurred. I’ve wondered if there is any CI or >> periodic build activity for these branches. >> >> David Jencks >> >>> On Dec 17, 2021, at 7:34 AM, Gregor Zurowski >> wrote: >>> >>> Hi Everyone: >>> >>> I am trying to cut the release candidate for Camel 3.7.7 but running >>> into the following issue towards the very end of the `release:prepare` >>> phase: >>> >>> ``` >>> [INFO] [INFO] ---< org.apache.camel:docs >>>> >>> [INFO] [INFO] Building Camel :: Docs 3.7.7 >>> [505/506] >>> [INFO] [INFO] [ pom >>> ]- >>> >>> [...] >>> >>> [INFO] [ERROR] error >>> >> https://gitlab.com/antora/xref-validator/repository/archive.tar.gz?ref=84f28a20b8be888b7664a4540cda5aca56de0824 >> : >>> Extracting tar content of undefined failed, the file appears to be >>> corrupt: "Invalid tar header. Maybe the tar is corrupted or it needs >>> to be gunzipped?" >>> [INFO] [INFO] info Visit https://yarnpkg.com/en/docs/cli/install for >>> documentation about this command. >>> ``` >>> >>> The summary contains the following error message: >>> >>> ``` >>> [INFO] [ERROR] Failed to execute goal >>> com.github.eirslett:frontend-maven-plugin:1.6:yarn (yarn-install) on >>> project docs: Failed to run task: 'yarn install --no-progress --force >>> --non-interactive --frozen-lockfile' failed. >>> org.apache.commons.exec.ExecuteException: Process exited with an >>> error: 1 (Exit value: 1) -> [Help 1] >>> ``` >>> >>> I can't access the gitlab hosted archive, instead it's redirecting >>> (HTTP 302) to Gitlab's login page. >>> >>> Any ideas about this error? >>> >>> Thanks in advance, >>> Gregor >> >>
Re: Issue preparing Camel 3.7.7 release
Yikes, we should not be trying to use the xref validator in any circumstances. I’ll look into this shortly unless someone beats me to it. Has anyone built the 3.7.x branch successfully recently? I have often found that I cannot (mvn) build non-main branches of camel subprojects successfull, often it seems that snapshots have not been deployed and all sorts of other problems have occurred. I’ve wondered if there is any CI or periodic build activity for these branches. David Jencks > On Dec 17, 2021, at 7:34 AM, Gregor Zurowski wrote: > > Hi Everyone: > > I am trying to cut the release candidate for Camel 3.7.7 but running > into the following issue towards the very end of the `release:prepare` > phase: > > ``` > [INFO] [INFO] ---< org.apache.camel:docs >> > [INFO] [INFO] Building Camel :: Docs 3.7.7 > [505/506] > [INFO] [INFO] [ pom > ]- > > [...] > > [INFO] [ERROR] error > https://gitlab.com/antora/xref-validator/repository/archive.tar.gz?ref=84f28a20b8be888b7664a4540cda5aca56de0824: > Extracting tar content of undefined failed, the file appears to be > corrupt: "Invalid tar header. Maybe the tar is corrupted or it needs > to be gunzipped?" > [INFO] [INFO] info Visit https://yarnpkg.com/en/docs/cli/install for > documentation about this command. > ``` > > The summary contains the following error message: > > ``` > [INFO] [ERROR] Failed to execute goal > com.github.eirslett:frontend-maven-plugin:1.6:yarn (yarn-install) on > project docs: Failed to run task: 'yarn install --no-progress --force > --non-interactive --frozen-lockfile' failed. > org.apache.commons.exec.ExecuteException: Process exited with an > error: 1 (Exit value: 1) -> [Help 1] > ``` > > I can't access the gitlab hosted archive, instead it's redirecting > (HTTP 302) to Gitlab's login page. > > Any ideas about this error? > > Thanks in advance, > Gregor
Re: Camel Release | Website Build Issues
FWIW I’m working on a local build setup for (main) camel and discovered some
gaps in @djencks/antora-source-watch exposed by the `components` component
being distributed across `camel` and `camel-spring-boot`. Perhaps it will be
ready tomorrow.
I’d find it much easier to take complaints about the website build seriously if
there was active consideration and discussion around my attempts to improve the
situation. It’s been a couple days since my recent email on the subject with no
responses that I can see. In particular future(1) would probably mean a local
build of a subproject to check for errors could be done with no preparation
other than having camel-website cloned appropriately, node installed, and take
about a minute. It would be great to have a CI check as well, but this or even
the local builds I’m working on now would be far better than the current
situation where I think no one knows about the instructions
https://camel.apache.org/manual/improving-the-documentation.html#_local_build_instructions
<https://camel.apache.org/manual/improving-the-documentation.html#_local_build_instructions>
and they are far too hard to follow.
David Jencks
> On Dec 16, 2021, at 2:46 PM, David Jencks wrote:
>
> I followed the link Gregor provided and see this:
>
> ➤ YN: [18:59:31.230] WARN (asciidoctor): skipping reference to missing
> attribute: headername
> ➤ YN: file:
> docs/components/modules/ROOT/pages/salesforce-component.adoc
> ➤ YN: source: https://github.com/apache/camel.git
> <https://github.com/apache/camel.git> (refname: main, start path:
> docs/components)
> ➤ YN: [18:59:31.235] WARN (asciidoctor): skipping reference to missing
> attribute: id
> ➤ YN: file:
> docs/components/modules/ROOT/pages/salesforce-component.adoc
> ➤ YN: source: https://github.com/apache/camel.git
> <https://github.com/apache/camel.git> (refname: main, start path:
> docs/components)
>
>
> I’m not sure how much more clear the build could possibly be about what is
> wrong???
>
> Back before Antora 3 these errors would still be present but not result in a
> failed build. You could look at the console output, but no one ever did.
>
> To fix:
>
> - the indicated file is a symlink, so follow it to it’s source
> components/camel-salesforce/camel-salesforce-component/src/main/docs/salesforce-component.adoc
>
> - look at git log to see who to blame….
>
> - find the newly added non-attributes {header name} and {id} and escape them
>
> ...
>
> https://github.com/apache/camel/pull/6550
> <https://github.com/apache/camel/pull/6550>
>
> It’s not nearly easy enough to do a build to see if you’ve broken the site,
> but my POC makes it at least plausible.
>
> Browser plugins and the Intellij asciidoc plugin are not an adequate
> replacement for building the site.
>
> I suggest that no one change an adoc file without at least checking the
> antora build locally. Without local checks and PR builds we can expect the
> website to be always broken. Unfortunately I still don’t know of an
> easy-enough way to have PR builds.
>
> David Jencks
>
>> On Dec 16, 2021, at 11:38 AM, Andrea Cosentino > <mailto:anco...@gmail.com>> wrote:
>>
>> To be clear, I really appreciate the work done on docs and website, but
>> it's really frustrating to get a different error each time you try to have
>> a green ci build.
>>
>> Il gio 16 dic 2021, 20:36 Andrea Cosentino > <mailto:anco...@gmail.com>> ha scritto:
>>
>>> Thanks Gregor for spending your time on this.
>>>
>>> I would like to stress once again how much it's becoming non-deterministic
>>> to build the website.
>>>
>>> This is becoming everyday more complicated than it should be.
>>>
>>> I think we need to focus on this. Because we shouldn't postpone a release
>>> for reason like this.
>>>
>>> Building the website should be something like black magic and voodoo
>>> invocation. It should work much more easily.
>>>
>>> I'll start another dev discussion about this argument.
>>>
>>> Il gio 16 dic 2021, 20:17 Gregor Zurowski >> <mailto:gre...@list.zurowski.org>> ha
>>> scritto:
>>>
>>>> Hi Everyone:
>>>>
>>>> I wanted to finish up the steps necessary to release Camel 3.14.0, but
>>>> I am running into issues with the website build:
>>>>
>>>> ```
>>>> YN: ERROR: "build:antora" exited with 1.
>>>> YN: The command failed for workspaces that are depended upon by
>>>> other workspaces; can't satisfy the dependency graph
>>>> ```
>>>> See:
>>>> https://ci-builds.apache.org/job/Camel/job/Camel.website/job/main/630/console
>>>>
>>>> <https://ci-builds.apache.org/job/Camel/job/Camel.website/job/main/630/console>
>>>>
>>>> Any ideas why that's happening?
>>>>
>>>> Thanks in advance
>>>> Gregor
>>>>
>>>
>
Re: [Website] POC local/partial builds
Although there have been some comments on the associated PRs, and I’ve merged some of them, I’m slightly disappointed that there has been no discussion here, although there have been complaints about one of the problems this is intended to help with (broken website builds). I mention in future work (3) that it might be possible to adapt some of this for docs PR builds. Here’s what I have in mind: 1. Something in the CI detects that there’s a change impacting the docs (or we just run this for every PR). 2. CI for the PR extracts the main repo, the PR repo, the original branch, and the PR branch and sends them (how?) to camel-website CI. 3. CI for camel-website takes this information and constructs a snippet like this: ``` - require: '@djencks/antora-source-map' source-map: - url: ‘$ORIGINAL_REPO' mapped-url: ‘$PR_REPO' branches: - branch: $ORIGINAL_BRANCH mapped-branch: $PR_BRANCH ``` This gets appended to the antora playbook and the antora build runs. (or a full build) To make it quicker, if a current site-manifest is available, we could also append something for @djencks/antora-source-watch and @djencks/antora-site-manifest to only build the stuff from the changed repo (although this can often be tricky due to interrelationships between projects, e.g. camel and camel-spring-boot). Conceptually this seems pretty simple, but I don’t know enough about GitHub actions to know whether it can be done or how to do it. I’ve found the GitHub actions docs hard to understand. If anyone wants to implement this or provide advice about how to set up the information transfer and triggers that would make it much more likely to happen. David Jencks > On Dec 14, 2021, at 6:38 PM, David Jencks wrote: > > I have a POC for a reliable way to locally build small parts of the > (Antora-built) website, e.g. a subproject. > > Prerequisites: > > 1. camel-website must be cloned next to the subproject: e.g. if the “main” > directory is camel-projects, you’ll have camel-projects/camel-website and > camel-projects/camel-quarkus. > Note: it would be possible to have camel-website under the subproject, but > that would make working on more than one subproject implausible. > > 2. You’re using linux, macOS, or another unix-like OS…. there’s a shell > script. > > What to do: > > 1. Initially build the camel ui. This is easy on linux systems and difficult > on macOS; you have to run yarn to recompile some dependencies. It may be > easier to build in docker. > > 2. In the subproject, initially run `./loca-build.sh full` once to perform a > full Antora build that includes your local subproject state, to generate the > ‘site-manifest’ needed for the partial build. > > 3. After this, run `./local-build.sh` to continuously update the full build > with changes in the subproject. This starts a local web server which is > slightly more capable than looking at the file urls. > This does an initial partial build, and then watches for file changes. The > whole site is viewable. > > Limitations: > > 1. At the moment, xref checks from the subproject to the rest of the site are > validated correctly, but xrefs into the subproject from the rest of the site > are not. I’m hoping to extend the site-manifest to fix this. > > 2. You have to do a full build locally once to construct the site manifest > and the local content from the parts of the site outside the subproject. This > takes about 7 minutes on my laptop. If there are significant changes to the > website you need to redo this local build, and there’s no way to detect when > this is needed. > > 3. You have to build the camel ui once which is difficult on non-linux > systems. This could easily be fixed by checking in the built ui bundle > whenever it changes. > > Possible changes/future work: > > 1. The site-manifest can be built at minimal cost during the official website > build, and served from the website. It’s about 2.5 MB, so I don’t think it’s > a significant drain on resources. Doing just this would eliminate the need > for step (2), building the full site locally, and you’d still have good xref > checks, but with just this step you’d get a site where only the subproject > pages are local, and links out of the subproject go to the main site; links > into the subproject would stay on the main site. I don’t like this much, but > perhaps it’s completely adequate for most site development. > > 2. In addition to (1), the zip-publisher could be added to the build, to zip > up the whole Antora part of the site; this could be downloaded instead of > step (2). This would be pretty easy to set up, but the resulting zip is about > 110MB. I don’t know what infra would think of
Re: Camel Release | Website Build Issues
I followed the link Gregor provided and see this:
➤ YN: [18:59:31.230] WARN (asciidoctor): skipping reference to missing
attribute: headername
➤ YN: file: docs/components/modules/ROOT/pages/salesforce-component.adoc
➤ YN: source: https://github.com/apache/camel.git
<https://github.com/apache/camel.git> (refname: main, start path:
docs/components)
➤ YN: [18:59:31.235] WARN (asciidoctor): skipping reference to missing
attribute: id
➤ YN: file: docs/components/modules/ROOT/pages/salesforce-component.adoc
➤ YN: source: https://github.com/apache/camel.git
<https://github.com/apache/camel.git> (refname: main, start path:
docs/components)
I’m not sure how much more clear the build could possibly be about what is
wrong???
Back before Antora 3 these errors would still be present but not result in a
failed build. You could look at the console output, but no one ever did.
To fix:
- the indicated file is a symlink, so follow it to it’s source
components/camel-salesforce/camel-salesforce-component/src/main/docs/salesforce-component.adoc
- look at git log to see who to blame….
- find the newly added non-attributes {header name} and {id} and escape them
...
https://github.com/apache/camel/pull/6550
<https://github.com/apache/camel/pull/6550>
It’s not nearly easy enough to do a build to see if you’ve broken the site, but
my POC makes it at least plausible.
Browser plugins and the Intellij asciidoc plugin are not an adequate
replacement for building the site.
I suggest that no one change an adoc file without at least checking the antora
build locally. Without local checks and PR builds we can expect the website to
be always broken. Unfortunately I still don’t know of an easy-enough way to
have PR builds.
David Jencks
> On Dec 16, 2021, at 11:38 AM, Andrea Cosentino wrote:
>
> To be clear, I really appreciate the work done on docs and website, but
> it's really frustrating to get a different error each time you try to have
> a green ci build.
>
> Il gio 16 dic 2021, 20:36 Andrea Cosentino ha scritto:
>
>> Thanks Gregor for spending your time on this.
>>
>> I would like to stress once again how much it's becoming non-deterministic
>> to build the website.
>>
>> This is becoming everyday more complicated than it should be.
>>
>> I think we need to focus on this. Because we shouldn't postpone a release
>> for reason like this.
>>
>> Building the website should be something like black magic and voodoo
>> invocation. It should work much more easily.
>>
>> I'll start another dev discussion about this argument.
>>
>> Il gio 16 dic 2021, 20:17 Gregor Zurowski ha
>> scritto:
>>
>>> Hi Everyone:
>>>
>>> I wanted to finish up the steps necessary to release Camel 3.14.0, but
>>> I am running into issues with the website build:
>>>
>>> ```
>>> YN: ERROR: "build:antora" exited with 1.
>>> YN: The command failed for workspaces that are depended upon by
>>> other workspaces; can't satisfy the dependency graph
>>> ```
>>> See:
>>> https://ci-builds.apache.org/job/Camel/job/Camel.website/job/main/630/console
>>>
>>> Any ideas why that's happening?
>>>
>>> Thanks in advance
>>> Gregor
>>>
>>
[Website] POC local/partial builds
tions to the main Antora playbook to result in local playbooks for full and partial builds, and runs the builds. Where is it?!?!? Camel-website issue: https://github.com/apache/camel-website/issues/720 <https://github.com/apache/camel-website/issues/720> camel-website PR https://github.com/apache/camel-website/pull/721 <https://github.com/apache/camel-website/pull/721> camel-quarkus PR https://github.com/apache/camel-quarkus/pull/3385 <https://github.com/apache/camel-quarkus/pull/3385> camel-kamelets PR https://github.com/apache/camel-kamelets/pull/630 <https://github.com/apache/camel-kamelets/pull/630> I think you can try this out by making sure your clones are next to one another as in prerequisite (1), pulling down my branches, and following the instructions. Comments? David Jencks
Re: Draft Camel Board Report For December 21
Minor points… - There were some camel-k-runtime releases as well, and possibly others I’m not aware of. - Camel releases had synchronized camel-spring-boot and camel-karaf releases - Possibly it would be clearer to put the JIRA issue count and GitHub issue count next to one another and mention that JIRA issues are used for camel/camel-karaf/camel-spring-boot and GitHub issues are used for all other subprojects. You could perhaps also combine the issue counts. Also typo noted inline. David Jencks > On Dec 5, 2021, at 10:38 PM, Andrea Cosentino wrote: > > Hello, > > Here is the draft for December board report. I'm planning to submit it this > afternoon, so if you have feedback please let me know today. Thanks. > > ## Description: > The mission of Apache Camel is the creation and maintenance of an > open-source > integration framework based on known Enterprise Integration Patterns. > > ## Issues: > There are no issues requiring board attention at this time. > > ## Membership Data: > Apache Camel was founded 2008-12-17 (13 years ago) > There are currently 79 committers and 38 PMC members in this project. > The Committer-to-PMC ratio is roughly 5:3. > > Community changes, past quarter: > - No new PMC members. Last addition was James Netherton on 2021-04-12. > - No new committers. Last addition was Marat Gubaidullin on 2021-07-29. > > ## Project Activity: > - We released Camel 3.7.6 > - We released Camel 3.11.2 > - We released Camel 3.11.3 > - We released Camel 3.11.4 > - We released Camel 3.12.0 > - We released Camel 3.13.0 > - We released 3.11.0 release: this releases train is the LTS release with > 3.7.x. We're going to stop releasing 3.7.x releases and going ahead with > two > LTS 3.11.x and 3.14.x. > - We are going to release 3.14.0 in the Middle of December: this release > will > be the last supporting Java 8, so we're going to support it for 2 years > instead of 1. intermediate development releases. > - We released Camel K 1.6.1 > - We released Camel K 1.7.0 > - We are improving the Camel-K experience and we are expanding and improving > the Kamelet concept, by introducing more Kameletes to the provided typo Kameletes >> Kamelets > catalog. > The amount of Kamelets is increasing in a really impressive way. > - The Camel-Quarkus work is going ahead following the main camel releases > with > multiple releases > - We released Camel-quarkus 2.3.0 > - We released Camel-quarkus 2.4.0 > - We released Camel-quarkus 2.5.0 > - We are continuing working on the Camel-kafka-connector for basing it on > Kamelet concept > - 0.11.0 on 21 Sep 2021 > - We are going to release 1.0 based on 3.14.0 LTS soon, probably in > January. > - We added two more projects to the ecosystem > - https://github.com/apache/camel-kameleon > - https://github.com/apache/camel-karavan > > ## Community Health: > - dev@camel.apache.org had a 45% increase in traffic in the past quarter > (501 > emails compared to 344): This increase is related to 3.14.x LTS release > and > migration effort and some camel-k discussion. > - iss...@camel.apache.org had a 58% increase in traffic in the past quarter > (2360 emails compared to 1488): We are working a lot on JIRA to clean up > and > setup everything related to 3.14.0 release, also we have some issues and > discussion around the website. > - us...@camel.apache.org had a 9% increase in traffic in the past quarter > (326 emails compared to 299): the situation is more or less the same of > the > last quarter, the increase is probably related to users moving from Camel > 2.x and 3.x > - 212 issues opened in JIRA, past quarter (-42% decrease) and 209 issues > closed in JIRA, past quarter (-42% decrease): as reported above, the > situation is stable. The activity related to Camel 2 is near to zero and > we > are focusing on old opened issues. We are getting feedback about new LTS > releases, but many of them are coming from other channel like the zulip > chat. > - 3122 commits in the past quarter (7% decrease) and 119 code contributors > in > the past quarter (4% decrease): the core Camel team is stabilizing the > codebase so there is a little decrease in number of commits, contributions > related to documentation and website is increasing. We are much more > focusing on giving to the community something stable with 3.14.0 and when > it > will be released we'll restart to innovate with the next dev releases. > - 875 PRs opened on GitHub, past quarter (-7% decrease) and 872 PRs closed > on > GitHub, past quarter (-8% decrease): the code stabilization and less work > on > the camel kafka connector side explain the decrease in number of PRs open > and closed. Came
Re: Referential integrity in the website, or, keeping cross-component xrefs working and accurate.
I opened https://github.com/apache/camel-website/issues/701…. now I just have
to do the work :-)
David Jencks
> On Nov 29, 2021, at 10:23 PM, Zoran Regvart wrote:
>
> +1 I think this is an awesome idea, not only it helps with the maintenance it
> provides a nice note for end users. I like this very much.
>
> zoran
> --
> Sent from mobile
>
>> On 29. Nov 2021, at 20:11, David Jencks wrote:
>>
>> Most or all of the camel subprojects depend on other subprojects.
>> For each subproject version, there are specific dependency versions.
>> Normally, in the documentation this will be expressed by xrefs from one
>> component only referring to the dependency version of the other component’s
>> docs.
>>
>> FWIW, every time I’ve looked at a subproject index page, I wonder, “what
>> does this depend on and at what versions?”.
>>
>> I think this should be expressed clearly and visibly in the documentation,
>> and in a way so that it is obvious if something’s missing or if a link is to
>> the wrong version.
>>
>> We address some of this in some antora.yml files with attributes for the
>> dependency versions, e.g.
>>
>> ```
>> asciidoc:
>> attributes:
>> camel-version: 3.13.x # or next? camel-quarkus next currently uses 3.13.x
>> camel-quarkus-version: next
>> camel-kamelets-version: next
>> ```
>>
>> Then xrefs can use the appropriate version with something like
>> xref:{camel-quarkus-version}@camel-quarkus:foo.adoc[].
>>
>> I propose:
>>
>> 1. Expanding this consistently to all subprojects and all dependencies.
>>
>> 2. Having a section documenting this and a few other things prominently on
>> each subproject index page. What I’ve come up with so far is
>> rendered in preview:
>>
>> ```
>> This version ({page-component-display-version}) of camel-k depends on:
>>
>> Camel
>>
>> at version 3.13.x
>> next@camel-quarkus::index.html
>>
>> at version next
>> next@camel-kamelets::index.html
>>
>> at version next
>> This is the development version of camel-k. It should not be used in
>> production.
>> ```
>>
>> source adoc:
>> ```
>> [NOTE]
>> --
>> This version ({page-component-display-version}) of camel-k depends on:
>>
>> * xref:{camel-version}@components::index.adoc[Camel] at version
>> {camel-version}
>> * xref:{camel-quarkus-version}@camel-quarkus::index.adoc[] at version
>> {camel-quarkus-version}
>> * xref:{camel-kamelets-version}@camel-kamelets::index.adoc[] at version
>> {camel-kamelets-version}
>>
>> ifdef::lts[This long term service release will be supported until {lts}.]
>> ifndef::lts[]
>> ifdef::prerelease[This is the development version of camel-k. It should not
>> be used in production.]
>> ifndef::prerelease[This release will not be updated, but rather replaced by
>> a new release.]
>> endif::[]
>> --
>> ```
>>
>> This specific proposal involves also adding an lts attribute with the date
>> out of service and a prerelease boolean attribute as appropriate.
>>
>> Having these explicit links will mean that, among other things, we can’t
>> remove a version of a component if it’s still being used by another
>> component version, and it will answer the most frequent question I have
>> looking around the docs.
>>
>> Thoughts?
>>
>> David Jencks
Sorting properties in kamelets?
I’ve come up with a way to generate the Kamelet pages in the Antora build directly from the kamelet yaml files. The one remaining problem is that the table of properties is not sorted. While I can probably find a way to add sorting capabilities, it occurs to me that it might be desirable to have the kamelet yaml in a more canonical form, with the properties already sorted properly. Is this a good idea? If so I’ll see about implementing it. David Jencks
Referential integrity in the website, or, keeping cross-component xrefs working and accurate.
Most or all of the camel subprojects depend on other subprojects.
For each subproject version, there are specific dependency versions. Normally,
in the documentation this will be expressed by xrefs from one component only
referring to the dependency version of the other component’s docs.
FWIW, every time I’ve looked at a subproject index page, I wonder, “what does
this depend on and at what versions?”.
I think this should be expressed clearly and visibly in the documentation, and
in a way so that it is obvious if something’s missing or if a link is to the
wrong version.
We address some of this in some antora.yml files with attributes for the
dependency versions, e.g.
```
asciidoc:
attributes:
camel-version: 3.13.x # or next? camel-quarkus next currently uses 3.13.x
camel-quarkus-version: next
camel-kamelets-version: next
```
Then xrefs can use the appropriate version with something like
xref:{camel-quarkus-version}@camel-quarkus:foo.adoc[].
I propose:
1. Expanding this consistently to all subprojects and all dependencies.
2. Having a section documenting this and a few other things prominently on each
subproject index page. What I’ve come up with so far is
rendered in preview:
```
This version ({page-component-display-version}) of camel-k depends on:
Camel
at version 3.13.x
next@camel-quarkus::index.html
at version next
next@camel-kamelets::index.html
at version next
This is the development version of camel-k. It should not be used in production.
```
source adoc:
```
[NOTE]
--
This version ({page-component-display-version}) of camel-k depends on:
* xref:{camel-version}@components::index.adoc[Camel] at version {camel-version}
* xref:{camel-quarkus-version}@camel-quarkus::index.adoc[] at version
{camel-quarkus-version}
* xref:{camel-kamelets-version}@camel-kamelets::index.adoc[] at version
{camel-kamelets-version}
ifdef::lts[This long term service release will be supported until {lts}.]
ifndef::lts[]
ifdef::prerelease[This is the development version of camel-k. It should not be
used in production.]
ifndef::prerelease[This release will not be updated, but rather replaced by a
new release.]
endif::[]
--
```
This specific proposal involves also adding an lts attribute with the date out
of service and a prerelease boolean attribute as appropriate.
Having these explicit links will mean that, among other things, we can’t remove
a version of a component if it’s still being used by another component version,
and it will answer the most frequent question I have looking around the docs.
Thoughts?
David Jencks
Re: [Website] activemq URI syntax inconsistency
Perhaps there are (at least) two issues here: 1. How many colons? The “grammar” activemq:destinationType:destinationName can’t possibly under any interpretation yield a sentence with one colon, so something like activemq:[destinationType:]destinationName is more correct. 2. What are the destinationTypes? The text in activemq.adoc says [queue|topic] but JmsEndpoint has ``` @UriPath(defaultValue = "queue", enums = "queue,topic,temp-queue,temp-topic", description = "The kind of destination to use") private String destinationType; ``` indicating 4 possibilities. —— Is the syntax string from the java code used programmatically, or is it purely documentation? Looking at several of these in the docs, I have some trouble telling what is a literal and what is a name of a path option. I would prefer the syntax expression to indicate optional path options and to distinguish between literals and options. So, for activemq it would look like activemq:[:] I wonder if it would be useful also to point out in the hand-written URI Format section that this is a simplification of what is completely described in the Endpoint Options section. I haven’t considered how hard it would be to update the syntax source and docs for all the components, but if it is agreed to be a good idea I’d consider it. David Jencks > On Nov 28, 2021, at 10:15 PM, Claus Ibsen wrote: > > Hi > > The correct syntax is in the json file, that are taken from the source code > > You are not correct about 2 colons, as when the option is left out (to > use its default value) then the colon is not needed either, so you > just use > > activemq:cheese > > when its a queue, and if you need topic then use > > activemq:topic:wines > > > > On Sun, Nov 28, 2021 at 6:58 PM David Jencks wrote: >> >> On the Activemq component page, there are two descriptions of the URI syntax: >> >> hardcoded in the .adoc source, >> https://camel.apache.org/components/next/activemq-component.html#_uri_format: >>activemq:[queue:|topic:]destinationName[?options] >> >> >> from the .json component.syntax property, shown at >> https://camel.apache.org/components/next/activemq-component.html#_endpoint_options: >> activemq:destinationType:destinationName >> >> In the immediately following table, the default for destinationType is >> listed as “queue”, so evidently it can be left out. >> >> The first syntax results in activemq:destinationName whereas the second >> results in activemq::destinationName (one vs two colons). >> >> - Which is correct? >> - If the first, should the component.syntax property be >> activemq:[destinationType:]destinationName ? >> >> In any case, should there be a distinction between literals such as >> ‘activemq’ or ‘queue’ and symbols such as ‘destinationName’ (e.g. >> ``)? >> >> David Jencks > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2
Re: Kamelets releases???
So I see version 0.5.0 in https://downloads.apache.org/camel/camel-kamelets/0.5.0/ but there’s no corresponding branch in GitHub, although there is a tag. Is this intentional? David Jencks > On Nov 19, 2021, at 3:03 PM, David Jencks wrote: > > I’m really glad to find out that camel-kamelets are voted on as part of the > camel-k release. > I’m happy for one vote to include any number of subprojects/artifacts. > > If something gets voted on, then I think the voted-on artifacts should be > listed on the downloads page in some form. I think this is a requirement of > Apache policy. Since AFAICT they aren’t there, there are no release branches, > and I didn’t think to look in the camel-k vote, I wondered if there were > actual voted-on releases. > > I also think that if there are released versions of a subproject that should > be reflected in the documentation. This can be dealt with using tags but it’s > much more flexible to use release branches, and that would bring the project > in line with every other camel subproject. > > If kamelets are effectively a part of camel-k, does it make sense to have a > separate documentation component for them? camel-k-runtime docs are included > under the camel-k docs without a separate component. We could easily do the > same for kamelets. If that doesn’t make sense, does it make sense to align > the versions? > > Thanks > > David Jencks > >> On Nov 19, 2021, at 2:14 PM, Andrea Cosentino wrote: >> >> By the way if you look at the vote for 1.7.0, but also for the old ones, >> camel-kamelets is listed under vote, like camel-k-runtime. >> >> Search for [VOTE] Release Apache Camel K 1.7.0 and related libraries >> >> If we have 3 artifacts for make camel-k release with a 3 days time for >> vote, this means at least 5 days for each artifact to release, so for >> releasing a camel-k version, we should have at least 15 days of vote + >> release + alignment. >> >> This has no meaning and the artifacts don't have any sense outside of >> camel-k, so having separated votes doesn't make sense. >> >> Also, in 2021, 15 days for releasing a single artifact is frankly a joke. >> >> Il giorno ven 19 nov 2021 alle ore 23:06 Andrea Cosentino >> ha scritto: >> >>> The answer is simple: kamelets are part of Camel-k like camel-k-runtime, >>> so the camel-kamelets is part of the camel-k release, in fact, >>> Camel-kamelets is part of the dependency needed to release camel-k. >>> >>> To me it doesn't makes sense to vote for this , because the kamelets could >>> only be used in camel-k or on camel-kamelets main side. We are talking >>> about a catalog more or less. It's not something consumable outside of a >>> camel runtime. >>> >>> This has been discussed by the way in the past. >>> >>> In terms of ASF policy, it is possible to release multiple artifacts, if >>> they are part of a release train. >>> >>> It looks like you're looking for finding problems where there aren't >>> problems :-) >>> >>> Il ven 19 nov 2021, 22:53 David Jencks ha >>> scritto: >>> >>>> I’m uneasy about the use of the camel-kamelets subproject. AFAICT there >>>> are no voted-on releases, and the website certainly only has a ‘next’ >>>> version. >>>> >>>> According to my understanding of Apache policy this means that no one >>>> other than camel developers should be using kamelets, and they certainly >>>> shouldn’t be used in production. >>>> >>>> Furthermore, there seems to be a usage of kamelets by camel-k, >>>> corresponding to a tag in camel-kamelets. I would expect a subproject >>>> release to only depend on voted on and released versions of other >>>> subprojects (as well, of course, released versions of other software). >>>> >>>> I would expect the cleanest solution would be to actually release >>>> camel-kamelets after votes. We’d then be able to have non-prerelease >>>> camel-kamelets documentation on the website and document the version links >>>> between at least camel-k and camel-kamelets >>>> >>>> Otherwise, I’d like an explanation of how the current state of affairs is >>>> consistent with Apache policy…. I’m no expert, but this situation seems >>>> highly unusual to me. >>>> >>>> Thanks, >>>> David Jencks >>> >>> >
[Website] activemq URI syntax inconsistency
On the Activemq component page, there are two descriptions of the URI syntax: hardcoded in the .adoc source, https://camel.apache.org/components/next/activemq-component.html#_uri_format: activemq:[queue:|topic:]destinationName[?options] from the .json component.syntax property, shown at https://camel.apache.org/components/next/activemq-component.html#_endpoint_options: activemq:destinationType:destinationName In the immediately following table, the default for destinationType is listed as “queue”, so evidently it can be left out. The first syntax results in activemq:destinationName whereas the second results in activemq::destinationName (one vs two colons). - Which is correct? - If the first, should the component.syntax property be activemq:[destinationType:]destinationName ? In any case, should there be a distinction between literals such as ‘activemq’ or ‘queue’ and symbols such as ‘destinationName’ (e.g. ``)? David Jencks
Re: Kamelets releases???
I’m really glad to find out that camel-kamelets are voted on as part of the camel-k release. I’m happy for one vote to include any number of subprojects/artifacts. If something gets voted on, then I think the voted-on artifacts should be listed on the downloads page in some form. I think this is a requirement of Apache policy. Since AFAICT they aren’t there, there are no release branches, and I didn’t think to look in the camel-k vote, I wondered if there were actual voted-on releases. I also think that if there are released versions of a subproject that should be reflected in the documentation. This can be dealt with using tags but it’s much more flexible to use release branches, and that would bring the project in line with every other camel subproject. If kamelets are effectively a part of camel-k, does it make sense to have a separate documentation component for them? camel-k-runtime docs are included under the camel-k docs without a separate component. We could easily do the same for kamelets. If that doesn’t make sense, does it make sense to align the versions? Thanks David Jencks > On Nov 19, 2021, at 2:14 PM, Andrea Cosentino wrote: > > By the way if you look at the vote for 1.7.0, but also for the old ones, > camel-kamelets is listed under vote, like camel-k-runtime. > > Search for [VOTE] Release Apache Camel K 1.7.0 and related libraries > > If we have 3 artifacts for make camel-k release with a 3 days time for > vote, this means at least 5 days for each artifact to release, so for > releasing a camel-k version, we should have at least 15 days of vote + > release + alignment. > > This has no meaning and the artifacts don't have any sense outside of > camel-k, so having separated votes doesn't make sense. > > Also, in 2021, 15 days for releasing a single artifact is frankly a joke. > > Il giorno ven 19 nov 2021 alle ore 23:06 Andrea Cosentino > ha scritto: > >> The answer is simple: kamelets are part of Camel-k like camel-k-runtime, >> so the camel-kamelets is part of the camel-k release, in fact, >> Camel-kamelets is part of the dependency needed to release camel-k. >> >> To me it doesn't makes sense to vote for this , because the kamelets could >> only be used in camel-k or on camel-kamelets main side. We are talking >> about a catalog more or less. It's not something consumable outside of a >> camel runtime. >> >> This has been discussed by the way in the past. >> >> In terms of ASF policy, it is possible to release multiple artifacts, if >> they are part of a release train. >> >> It looks like you're looking for finding problems where there aren't >> problems :-) >> >> Il ven 19 nov 2021, 22:53 David Jencks ha >> scritto: >> >>> I’m uneasy about the use of the camel-kamelets subproject. AFAICT there >>> are no voted-on releases, and the website certainly only has a ‘next’ >>> version. >>> >>> According to my understanding of Apache policy this means that no one >>> other than camel developers should be using kamelets, and they certainly >>> shouldn’t be used in production. >>> >>> Furthermore, there seems to be a usage of kamelets by camel-k, >>> corresponding to a tag in camel-kamelets. I would expect a subproject >>> release to only depend on voted on and released versions of other >>> subprojects (as well, of course, released versions of other software). >>> >>> I would expect the cleanest solution would be to actually release >>> camel-kamelets after votes. We’d then be able to have non-prerelease >>> camel-kamelets documentation on the website and document the version links >>> between at least camel-k and camel-kamelets >>> >>> Otherwise, I’d like an explanation of how the current state of affairs is >>> consistent with Apache policy…. I’m no expert, but this situation seems >>> highly unusual to me. >>> >>> Thanks, >>> David Jencks >> >>
Kamelets releases???
I’m uneasy about the use of the camel-kamelets subproject. AFAICT there are no voted-on releases, and the website certainly only has a ‘next’ version. According to my understanding of Apache policy this means that no one other than camel developers should be using kamelets, and they certainly shouldn’t be used in production. Furthermore, there seems to be a usage of kamelets by camel-k, corresponding to a tag in camel-kamelets. I would expect a subproject release to only depend on voted on and released versions of other subprojects (as well, of course, released versions of other software). I would expect the cleanest solution would be to actually release camel-kamelets after votes. We’d then be able to have non-prerelease camel-kamelets documentation on the website and document the version links between at least camel-k and camel-kamelets Otherwise, I’d like an explanation of how the current state of affairs is consistent with Apache policy…. I’m no expert, but this situation seems highly unusual to me. Thanks, David Jencks
Re: [Website] .htaccess Rewrite rules not working?
Just so this isn’t left dangling any longer, I came up with https://github.com/apache/camel-website/pull/673 which - uses Antora to generate the components/latest to components/3.13.x redirects, - Adds some modified redirectMatch rules to redirect components/ to components/3.13.x - merges this into the static/.htaccess, removing some old stuff from camel-quarkus and Zoran’s manually written rewrite rules. This also lets camel projects use the page-aliases feature. David Jencks > On Nov 2, 2021, at 2:23 AM, Zoran Regvart wrote: > > Hi David, > I'm a bit confused as to what are we trying to achieve here, redirect > /latest to /, e.g.: > > /camel/components/latest to /camel/components/3.12.x > > Am I on point thus far? Some questions below: > > On Sun, Oct 31, 2021 at 4:26 PM David Jencks <mailto:david.a.jen...@gmail.com>> wrote: >> >> I looked into this a little bit, using the Antora 3 `latest-version-segment` >> feature: >> >> urls: >> redirect_facility: httpd >> latest_version_segment_strategy: redirect:from >> latest_version_segment: latest >> >> This generates the following in the generated .htaccess: >> >> Redirect 302 /manual/latest /manual >> Redirect 302 /components/latest /components/3.12.x >> Redirect 302 /camel-k/latest /camel-k/1.6.x >> Redirect 302 /camel-kamelets/latest /camel-kamelets/next >> Redirect 302 /camel-quarkus/latest /camel-quarkus/2.4.x >> Redirect 302 /camel-kafka-connector/latest /camel-kafka-connector/0.11.0 >> Redirect 302 /camel-spring-boot/latest /camel-spring-boot/3.12.x >> Redirect 302 /camel-karaf/latest /camel-karaf/3.12.x >> - I think in any case we should use Redirect rather than Rewrite as in fact >> we want to generate redirects. > > Both generate a HTTP redirect, and by that I mean 3xx status and a > `Location` header, what am I missing here? Btw I would also use > `Redirect`, it seems easier to follow and maintain. > >> - I think we should use 302 rather than 301 since these redirects will >> change as we release newer versions. > > +1 and I think we should do this for both `.../next` and `.../latest` URLs > >> - I think that a RedirectMatch rule that redirects /> version or `next`>/ to the same target is likely to work: such a rule can >> easily be generated from the above. > > Do we need to? Couldn't we use the above generated .htaccess file directly? > >> So I’m thinking: >> >> - extract the above from the generated .htaccess >> - transform to the >>> / form, >> and add to generated .htaccess >> - insert the generated .htaccess in an appropriate place in static .htaccess > > Would Antora generate the .htaccess file in the root of the website > (i.e. in `documentation` and then copied over to `public`) when built, > so it would overwrite the existing .htaccess file from static? If not, > i.e. it would generate a .htaccess file per Antora component, we could > just leave those there. > >> I don’t have any good ideas yet on how or where to do this. I imagine the >> gulp file would be a good place but I’m not an expert on having gulp >> transform individual files. > > So, if I'm understanding this right, the problem is that the two (or > more?) .htaccess files would be generated in the same location and end > up overwriting each other. In that case we would need some way of > merging them and that we can do using Gulp. > > Another idea could be to add a Hugo template with similar logic as we > do on the download page and generate the desired .htaccess file. The > downloads page is fed via the release note's front matter and _if_ we > make sure that the documentation is ready for a release, which hasn't > been the case for 100% of the releases thus far, that's an approach > that can be made to work. > > zoran > -- > Zoran Regvart
Re: What can we do to make the documentation and website build more resilient?
Inline... > On Nov 13, 2021, at 12:40 PM, Zoran Regvart wrote: > > Hi David, > lots of great stuff here, I'll try to keep my replies short though... > > On Sat, Nov 13, 2021 at 7:47 PM David Jencks <mailto:david.a.jen...@gmail.com>> wrote: >> >> The Antora build part of the website is getting better at detecting problems >> and failing the build, and the website build seems to me to be failing more >> often. Perhaps we can find ways to improve our process so there are fewer >> problematic commits and it’s easier to detect and fix problems earlier. > > My intent with the website was that we should fail to publish as often > as necessary to do our best in not publishing a broken website. I > think we can tolerate website not being up to date with the latest for > a day or two. Absolutely! But there have been lots of times recently when I’ve broken the website and had no idea for days. > >> There are a few problems caused by interactions between near-in-time commits >> and commits that bring in stuff that is obsolete due to recent website build >> changes. Let’s ignore those :-)… especially the second kind will iron >> themselves out over time. > > +1 > >> So, people keep merging PRs that change the documentation without checking >> that it doesn’t break the website build, either locally or as a CI check on >> the PR. > > I think the xref syntax makes it difficult for folk to wrap their > head, I think everyone should be familiar with the documentation we > have here: > > https://github.com/apache/camel-website/#links-between-pages-in-antora-content > > <https://github.com/apache/camel-website/#links-between-pages-in-antora-content> I think that’s a really inaccessible location for the information…. I’d like to move it to a page in the manual, near the release guidelines. Something like “how to contribute to the docs”. > >> They theoretically could do a local website build that incorporates their >> changes, but right now it’s way too hard and time consuming. (I’ll discuss >> the problems with the projects that attempt to do a partial local build >> later) >> So one good step would be to make local website builds to check doc changes >> easy and quick. I’ve made some progress on this. > > +1 I think so too, local build and then the CI build against the git > repository should be our first lines of defense. We should make those > take seconds not minutes and then mandatory. > >> Another step would be for CI to check the website build on each PR, either >> the whole site or a partial build. I think GH actions can trigger each >> other, but I’ve never set it up. Do we have enough GH action time to do a >> full website build on every PR to any camel subproject? Is it practical to >> trigger the website build only when something documentation-related changes? >> (this detection would need to be carefully set up in each subproject) If >> these are possible I think we should just do this. It’s probably possible >> to set up quicker partial builds, but it’s decidedly more complicated. > > Most issues I've seen have been with xref linking, I think we should > focus on that first. Other checks I think fail less often. Perhaps not > building at all could be a good solution. For example in the Camel > main, Guillaume built a Maven plugin that checks for broken xrefs in > seconds. But that (currently) works only within the main Camel > repository. > > What if we could expand that or build comparable tooling that checks > xrefs in the adocs of the git repository the developer is working on > but also takes into account xrefs to other subprojects? Two ideas I > have here would be to clone other subprojects and build an index of > xrefs against them as well; or to use the sitemap XMLs (could be > fairly quick!) from the live website and reverse them back to xrefs > for checking. Well, the site manifest is like the sitemap with antora-compatible information. I think what I’m proposing with the sitemap and partial builds will be quick enough without another tool we struggle to keep up to date. > >> Another step would be to make it extremely visible when the jenkins website >> build fails. I try to follow the dev list pretty closely, and see a lot of >> GH PR CI build failures reported, but apparently the jenkins build has been >> failing for several days and I had no idea. > > +1 I think this is an area I can focus on next. I think we're in > agreement that we don't want to send emails to the dev@ mailing list, > one idea is to create GitHub issues; I just thought of a "status" > channel on Zulip. Or pe
What can we do to make the documentation and website build more resilient?
, leaving you with a functional local site. - Possibly the full Jenkins build could also package the Antora site as a zip archive, and local builds could fetch and unpack it rather than doing a full local build. With the site manifest, there’s still the problem of modifying the playbook to only build a little bit. I’ve written another extension that you configure with the part you want to build, and it applies appropriate filters. You can configure it down to one page. It also watches for changes and rebuilds when it detects a change: I think I’ll need to make that configurable since it’s great to see your changes quickly but not what you want for a build step. I have not yet tried to make it easy to select which subproject you want to build: so far it requires knowing how to configure the extensions. I’ve started having some ideas on how this might be done. What I’m envisioning and hoping for is a pre-PR process that involves running, in a local camel-website clone, something like `yarn partial-build-camel-quarkus` that will in less than a minute detect any errors and produce a local site you can look at with the local changes. Thoughts? David Jencks
Re: Linking to removed versions of documentation
> On Nov 13, 2021, at 1:31 AM, Zoran Regvart wrote: > > On Fri, Nov 12, 2021 at 8:55 PM David Jencks <mailto:david.a.jen...@gmail.com>> wrote: >> >> >> >>> On Nov 12, 2021, at 11:20 AM, Zoran Regvart wrote: >>> >>> Hi Cameleers, >>> Our current practice is that over time we would remove older versions >>> of documentation, we do this usually after a release of the new >>> version. We keep the LTS versions but we remove the older visions. >>> Sometimes we do this vigilantly, sometimes it takes a while till we >>> notice that an older version is still being published. >>> >>> Overall we do this for one primary reason, the build performance is >>> directly tied to the number of Antora components/branches being built. >>> If the build took a constant amount of time, regardless of the number >>> of components/branches being built we would probably leave more older >>> versions around. There is also some maintenance to keeping older >>> versions in case where we make a change on the website that is not >>> backward compatible and it breaks the older version -- but I don't >>> think we have had this issue thus far. >>> >>> This has recently been discussed (on Zulip or a GitHub issue, I can't >>> seem to find), a Camel user was asking about older versions of the >>> documentation; and again in the last few days the build was failing as >>> we pointed to an older version of the documentation from one of the >>> blog posts. >>> >>> I think we should discuss how we handle this, and explain our policy >>> somewhere on the website. >>> >>> What do you think, should we keep doing as we have been doing and >>> remove older versions? Should we keep all/more versions around? Can >>> someone think of a way we could keep the older versions around and not >>> jeopardize the build performance? >> >> There might be a way to keep older versions around without building them. >> Dan Allen came up with what he calls a “site manifest” which is a json file >> with all the pages in a site listed with enough info that xrefs to them can >> be computed. His original motivation was a “sharded site” which I think >> means there’s a big main site and then a supplemental site, not referenced >> from the main site, but with links to the main site. The site manifest from >> the main site lets xrefs from the subsidiary site to the main site be >> resolved. >> >> I’ve turned this into an Antora extension (with some modifications) and have >> been using it to have quick partial builds of the camel site, something I >> intend to talk about soon. > > Oh, that sounds really interesting. > >> I think we can use it sort of in reverse to publish a site manifest for each >> obsolete component version and use it to add to the component explorer. It >> will take some thinking and a bit of work but should not be too hard. > > Hmm I need to wrap my head around this, sounds like we would build the > version against the site manifest and never build it again? How's that > different from building it, keeping the HTML files around and then > removing that version from the playbook? With this plan we’d keep the HTML files around: the difference is that Antora would know about the old pages, so links to them would work and the versions would show up in the component explorer. > > We use hash of the CSS files for cache busting, so we'd need to keep > the older version of CSS files as well. Not really an issue, just > something we need to accomodate for. I don’t understand how this works. I would expect that everything Antora related would use the same UI and if we changed the html generation enough (maybe Antora 4 or if we added a custom template) we’d do a one-time regeneration of all the old docs. > > What would happen when an older version of the docs points to a page > in the user manual that we removed/renamed? Well, we shouldn’t be doing that now without leaving behind a redirect, hopefully soon via a page-aliases attribute :-) > >> I’d suggest that we do something to make it plain on each such page that the >> page is obsolete, and in the component explorer that the version is >> obsolete. This might involve a “final build” of the component version after >> a “retirement update”. Maybe we could have an “obsolete” watermark in the >> layout and set the “obsolete” layout in the antora.yml component descriptor. > > Yeah, I've seen banners displayed on older documentation pages > prompting users to use the latest documentation/version, that would be > helpful. > > I was also thinking about having a documentation ZIP distributed with > the release downloads. Probably using a different Antora UI, perhaps a > bit lighter and without the same menus as the published website. That could be interesting too. The site-manifest idea could be used to create working links to the actual user manual. How would you deal with links to the relevant obsolete versions of other subprojects? David Jencks
Re: Website updates after release
Master issue for the new version: https://github.com/apache/camel-website/issues/670 Release guide update PR: https://github.com/apache/camel/pull/6436 There were more parts than I thought! My PRs are on top of the PRs for https://github.com/apache/camel-website/pull/669: some of the PRs for #666 didn’t get reviewed yet, and one is used here. If that’s a problem I can redo the spring-boot PR to break it into two, one for this and one for #666. David Jencks > On Nov 12, 2021, at 8:58 AM, Gregor Zurowski wrote: > > Ok, thanks for taking care of this. I will review the documentation > once you have a PR for it. > > Thanks, > Gregor > > On Fri, Nov 12, 2021 at 3:38 PM David Jencks wrote: >> >> I’m happy to take care of this and update the instructions to the best of my >> ability :-) >> >>> On Nov 12, 2021, at 3:03 AM, Gregor Zurowski >>> wrote: >>> >>> Hi Everyone: >>> >>> I wanted to apply the changes described in >>> https://github.com/apache/camel/blob/main/docs/user-manual/modules/ROOT/pages/release-guide-website.adoc >>> after releasing Camel 3.13.0. It seems like a few things have changed >>> in the meantime, or maybe these are just issues in the current >>> document: >>> >>> 1) Based on the description in "Changes to main branch", we are >>> supposed to change `display-version` to the next version - in the >>> current case `camel-3.14.x`, but I currently see the `Next` value in >>> this tag instead of the previous version. Should I just leave `Next` >>> as-is? >>> >> >> It should stay Next >> >>> 2) In "Changes to the new branch" there is a reference to >>> `camel-spring-boot:docs/antora.yml` which does not exist. I assume >>> `camel-spring-boot:docs/components/antora.yml` should be updated >>> instead? >> >> I think I have a PR for the changes here already in the set related to >> https://github.com/apache/camel-website/pull/667 >> >>> >>> 3) The file >>> `camel-spring-boot:tooling/camel-spring-boot-docs-maven-plugin/src/main/java/org/apache/camel/springboot/maven/ExtMvelHelper.java` >>> does not exist in the Camel Spring Boot repository. >> >> Right, how the spring boots bits are generated has changed. >> >>> >>> Based on the findings above, I am reluctant to apply any changes, also >>> because I don't know the website build too well. Can someone please >>> take a look? >>> >>> Thanks in advance >>> Gregor >> >> David Jencks >>
Re: Linking to removed versions of documentation
> On Nov 12, 2021, at 11:20 AM, Zoran Regvart wrote: > > Hi Cameleers, > Our current practice is that over time we would remove older versions > of documentation, we do this usually after a release of the new > version. We keep the LTS versions but we remove the older visions. > Sometimes we do this vigilantly, sometimes it takes a while till we > notice that an older version is still being published. > > Overall we do this for one primary reason, the build performance is > directly tied to the number of Antora components/branches being built. > If the build took a constant amount of time, regardless of the number > of components/branches being built we would probably leave more older > versions around. There is also some maintenance to keeping older > versions in case where we make a change on the website that is not > backward compatible and it breaks the older version -- but I don't > think we have had this issue thus far. > > This has recently been discussed (on Zulip or a GitHub issue, I can't > seem to find), a Camel user was asking about older versions of the > documentation; and again in the last few days the build was failing as > we pointed to an older version of the documentation from one of the > blog posts. > > I think we should discuss how we handle this, and explain our policy > somewhere on the website. > > What do you think, should we keep doing as we have been doing and > remove older versions? Should we keep all/more versions around? Can > someone think of a way we could keep the older versions around and not > jeopardize the build performance? There might be a way to keep older versions around without building them. Dan Allen came up with what he calls a “site manifest” which is a json file with all the pages in a site listed with enough info that xrefs to them can be computed. His original motivation was a “sharded site” which I think means there’s a big main site and then a supplemental site, not referenced from the main site, but with links to the main site. The site manifest from the main site lets xrefs from the subsidiary site to the main site be resolved. I’ve turned this into an Antora extension (with some modifications) and have been using it to have quick partial builds of the camel site, something I intend to talk about soon. I think we can use it sort of in reverse to publish a site manifest for each obsolete component version and use it to add to the component explorer. It will take some thinking and a bit of work but should not be too hard. I’d suggest that we do something to make it plain on each such page that the page is obsolete, and in the component explorer that the version is obsolete. This might involve a “final build” of the component version after a “retirement update”. Maybe we could have an “obsolete” watermark in the layout and set the “obsolete” layout in the antora.yml component descriptor. David Jencks
Re: Website updates after release
I’m happy to take care of this and update the instructions to the best of my ability :-) > On Nov 12, 2021, at 3:03 AM, Gregor Zurowski wrote: > > Hi Everyone: > > I wanted to apply the changes described in > https://github.com/apache/camel/blob/main/docs/user-manual/modules/ROOT/pages/release-guide-website.adoc > after releasing Camel 3.13.0. It seems like a few things have changed > in the meantime, or maybe these are just issues in the current > document: > > 1) Based on the description in "Changes to main branch", we are > supposed to change `display-version` to the next version - in the > current case `camel-3.14.x`, but I currently see the `Next` value in > this tag instead of the previous version. Should I just leave `Next` > as-is? > It should stay Next > 2) In "Changes to the new branch" there is a reference to > `camel-spring-boot:docs/antora.yml` which does not exist. I assume > `camel-spring-boot:docs/components/antora.yml` should be updated > instead? I think I have a PR for the changes here already in the set related to https://github.com/apache/camel-website/pull/667 > > 3) The file > `camel-spring-boot:tooling/camel-spring-boot-docs-maven-plugin/src/main/java/org/apache/camel/springboot/maven/ExtMvelHelper.java` > does not exist in the Camel Spring Boot repository. Right, how the spring boots bits are generated has changed. > > Based on the findings above, I am reluctant to apply any changes, also > because I don't know the website build too well. Can someone please > take a look? > > Thanks in advance > Gregor David Jencks
Re: Broken xrefs in Camel Kafka Connector
I believe Andrea fixed this yesterday with
9c6826e7aad0119fb4c01925cf4bfc72e39114d7 (ckc). My PRs also fix it.
There’s another problem that will show up in the blog, which can be fixed with
this patch from https://github.com/apache/camel-website/pull/667:
diff --git a/content/blog/2020/12/CKC-idempotency-070/index.md
b/content/blog/2020/12/CKC-idempotency-070/index.md
index 4d25f25caa..da4cb5c81e 100644
--- a/content/blog/2020/12/CKC-idempotency-070/index.md
+++ b/content/blog/2020/12/CKC-idempotency-070/index.md
@@ -66,7 +66,7 @@ Some of the options can be used with their default value, in
this example we're
### A real example
-The best way to show how the idempotency feature works, in
camel-kafka-connector, it's through an example. We'll use the [AWS2-S3 Source
connector](/camel-kafka-connector/next/reference/connectors/camel-aws2-s3-kafka-source-connector.html)
+The best way to show how the idempotency feature works, in
camel-kafka-connector, it's through an example. We'll use the [AWS2-S3 Source
connector](/camel-kafka-connector/0.11.0/reference/connectors/camel-aws2-s3-kafka-source-connector.html)
As first step you'll need to fully build the [Camel-Kafka-connector
project](https://github.com/apache/camel-kafka-connector) and install the
connectors/camel-aws2-s3-kafka-connector zip package in your Kafka Broker
plugin.path. Once the connector is in the plugin.path location, just unzip it.
We describe how to build and unpack in the next steps:
This isn’t going to work for long, since the page referenced is no longer in
the next documentation, so will eventually disappear from the released branch,
even if we had a way to keep the specific release version correct.
I don’t have a solution in mind for this problem.
On a more general note, can we make jenkins website build failures email the
dev list? I have too hard a time detecting these problems. It seems that I
get plenty of emails from GH actions about other build failures, surely the
website is as important.
David Jencks
> On Nov 11, 2021, at 1:34 AM, Zoran Regvart wrote:
>
> Hi Cameleers,
> can someone (Andrea T.?) have a look at the broken xrefs in Camel
> Kafka Connector?
>
> This is reported on the website build[1]:
>
> ➤ YN:
> {"level":"error","time":1636576185286,"name":"asciidoctor","file":{"path":"docs/modules/ROOT/pages/reference/index.adoc"},"source":{"url":"https://github.com/apache/camel-kafka-connector.git","refname":"main","startPath":"docs"},"msg":"target
> of xref not found:
> reference/connectors/camel-syslog-kafka-sink-connector.adoc"}
> ➤ YN:
> {"level":"error","time":1636576185288,"name":"asciidoctor","file":{"path":"docs/modules/ROOT/pages/reference/index.adoc"},"source":{"url":"https://github.com/apache/camel-kafka-connector.git","refname":"main","startPath":"docs"},"msg":"target
> of xref not found:
> reference/connectors/camel-syslog-kafka-source-connector.adoc"}
> ➤ YN:
> {"level":"error","time":1636576213845,"name":"asciidoctor","file":{"path":"docs/modules/ROOT/nav.adoc"},"source":{"url":"https://github.com/apache/camel-kafka-connector.git","refname":"main","startPath":"docs"},"msg":"target
> of xref not found:
> reference/connectors/camel-syslog-kafka-source-connector.adoc"}
> ➤ YN:
> {"level":"error","time":1636576213846,"name":"asciidoctor","file":{"path":"docs/modules/ROOT/nav.adoc"},"source":{"url":"https://github.com/apache/camel-kafka-connector.git","refname":"main","startPath":"docs"},"msg":"target
> of xref not found:
> reference/connectors/camel-syslog-kafka-sink-connector.adoc"}
>
> So the index.adoc has links to several connectors that don't seem to resolve.
>
> zoran
>
> [1]
> https://ci-builds.apache.org/job/Camel/job/Camel.website/job/main/523/console
> --
> Zoran Regvart
[Website] Make Antora do more: replace symlinks and copying with Antora aggregate collector and generate docs directly from json files
(https://github.com/apache/camel-website/issues/666) Preview at https://pr-667--camel.netlify.app <https://pr-667--camel.netlify.app/> Using an observation and some initial code from Rob Winch I’ve developed an Antora extension that can scan for files not in the expected Antora structure and map them to Antora-appropriate locations. This can replace copying and symlinking files. Right now it can’t replace all the symlinking in main camel since that has a step of scanning .adocs for included example java code and then symlinking the used files. I think it’s practical to have the Antora extension do that too, but not yet. The PR for this issue applies this to camel-spring-boot and camel-kafka-connector. I’ve also enhanced the indexer used extensively for generating tables of components to act as an Antora extension and add a template instance page for each query result found. This lets us directly generate .adoc pages from .json files without needing a pre-existing .adoc page for each .json file. I think there are quite a few subprojects that have per-component documentation that is completely generated: to start with I’ve used camel-kafka-connector as an example. Further work in this PR set: Whatever is generating the json files for camel-kafka-connector is duplicating information. Properties with enumerated values have the values both as json elements and text in the description. For consistency across the site, I’m displaying the descriptions using uniform code that shows the json element values as a list, resulting in duplication, for instance https://pr-667--camel.netlify.app/camel-kafka-connector/next/reference/connectors/camel-activemq-kafka-sink-connector.html#_connector_option_camel_sink_path_destinationType. I’d like to remove the hardcoded text rendering from the description. I discovered a bug in how one of the asciidoctor extensions is adding header attributes and put in a quick workaround: this can be fixed properly with a new release of the extensions: the html is the same either way. If this approach seems like a good idea, after merging these I can work on other subprojects. There are some such as kamelets where this approach can be used simply and directly, and some, such as camel-quarkus, where I think it will work but may be more complicated. Take a look! David Jencks
Re: [website] Code highlighting
> On Nov 9, 2021, at 12:02 AM, Zoran Regvart wrote: > > Hi David, > > On Tue, Nov 9, 2021 at 2:14 AM David Jencks wrote: >> >> (prompted by seeing a recent highlight.js upgrade in the website) >> >> Someone pointed out shiki (https://shiki.matsu.io) on the Antora zulip >> recently and I wrote an Antora extension to set it up and use it as the >> (build time) syntax highlighter in Asciidoctor. >> >> The main point is that the syntax highlighting is grammar based, using the >> same parsers as VSCode. >> >> There isn’t all that much code to be highlighted in the docs, but I’ll make >> a demo to compare. Perhaps using it would be a good idea. >> >> My project is antora-shiki <https://gitlab.com/djencks/antora-shiki>. > > That looks interesting, I would like to trimm down on JavaScript usage > where appropriate. Though the Hugo bits also use highlight.js, so some > blog posts take advantage of this. I'd like to see a syntax > highlighter that doesn't use inline CSS also, does Shiki have this > option? It looks like themes.md <https://github.com/shikijs/shiki/blob/main/docs/themes.md#theming-with-css-variables> css variables for color names is as far as it goes right now, but the docs seem to indicate that it’s pretty easy to write a new theme. I’m not sure we’d want to get into that, but perhaps contributing it to shiki would be an option. David Jencks
[website] Code highlighting
(prompted by seeing a recent highlight.js upgrade in the website) Someone pointed out shiki (https://shiki.matsu.io) on the Antora zulip recently and I wrote an Antora extension to set it up and use it as the (build time) syntax highlighter in Asciidoctor. The main point is that the syntax highlighting is grammar based, using the same parsers as VSCode. There isn’t all that much code to be highlighted in the docs, but I’ll make a demo to compare. Perhaps using it would be a good idea. My project is antora-shiki <https://gitlab.com/djencks/antora-shiki>. David Jencks
Re: [camel-kafka-connector] Are there obsolete doc files?
Hi Andrea, Thanks for fixing this! > On Nov 7, 2021, at 1:29 AM, Andrea wrote: > > Hi David, > > sorry for the late response, we catch up in the chat and would like to > summarize the status of the effort here as well: > - there was problems in cleaning up documentation of some removed modules I > am working on it. > - I discovered a little problem in naming some file that I am addressing. > > As per generating the connector documentation in Antora I think it is a > welcome change! I think would be useful to have the possibility to also run > that generation local to the sub project thought, would that be possible? > Do you mean you’d like to have a quick local Antora build of just the camel-kafka-connector part of the site? I expect to have a separate email about that general issue soon. David Jencks > On Tue, Nov 2, 2021, at 14:54, David Jencks wrote: >> FWIW, my experiment seems to be working very well, generating the c-k-c >> connector doc pages directly from the .json files under connectors without >> any symlinks or copying. It relies on a lot of unpublished Antora >> extensions so I’m working on getting them ready and publishing them. It >> would be great to get a definitive answer about what should be present. >> There’s still the question of whether the catalog json files that aren’t >> copied from under connectors should be there. >> >> BTW it turns out there is one connector with non-generated documentation, >> camel-syslog-kafka-connector. >> >> >> David Jencks >> >>> On Nov 2, 2021, at 2:28 AM, Zoran Regvart wrote: >>> >>> Hi David, >>> >>> On Sun, Oct 31, 2021 at 4:02 PM David Jencks >>> wrote: >>>> TL/DR >>>> Can I assume that only projects currently under connectors should have >>>> pages in the website? (These are the pages indexed in reference/index.html >>>> and the nav) >>> >>> I do think so, perhaps Andrea T. can chime in here to help. I think it >>> would be a worthwhile effort to get those pages generated from the >>> JSON metadata, and thank you for taking this on. >>> >>> [snip] >>> >>>> I’m wondering if the processes that copy the adoc files under docs and the >>>> json files into the catalog should have a cleanup step to remove >>>> no-longer-current files. As noted above, I haven’t found how this copying >>>> is done. If my experiment succeeds, the adoc files won’t exist any more, >>>> being generated by Antora from the json files under connectors, so only >>>> the catalog would potentially need this cleaning step. >>> >>> Yeah, one of things we're not doing that well is cleaning up some of >>> the old stuff, but the catalog JSON files should be the most relevant >>> and up to date, so that would scratch that problem... >>> >>> zoran >>> -- >>> Zoran Regvart >> >>
Re: docs - how to add special docs for camel-spring-boot
> On Nov 6, 2021, at 11:44 PM, Claus Ibsen wrote: > > Hi > > The -starter JARs in camel-spring-boot no longer have docs, only a json file. > > There are a few starter JARs where we need to add some special > documentation. How should we do this now? > > In camel-quarkus they seems to be able to drop in usage.adoc / > limitations.adoc file > https://github.com/apache/camel-quarkus/tree/main/extensions/activemq/runtime/src/main/doc > > Maybe we need something similar? > > If so it would also make the docs similar between these two sub projects. I have a plan to generate the camel-quarkus docs differently, during the Antora build. I’m trying out the basis of the method on camel-kafka-connector and hope to have something to show in a day or two. Let’s try to pin down the requirements for camel-spring-boot a little more… 1. The camel-spring-boot docs never appear standalone, but always included at the end of main camel doc pages. So, if the extra content should appear before or after the existing content, one easy solution that doesn’t require changes in generation is to put the additional content in partials and manually add includes of it to the main camel doc page. If there will ever only be a few such camel-spring-boot projects needing extra documentation I’d recommend seriously considering this solution: it has the advantage of being very low-tech and visible. 2. If the additional doc bits need to be in the middle of the camel-spring-boot section, or if there are a lot of components needing additional docs, then a more automated solution would be appropriate. Here there are several choices as well. 2.a. If the additional content can just be included without any added section headers or other wrapping, then putting it in appropriately located and named partials and including it with an optional include in the template will work. 2.b. If the additional content needs to be wrapped with e.g. a section header, as camel-quarkus does, then something more complicated is needed. I think I know how to get Antora to assemble stuff as camel-quarkus does, but I won’t get to trying it for a few days. David Jencks > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2
Re: Website build broken
I’ve proposed a couple of PRs that do fix the website build and expose some info on this new springdoc thing, but I have no way to know how appropriate my new page is. https://github.com/apache/camel/pull/6375 https://github.com/apache/camel-spring-boot/pull/403 I’d suggest we apply the PRs and then improve the documentation. David Jencks > On Nov 3, 2021, at 10:15 AM, David Jencks wrote: > > Why does eaf68b5767c827d3b463b061a6e8aebc062cce46 add this file? > > docs/components/modules/spring-boot/partials/openapi-java-starter.adoc > > Obviously I need to document better how the spring-boot docs work now. Where? > > The source adoc file shouldn’t exist either: where did it come from? > > Running the build locally I see on > documentation/camel-spring-boot/next/list.html#_unused_spring_boot_starter_names: > > UNUSED SPRING-BOOT-STARTER NAMES > springdoc > > That means there is no main camel component adoc page referencing this > starter, so it won’t show up anywhere in the documentation. > > Indeed, there’s no springdoc component project, and the c-s-b pom hints it’s > an offshoot of the openapi-java component. > > I suspect the proper solution is to add a > components/camel-openapi-java/src/main/docs/camel-springdoc.adoc file with > > //Manually maintained attributes > :camel-spring-boot-name: springdoc > > in the header attributes and > > include::spring-boot:partial$starter.adoc[] > at the end. > > This whole experience indicates that the system I put in place works great > and that it isn’t documented enough. > > I guess adding instructions that appear when there’s a problem would be a > good place to start, but I don’t know how to get someone to look at that page. > > Running the site build locally I also see a ton of problems with camel-k and > kamelets that I haven’t looked at yet. > > For me the biggest problem is that I don’t see any notifications that the > website build is broken, and even if I do extracting the problem from the > build info is quite difficult. > > David Jencks > >> On Nov 3, 2021, at 6:48 AM, David Jencks > <mailto:david.a.jen...@gmail.com>> wrote: >> >> Maybe the copying isn’t configured quite right or the results of running it >> weren’t committed? IIRC it’s in the docs pom. I can investigate in a few >> hours. >> >> David Jencks >> >>> On Nov 3, 2021, at 6:43 AM, Claus Ibsen >> <mailto:claus.ib...@gmail.com>> wrote: >>> >>> Hi >>> >>> There is a new spring-boot module called camel-springdoc-starter which >>> could be the cause of this. >>> >>> I checked which -starter has .adoc files and its only >>> >>> ~/workspace/camel-spring-boot/components-starter main ❯ find . -name >>> '*.adoc' >>> ./camel-springdoc-starter/src/main/docs/springdoc-starter.adoc >>> ./camel-openapi-java-starter/src/main/docs/openapi-java-starter.adoc >>> >>> And then in the docs folder, then it is only one of them there is >>> located there. Maybe the file (springdoc-starter.adoc) must be >>> manually copied over there or something? >>> >>> ~/workspace/camel-spring-boot/docs main ❯ find . -name '*.adoc' >>> ./components/modules/spring-boot/partials/openapi-java-starter.adoc >>> ./components/modules/spring-boot/partials/starter.adoc >>> ./spring-boot/modules/ROOT/nav.adoc >>> ./spring-boot/modules/ROOT/pages/spring-boot-xml.adoc >>> ./spring-boot/modules/ROOT/pages/list.adoc >>> ./spring-boot/modules/ROOT/pages/spring-boot.adoc >>> ./spring-boot/modules/ROOT/pages/index.adoc >>> >>> On Wed, Nov 3, 2021 at 2:31 PM David Jencks >> <mailto:david.a.jen...@gmail.com>> wrote: >>>> >>>> Please don’t remove that include, it’s there to tell you that someone made >>>> the camel spring boot system inconsistent. The inconsistency is that >>>> there’s a .json file for a starter with no corresponding documentation >>>> page. If you build the site locally and look at the spring boot index >>>> page you’ll see what it is. If you can access the build results that >>>> failed you can see it there, but I don’t know how to do that. >>>> >>>> Where would be a good place to document how this works? >>>> >>>> David Jencks >>>> >>>> >>>> >>>>> On Nov 3, 2021, at 4:23 AM, Zoran Regvart >>>> <mailto:zo...@regvart.com>> wrote: >>>>> >>>>> Hi Cameleers, >&
Re: Website build broken
Why does eaf68b5767c827d3b463b061a6e8aebc062cce46 add this file? docs/components/modules/spring-boot/partials/openapi-java-starter.adoc Obviously I need to document better how the spring-boot docs work now. Where? The source adoc file shouldn’t exist either: where did it come from? Running the build locally I see on documentation/camel-spring-boot/next/list.html#_unused_spring_boot_starter_names: UNUSED SPRING-BOOT-STARTER NAMES springdoc That means there is no main camel component adoc page referencing this starter, so it won’t show up anywhere in the documentation. Indeed, there’s no springdoc component project, and the c-s-b pom hints it’s an offshoot of the openapi-java component. I suspect the proper solution is to add a components/camel-openapi-java/src/main/docs/camel-springdoc.adoc file with //Manually maintained attributes :camel-spring-boot-name: springdoc in the header attributes and include::spring-boot:partial$starter.adoc[] at the end. This whole experience indicates that the system I put in place works great and that it isn’t documented enough. I guess adding instructions that appear when there’s a problem would be a good place to start, but I don’t know how to get someone to look at that page. Running the site build locally I also see a ton of problems with camel-k and kamelets that I haven’t looked at yet. For me the biggest problem is that I don’t see any notifications that the website build is broken, and even if I do extracting the problem from the build info is quite difficult. David Jencks > On Nov 3, 2021, at 6:48 AM, David Jencks wrote: > > Maybe the copying isn’t configured quite right or the results of running it > weren’t committed? IIRC it’s in the docs pom. I can investigate in a few > hours. > > David Jencks > >> On Nov 3, 2021, at 6:43 AM, Claus Ibsen wrote: >> >> Hi >> >> There is a new spring-boot module called camel-springdoc-starter which >> could be the cause of this. >> >> I checked which -starter has .adoc files and its only >> >> ~/workspace/camel-spring-boot/components-starter main ❯ find . -name '*.adoc' >> ./camel-springdoc-starter/src/main/docs/springdoc-starter.adoc >> ./camel-openapi-java-starter/src/main/docs/openapi-java-starter.adoc >> >> And then in the docs folder, then it is only one of them there is >> located there. Maybe the file (springdoc-starter.adoc) must be >> manually copied over there or something? >> >> ~/workspace/camel-spring-boot/docs main ❯ find . -name '*.adoc' >> ./components/modules/spring-boot/partials/openapi-java-starter.adoc >> ./components/modules/spring-boot/partials/starter.adoc >> ./spring-boot/modules/ROOT/nav.adoc >> ./spring-boot/modules/ROOT/pages/spring-boot-xml.adoc >> ./spring-boot/modules/ROOT/pages/list.adoc >> ./spring-boot/modules/ROOT/pages/spring-boot.adoc >> ./spring-boot/modules/ROOT/pages/index.adoc >> >> On Wed, Nov 3, 2021 at 2:31 PM David Jencks wrote: >>> >>> Please don’t remove that include, it’s there to tell you that someone made >>> the camel spring boot system inconsistent. The inconsistency is that >>> there’s a .json file for a starter with no corresponding documentation >>> page. If you build the site locally and look at the spring boot index page >>> you’ll see what it is. If you can access the build results that failed you >>> can see it there, but I don’t know how to do that. >>> >>> Where would be a good place to document how this works? >>> >>> David Jencks >>> >>> >>> >>>> On Nov 3, 2021, at 4:23 AM, Zoran Regvart wrote: >>>> >>>> Hi Cameleers, >>>> The website has been broken for a few days now. One issue[1] reported >>>> by Claus was fixed by making sure that the Yarn cache is appropriate >>>> for the Docker/Linux build we perform on ci-builds.a.o. >>>> >>>> Now that issue seems to be resolved, but another issue occurred, and I >>>> can't really understand why, the `yarn workspaces foreach` seems to >>>> ignore the top-level workspace, and only the Antora UI is built. >>>> >>>> This issue seems to be resolved by upgrading Yarn & plugins, which I >>>> have prepared in a draft pull request[2], but that is now blocked by >>>> an error in the build traced back to a pull request[3] in >>>> camel-spring-boot. >>>> >>>> I'm not sure if I should outright remove that include, or if it serves >>>> a specific purpose that I don't understand yet. >>>> >>>> Can someone help me with this? David? >>>> >>>> zoran >>>> >>>> [1] https://github.com/apache/camel-website/issues/660 >>>> [2] https://github.com/apache/camel-website/pull/646 >>>> [3] https://github.com/apache/camel-spring-boot/pull/391 >>>> -- >>>> Zoran Regvart >>> >> >> >> -- >> Claus Ibsen >> - >> http://davsclaus.com @davsclaus >> Camel in Action 2: https://www.manning.com/ibsen2 >
Re: Website build broken
Maybe the copying isn’t configured quite right or the results of running it weren’t committed? IIRC it’s in the docs pom. I can investigate in a few hours. David Jencks > On Nov 3, 2021, at 6:43 AM, Claus Ibsen wrote: > > Hi > > There is a new spring-boot module called camel-springdoc-starter which > could be the cause of this. > > I checked which -starter has .adoc files and its only > > ~/workspace/camel-spring-boot/components-starter main ❯ find . -name '*.adoc' > ./camel-springdoc-starter/src/main/docs/springdoc-starter.adoc > ./camel-openapi-java-starter/src/main/docs/openapi-java-starter.adoc > > And then in the docs folder, then it is only one of them there is > located there. Maybe the file (springdoc-starter.adoc) must be > manually copied over there or something? > > ~/workspace/camel-spring-boot/docs main ❯ find . -name '*.adoc' > ./components/modules/spring-boot/partials/openapi-java-starter.adoc > ./components/modules/spring-boot/partials/starter.adoc > ./spring-boot/modules/ROOT/nav.adoc > ./spring-boot/modules/ROOT/pages/spring-boot-xml.adoc > ./spring-boot/modules/ROOT/pages/list.adoc > ./spring-boot/modules/ROOT/pages/spring-boot.adoc > ./spring-boot/modules/ROOT/pages/index.adoc > > On Wed, Nov 3, 2021 at 2:31 PM David Jencks wrote: >> >> Please don’t remove that include, it’s there to tell you that someone made >> the camel spring boot system inconsistent. The inconsistency is that >> there’s a .json file for a starter with no corresponding documentation page. >> If you build the site locally and look at the spring boot index page you’ll >> see what it is. If you can access the build results that failed you can see >> it there, but I don’t know how to do that. >> >> Where would be a good place to document how this works? >> >> David Jencks >> >> >> >>> On Nov 3, 2021, at 4:23 AM, Zoran Regvart wrote: >>> >>> Hi Cameleers, >>> The website has been broken for a few days now. One issue[1] reported >>> by Claus was fixed by making sure that the Yarn cache is appropriate >>> for the Docker/Linux build we perform on ci-builds.a.o. >>> >>> Now that issue seems to be resolved, but another issue occurred, and I >>> can't really understand why, the `yarn workspaces foreach` seems to >>> ignore the top-level workspace, and only the Antora UI is built. >>> >>> This issue seems to be resolved by upgrading Yarn & plugins, which I >>> have prepared in a draft pull request[2], but that is now blocked by >>> an error in the build traced back to a pull request[3] in >>> camel-spring-boot. >>> >>> I'm not sure if I should outright remove that include, or if it serves >>> a specific purpose that I don't understand yet. >>> >>> Can someone help me with this? David? >>> >>> zoran >>> >>> [1] https://github.com/apache/camel-website/issues/660 >>> [2] https://github.com/apache/camel-website/pull/646 >>> [3] https://github.com/apache/camel-spring-boot/pull/391 >>> -- >>> Zoran Regvart >> > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2
Re: Website build broken
Please don’t remove that include, it’s there to tell you that someone made the camel spring boot system inconsistent. The inconsistency is that there’s a .json file for a starter with no corresponding documentation page. If you build the site locally and look at the spring boot index page you’ll see what it is. If you can access the build results that failed you can see it there, but I don’t know how to do that. Where would be a good place to document how this works? David Jencks > On Nov 3, 2021, at 4:23 AM, Zoran Regvart wrote: > > Hi Cameleers, > The website has been broken for a few days now. One issue[1] reported > by Claus was fixed by making sure that the Yarn cache is appropriate > for the Docker/Linux build we perform on ci-builds.a.o. > > Now that issue seems to be resolved, but another issue occurred, and I > can't really understand why, the `yarn workspaces foreach` seems to > ignore the top-level workspace, and only the Antora UI is built. > > This issue seems to be resolved by upgrading Yarn & plugins, which I > have prepared in a draft pull request[2], but that is now blocked by > an error in the build traced back to a pull request[3] in > camel-spring-boot. > > I'm not sure if I should outright remove that include, or if it serves > a specific purpose that I don't understand yet. > > Can someone help me with this? David? > > zoran > > [1] https://github.com/apache/camel-website/issues/660 > [2] https://github.com/apache/camel-website/pull/646 > [3] https://github.com/apache/camel-spring-boot/pull/391 > -- > Zoran Regvart
Re: [camel-kafka-connector] Are there obsolete doc files?
FWIW, my experiment seems to be working very well, generating the c-k-c connector doc pages directly from the .json files under connectors without any symlinks or copying. It relies on a lot of unpublished Antora extensions so I’m working on getting them ready and publishing them. It would be great to get a definitive answer about what should be present. There’s still the question of whether the catalog json files that aren’t copied from under connectors should be there. BTW it turns out there is one connector with non-generated documentation, camel-syslog-kafka-connector. David Jencks > On Nov 2, 2021, at 2:28 AM, Zoran Regvart wrote: > > Hi David, > > On Sun, Oct 31, 2021 at 4:02 PM David Jencks wrote: >> TL/DR >> Can I assume that only projects currently under connectors should have pages >> in the website? (These are the pages indexed in reference/index.html and the >> nav) > > I do think so, perhaps Andrea T. can chime in here to help. I think it > would be a worthwhile effort to get those pages generated from the > JSON metadata, and thank you for taking this on. > > [snip] > >> I’m wondering if the processes that copy the adoc files under docs and the >> json files into the catalog should have a cleanup step to remove >> no-longer-current files. As noted above, I haven’t found how this copying is >> done. If my experiment succeeds, the adoc files won’t exist any more, being >> generated by Antora from the json files under connectors, so only the >> catalog would potentially need this cleaning step. > > Yeah, one of things we're not doing that well is cleaning up some of > the old stuff, but the catalog JSON files should be the most relevant > and up to date, so that would scratch that problem... > > zoran > -- > Zoran Regvart
Re: [Website] .htaccess Rewrite rules not working?
I looked into this a little bit, using the Antora 3 `latest-version-segment`
feature:
urls:
redirect_facility: httpd
latest_version_segment_strategy: redirect:from
latest_version_segment: latest
This generates the following in the generated .htaccess:
Redirect 302 /manual/latest /manual
Redirect 302 /components/latest /components/3.12.x
Redirect 302 /camel-k/latest /camel-k/1.6.x
Redirect 302 /camel-kamelets/latest /camel-kamelets/next
Redirect 302 /camel-quarkus/latest /camel-quarkus/2.4.x
Redirect 302 /camel-kafka-connector/latest /camel-kafka-connector/0.11.0
Redirect 302 /camel-spring-boot/latest /camel-spring-boot/3.12.x
Redirect 302 /camel-karaf/latest /camel-karaf/3.12.x
- I think in any case we should use Redirect rather than Rewrite as in fact we
want to generate redirects.
- I think we should use 302 rather than 301 since these redirects will change
as we release newer versions.
- I think that a RedirectMatch rule that redirects // to the same target is likely to work: such a rule can
easily be generated from the above.
So I’m thinking:
- extract the above from the generated .htaccess
- transform to the >>> / form, and
add to generated .htaccess
- insert the generated .htaccess in an appropriate place in static .htaccess
I don’t have any good ideas yet on how or where to do this. I imagine the gulp
file would be a good place but I’m not an expert on having gulp transform
individual files.
David Jencks
> On Oct 29, 2021, at 7:40 AM, Zoran Regvart wrote:
>
> Hi David,
>
> On Fri, Oct 29, 2021 at 4:24 PM David Jencks wrote:
>>
>> Well, I agree and am considering how to make that happen. This redirects
>> the previous `latest` links to the same pages which are now at `next` so at
>> least we don’t have broken links (someone pointed one out on Zulip just now).
>>
>> Currently the latest released main camel version is 3.12.x. Taking activemq
>> as an example, there are 3 pages that I think should end up in the same
>> place:
>>
>> https://camel.apache.org/components/activemq-component.html
>> https://camel.apache.org/components/latest/activemq-component.html
>> https://camel.apache.org/components/3.12.x/activemq-component.html
>
> In that case the RewriteRule needs to be changed not to redirect the
> first URL to `/components/next/...`. How would we go about redirecting
> "" and "latest" to 3.12.x, we'd need to compute what the latest
> release is. Which we do in in Hugo bits for the download page
>
>> Antora can produce a rewrite rule (I think) to deal with one redirection.
>> Using this would mean we’d have to incorporate an Antora generated .htaccess
>> into a static one, which would be nice for other reasons, namely we can have
>> working page aliases.
>
> Can Antora help with the above ("" and "latest -> 3.12.x)?
>
>> I don’t think we want to have a manually maintained rewrite rule that
>> directs to the concrete version.
>
> Not at all, this would be error prone and we would forget that it
> needs to be done.
>
>> Which of these should be the URL the other two redirect to? I’d think the
>> one with the concrete version 3.12.x.
>
> 3.12.x makes sense for me, but only if we have a way to auto-maintain
> it, otherwise it's just more work to keep track of...
>
>> Do we actually want all 3 urls to work or can we get by with two?
>
> Not sure, having all three makes sense to me, the first one ("") bit
> less than the others, but if possible I'd still include it...
>
> (2c)
>
> zoran
> --
> Zoran Regvart
[camel-kafka-connector] Are there obsolete doc files?
35 - connectors/camel-aws2-s3-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/aws2s3/CamelAws2s3SourceConnectorConfig.java | 392 --- connectors/camel-aws2-s3-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/aws2s3/CamelAws2s3SourceTask.java | 39 -- connectors/camel-aws2-s3-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/aws2s3/aggregation/NewlineAggregationStrategy.java | 44 -- connectors/camel-aws2-s3-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/aws2s3/converters/S3ObjectConverter.java | 45 -- connectors/camel-aws2-s3-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/aws2s3/models/StorageHeader.java | 28 connectors/camel-aws2-s3-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/aws2s3/models/StorageRecord.java | 30 connectors/camel-aws2-s3-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/aws2s3/serializers/S3ObjectSerializer.java | 57 connectors/camel-aws2-s3-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/aws2s3/transformers/JSONToRecordTransforms.java | 76 -- connectors/camel-aws2-s3-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/aws2s3/transformers/RecordToJSONTransforms.java | 86 connectors/camel-aws2-s3-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/aws2s3/transformers/S3ObjectTransforms.java | 55 connectors/camel-aws2-s3-kafka-connector/src/main/resources/META-INF/LICENSE.txt | 203 --- connectors/camel-aws2-s3-kafka-connector/src/main/resources/META-INF/NOTICE.txt | 11 -- 26 files changed, 3227 deletions(-) commit 2267ac23a54a2585d3cd9680d0a60c2dc6983f1f Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Date: Fri Oct 8 03:44:59 2021 + [create-pull-request] automated change connectors/camel-aws2-s3-kafka-connector/src/generated/resources/camel-aws2-s3-sink.json | 4 ++-- connectors/camel-aws2-s3-kafka-connector/src/generated/resources/camel-aws2-s3-source.json | 4 ++-- connectors/camel-aws2-s3-kafka-connector/src/main/docs/camel-aws2-s3-kafka-sink-connector.adoc | 4 ++-- connectors/camel-aws2-s3-kafka-connector/src/main/docs/camel-aws2-s3-kafka-source-connector.adoc | 4 ++-- 4 files changed, 8 insertions(+), 8 deletions(-) ... I’m wondering if the processes that copy the adoc files under docs and the json files into the catalog should have a cleanup step to remove no-longer-current files. As noted above, I haven’t found how this copying is done. If my experiment succeeds, the adoc files won’t exist any more, being generated by Antora from the json files under connectors, so only the catalog would potentially need this cleaning step. FWIW I found these files looking for adoc and json files with converters, transforms, or aggregation strategies…. I think these aws2-s3 ones are the only ones using these features. David Jencks
Re: [Website] .htaccess Rewrite rules not working?
Well, I agree and am considering how to make that happen. This redirects the previous `latest` links to the same pages which are now at `next` so at least we don’t have broken links (someone pointed one out on Zulip just now). Currently the latest released main camel version is 3.12.x. Taking activemq as an example, there are 3 pages that I think should end up in the same place: https://camel.apache.org/components/activemq-component.html https://camel.apache.org/components/latest/activemq-component.html https://camel.apache.org/components/3.12.x/activemq-component.html Antora can produce a rewrite rule (I think) to deal with one redirection. Using this would mean we’d have to incorporate an Antora generated .htaccess into a static one, which would be nice for other reasons, namely we can have working page aliases. I don’t think we want to have a manually maintained rewrite rule that directs to the concrete version. Which of these should be the URL the other two redirect to? I’d think the one with the concrete version 3.12.x. Do we actually want all 3 urls to work or can we get by with two? David Jencks > On Oct 29, 2021, at 7:10 AM, Antonin Stefanutti > wrote: > > Naively / intuitively, I would assume 'latest' points to the latest released > version, while 'next' points to what is going to be the next version released. > > If I understand it correctly, 'latest' is redirected to 'next'. > > That being said, I appreciate it may be more complex to implement, if that > makes sense at all. > >> On 29 Oct 2021, at 15:39, Zoran Regvart wrote: >> >> Hi Cameleers, >> I've created a PR that I think will solve the issues outlined in this thread: >> >> https://github.com/apache/camel-website/pull/659 >> >> Please have a look, there is a test script there I'd be happy to hear >> if I missed a test case there, so I can add it and make sure the >> redirects work as expected. >> >> zoran >> -- >> Zoran Regvart >
Re: [Website] .htaccess Rewrite rules not working?
Well, that’s not a very useful url… I tried % curl -k https://camel.apache.org/components/activemq-component.html Page Not Found :: Apache Camel https://camel.apache.org/components/latest/activemq-component.html I also got the error page for % curl -k https://camel.apache.org/components/latest (curl on my mac doesn’t seem to know about any certificates, thus the -k) This matches the behavior in my browser when I try to use the redirects on an actual page. David Jencks > On Oct 28, 2021, at 12:53 PM, Zoran Regvart wrote: > > Hi David, > I've tried the one on 1288 line: > > $ curl https://camel.apache.org/components/ > > > 301 Moved Permanently > > Moved Permanently > The document has moved href="https://camel.apache.org/components/next/;>here. > > > and a couple below it, and it works on my end, a typo at your end? > > zoran > > On Wed, Oct 27, 2021 at 11:56 PM David Jencks > wrote: >> >> It appears the rewrite rules in camel-website static/.htaccess starting at >> line 1288 aren’t working. Does anyone know now to investigate what’s going >> on? >> Did they work before the recent latest >> next change? >> >> David Jencks > > > > -- > Zoran Regvart
[Website] .htaccess Rewrite rules not working?
It appears the rewrite rules in camel-website static/.htaccess starting at line 1288 aren’t working. Does anyone know now to investigate what’s going on? Did they work before the recent latest >> next change? David Jencks
Re: [Website and many projects] Please check the preview and review
Hi Zoran, > On Oct 26, 2021, at 12:58 AM, Zoran Regvart wrote: > > Hi David, > > On Mon, Oct 25, 2021 at 5:07 PM David Jencks wrote: >> The promise of pnp seems to be only partially fulfilled. If I run yarn >> build-all on a clean checkout, it fails because the stuff in >> antora-ui-camel/.yarn/unplugged is for linux, not macOS. I have to run yarn >> workspaces foreach rebuild in order to get a working build. I’m going to >> add this to the Jenckinsfile so if I check in depedency updates it won’t >> break the jenkins build. > > It does look to me like the cache was platform-dependent, I think it's > also Node/Gyp/C++/linker tooling dependent, I did run into that on > #646[1], not sure what the best option is. I'm a bit scared that > without proper cache check we'll end up getting into all sorts of > issues. > I think .yarn/cache is unmodified but .yarn/unplugged changes on these rebuilds. I think it’s more important that everyone can check in changes to the dependencies than these warnings disappear. If you can find a way to get the checked in .yarn/unplugged to match the Jenkins environment, great, as long as we keep the rebuild-all script for people on other platforms :-) If the Jenkins and GitHub environments are sufficiently similar, perhaps the regen bot or something like it could produce a PR with the rebuild changes. David Jencks
[Website] Preview of unversioned manual, and 'next' for prerelease versions
Please take a look at the preview for https://github.com/apache/camel-website/issues/642 at https://pr-654--camel.netlify.app <https://pr-654--camel.netlify.app/> The main features are: - The manual is unversioned, i.e. pages at /manual/ rather than /manual/latest - The component explorer doesn’t show a version for the manual: you click directly on ‘manual’. - The drop down for the list of component/versions has the caret in a different place when ‘manual’ is the current component. This is probably easy to fix if you know just a little more css than me. - The released camel-quarkus docs shown are 2.4.x and the released camel-k docs are 1.6.x - all the ‘latest’ versions are now called ‘next’ and the url version segment is ‘next’ - also, the kamelets catalog is working again. I’d like to merge this tomorrow if there are no big objections. David Jencks
Re: [RESULT] [VOTE] Release Apache Camel Quarkus 2.4.0
Am I correct in thinking that this means the website should point to the camel-quarkus 2.4.x docs and the earlier ones should be removed? I’m working on https://github.com/apache/camel-quarkus/issues/3191. David Jencks > On Oct 25, 2021, at 9:40 AM, Peter Palaga wrote: > > Hi all, > > the vote passed with the following result > > 7 +1 binding votes: > * James Netherton > * Zoran Regvart > * Claus Ibsen > * Andrea Cosentino > * Jean-Baptiste Onofré > * Luca Burgazzoli > * Alexandre Gallice > > 3 +1 non-binding: > * Zineb Bendhiba > * Otavio Piske > * Zheng Feng > > The artifacts are being published now. > > Thanks for the votes everyone! > > -- Peter >
Re: [Website and many projects] Please check the preview and review
Hi Zoran, Many thanks for fixing this. Your explanation is what I thought should be happening but…. The yarn docs indicate that pnp is supposed to be the default strategy for .yarnrc nodeLinker, however when I run yarn install it adds node_modules and removes .pnp.js. If I explicitly set nodeLinker: “pnp” then it removes node_modules and brings back .pnp.js. I’m going to add the explicit setting. (it turns out that something added a .yarnrc.yml file in a parent directory with nodeLinker: “node-modules”…. it wasn’t a manual action on my part) With this setting, running yarn install only results in changes in .yarn/build-state.yml and files under .yarn/unplugged. I can undo these changes and run yarn install —immutable successfully, but if I change package.json to use Antora 3.0.0-alpha.10 then it fails. I’ll try changing the pr.yaml like this: -run: yarn workspaces foreach install --check-cache +run: yarn workspaces foreach install --check-cache --immutable The promise of pnp seems to be only partially fulfilled. If I run yarn build-all on a clean checkout, it fails because the stuff in antora-ui-camel/.yarn/unplugged is for linux, not macOS. I have to run yarn workspaces foreach rebuild in order to get a working build. I’m going to add this to the Jenckinsfile so if I check in depedency updates it won’t break the jenkins build. I opened https://github.com/apache/camel-website/issues/652 to track this. David Jencks > On Oct 25, 2021, at 2:02 AM, Zoran Regvart wrote: > > Hi David, > I think the issue should be fixed now[1]. The goal of Yarn cache as > implemented using PnP is to maintain a hashed artifact of the > dependency in `.yarn/cache`, so any time a dependency is changed a > corresponding change needs to be made in `.yarn/cache`, for this > running `yarn install` is all that should be required. > > The build does not download any dependencies, and with the exception > of accessing git repositories and GitHub API it should be offline. > This serves mainly two purposes: build remains stable (i.e. it's not > affected by NPM/Yarn outages or limits) and build performance is > increased (i.e. no download latency). > > It seems that the check for cache being up to date doesn't fail the > build[2], that should be improved so we don't end up building with the > wrong dependencies. > > zoran > > [1] > https://github.com/apache/camel-website/commit/83b4533ce1d97132e1c109b396ddcfc709b63822 > > <https://github.com/apache/camel-website/commit/83b4533ce1d97132e1c109b396ddcfc709b63822> > [2] https://github.com/apache/camel-website/pull/648/checks#step:3:15 > <https://github.com/apache/camel-website/pull/648/checks#step:3:15> > > On Sun, Oct 24, 2021 at 4:17 PM David Jencks <mailto:david.a.jen...@gmail.com>> wrote: >> >> I’m afraid Zoran will have to fix this. I’m totally confused by what yarn >> is doing. >> >> - mvn approach: >> getting back to a git clean state, running mvn clean package appears to be >> using npm 10 and yarn 1.14 and quickly errors. >> >> - yarn approach: >> >> david@mbp2 camel-website % yarn --version >> 2.4.0 >> >> Running yarn install: >> >> david@mbp2 camel-website % git status >> On branch main >> Your branch is up to date with 'origin/main'. >> >> Changes not staged for commit: >> (use "git add/rm ..." to update what will be committed) >> (use "git restore ..." to discard changes in working directory) >>deleted:.pnp.js >>modified: .yarn/build-state.yml >>deleted: >> .yarn/cache/@djencks-asciidoctor-antora-indexer-npm-0.0.6-4db3f3a720-70a23e1885.zip >>deleted: >> .yarn/cache/@djencks-asciidoctor-jsonpath-npm-0.0.4-daa788a548-ba33c6567c.zip >>modified: yarn.lock >> >> Untracked files: >> (use "git add ..." to include in what will be committed) >> >> .yarn/cache/@djencks-asciidoctor-antora-indexer-npm-0.1.0-alpha.1-015913d711-cbc7c0da4d.zip >> >> .yarn/cache/@djencks-asciidoctor-jsonpath-npm-0.1.0-alpha.1-7faec6e075-36431697f1.zip >> >> .yarn/cache/@djencks-asciidoctor-report-support-npm-0.1.0-alpha.1-8796513cc6-12900388ef.zip >>node_modules/ >> >> Why is there now a node_modules removing it prevents yarn from running. >> Before trying to clean up this problem I had a lot of uncommitted local >> files but I could run yarn install without deleting .pnp.js or getting a >> node_modules. >> >> I also suspect that even if I got a plausible setup locally it wouldn’t work >> to commit it since I’m on a mac and the
Re: [Website and many projects] Please check the preview and review
I opened Yarn cached dependencies are not correct for Jenkins build, and there are no instructions on how to update them <https://github.com/apache/camel-website/issues/651> to more officially ask Zoran for help on this. I can come up with something that makes the jenkins build work, but it will at least break the zero-install goal. I also discovered the kamelets index page is broken, and have a fix. I expect to have a preview for Use next instead of pre-release display label <https://github.com/apache/camel-website/issues/642> tomorrow and the fix will be there. David Jencks > On Oct 24, 2021, at 7:17 AM, David Jencks wrote: > > I’m afraid Zoran will have to fix this. I’m totally confused by what yarn is > doing. > > - mvn approach: > getting back to a git clean state, running mvn clean package appears to be > using npm 10 and yarn 1.14 and quickly errors. > > - yarn approach: > > david@mbp2 camel-website % yarn --version > 2.4.0 > > Running yarn install: > > david@mbp2 camel-website % git status > On branch main > Your branch is up to date with 'origin/main'. > > Changes not staged for commit: > (use "git add/rm ..." to update what will be committed) > (use "git restore ..." to discard changes in working directory) > deleted:.pnp.js > modified: .yarn/build-state.yml > deleted: > .yarn/cache/@djencks-asciidoctor-antora-indexer-npm-0.0.6-4db3f3a720-70a23e1885.zip > > <mailto:yarn/cache/@djencks-asciidoctor-antora-indexer-npm-0.0.6-4db3f3a720-70a23e1885.zip> > deleted: > .yarn/cache/@djencks-asciidoctor-jsonpath-npm-0.0.4-daa788a548-ba33c6567c.zip > <mailto:yarn/cache/@djencks-asciidoctor-jsonpath-npm-0.0.4-daa788a548-ba33c6567c.zip> > modified: yarn.lock > > Untracked files: > (use "git add ..." to include in what will be committed) > > .yarn/cache/@djencks-asciidoctor-antora-indexer-npm-0.1.0-alpha.1-015913d711-cbc7c0da4d.zip > > <mailto:yarn/cache/@djencks-asciidoctor-antora-indexer-npm-0.1.0-alpha.1-015913d711-cbc7c0da4d.zip> > > .yarn/cache/@djencks-asciidoctor-jsonpath-npm-0.1.0-alpha.1-7faec6e075-36431697f1.zip > > <mailto:yarn/cache/@djencks-asciidoctor-jsonpath-npm-0.1.0-alpha.1-7faec6e075-36431697f1.zip> > > .yarn/cache/@djencks-asciidoctor-report-support-npm-0.1.0-alpha.1-8796513cc6-12900388ef.zip > > <mailto:yarn/cache/@djencks-asciidoctor-report-support-npm-0.1.0-alpha.1-8796513cc6-12900388ef.zip> > node_modules/ > > Why is there now a node_modules removing it prevents yarn from running. > Before trying to clean up this problem I had a lot of uncommitted local > files but I could run yarn install without deleting .pnp.js or getting a > node_modules. > > I also suspect that even if I got a plausible setup locally it wouldn’t work > to commit it since I’m on a mac and the CI/Jenkins runs aren’t: the README.md > line 126 says I should expect to run yarn rebuild, which doesn’t appear to be > done by the Jenkins script. > > David Jencks > >> On Oct 23, 2021, at 6:59 AM, David Jencks > <mailto:david.a.jen...@gmail.com>> wrote: >> >> Something is dreadfully wrong with the jenkins website build. It looks as >> if the jenkins build is not using the updated indexer and jenkins extensions. >> >> I don’t really understand how the yarn pnp stuff works, and I did not update >> any of the yarn caches…. I’m a bit scared off by stuff like this when I run >> yarn locally: >> >> ➤ YN: ┌ Link step >> ➤ YN0062: │ @netlify/traffic-mesh-agent-linux-x64@npm:0.27.0 The platform >> darwin is incompatible with this module, link skipped. >> ➤ YN0062: │ @netlify/traffic-mesh-agent-win32-x64@npm:0.27.0 The platform >> darwin is incompatible with this module, link skipped. >> >> >> Does yarn update the cache so it will work on any platform? It would be >> great if there was documentation in the README about what to do if you >> change the dependencies in package.json. >> >> The default values do appear in the preview. >> >> David Jencks >> >>> On Oct 23, 2021, at 2:09 AM, Claus Ibsen >> <mailto:claus.ib...@gmail.com>> wrote: >>> >>> Hi >>> >>> The default value is not yet appearing. >>> >>> On Sat, Oct 23, 2021 at 10:03 AM Zoran Regvart >> <mailto:zo...@regvart.com>> wrote: >>>> >>>> Hi David, >>>> everything looks good, thank you for this effort! >>>> >>>> zoran >>>> >>>>> On 22 Oct 202
Re: [Website and many projects] Please check the preview and review
I’m afraid Zoran will have to fix this. I’m totally confused by what yarn is doing. - mvn approach: getting back to a git clean state, running mvn clean package appears to be using npm 10 and yarn 1.14 and quickly errors. - yarn approach: david@mbp2 camel-website % yarn --version 2.4.0 Running yarn install: david@mbp2 camel-website % git status On branch main Your branch is up to date with 'origin/main'. Changes not staged for commit: (use "git add/rm ..." to update what will be committed) (use "git restore ..." to discard changes in working directory) deleted:.pnp.js modified: .yarn/build-state.yml deleted: .yarn/cache/@djencks-asciidoctor-antora-indexer-npm-0.0.6-4db3f3a720-70a23e1885.zip deleted: .yarn/cache/@djencks-asciidoctor-jsonpath-npm-0.0.4-daa788a548-ba33c6567c.zip modified: yarn.lock Untracked files: (use "git add ..." to include in what will be committed) .yarn/cache/@djencks-asciidoctor-antora-indexer-npm-0.1.0-alpha.1-015913d711-cbc7c0da4d.zip .yarn/cache/@djencks-asciidoctor-jsonpath-npm-0.1.0-alpha.1-7faec6e075-36431697f1.zip .yarn/cache/@djencks-asciidoctor-report-support-npm-0.1.0-alpha.1-8796513cc6-12900388ef.zip node_modules/ Why is there now a node_modules removing it prevents yarn from running. Before trying to clean up this problem I had a lot of uncommitted local files but I could run yarn install without deleting .pnp.js or getting a node_modules. I also suspect that even if I got a plausible setup locally it wouldn’t work to commit it since I’m on a mac and the CI/Jenkins runs aren’t: the README.md line 126 says I should expect to run yarn rebuild, which doesn’t appear to be done by the Jenkins script. David Jencks > On Oct 23, 2021, at 6:59 AM, David Jencks wrote: > > Something is dreadfully wrong with the jenkins website build. It looks as > if the jenkins build is not using the updated indexer and jenkins extensions. > > I don’t really understand how the yarn pnp stuff works, and I did not update > any of the yarn caches…. I’m a bit scared off by stuff like this when I run > yarn locally: > > ➤ YN: ┌ Link step > ➤ YN0062: │ @netlify/traffic-mesh-agent-linux-x64@npm:0.27.0 The platform > darwin is incompatible with this module, link skipped. > ➤ YN0062: │ @netlify/traffic-mesh-agent-win32-x64@npm:0.27.0 The platform > darwin is incompatible with this module, link skipped. > > > Does yarn update the cache so it will work on any platform? It would be > great if there was documentation in the README about what to do if you change > the dependencies in package.json. > > The default values do appear in the preview. > > David Jencks > >> On Oct 23, 2021, at 2:09 AM, Claus Ibsen > <mailto:claus.ib...@gmail.com>> wrote: >> >> Hi >> >> The default value is not yet appearing. >> >> On Sat, Oct 23, 2021 at 10:03 AM Zoran Regvart > <mailto:zo...@regvart.com>> wrote: >>> >>> Hi David, >>> everything looks good, thank you for this effort! >>> >>> zoran >>> >>>> On 22 Oct 2021, at 02:45, David Jencks >>> <mailto:david.a.jen...@gmail.com>> wrote: >>>> >>>> I’ve prepared a website update to the recent incompatible-syntax upgrade >>>> of my indexer and jsonpath Antora extensions that build most of the tables >>>> and a lot of content. The preview is at https://pr-648--camel.netlify.app >>>> <https://pr-648--camel.netlify.app/> <https://pr-648--camel.netlify.app/ >>>> <https://pr-648--camel.netlify.app/>>. I’d appreciate lots of eyes to see >>>> if I broke anything >>>> >>>> I think this is ready to merge… there are 17 PRs to merge, some of which >>>> need to be adjusted before merging (the website PR and 2 of the >>>> camel-quarkus PRs). >>>> >>>> This also fixes some links to wrong versions and deals with Publish docs >>>> for the last stable release apache/camel-quarkus#3191 >>>> <https://github.com/apache/camel-quarkus/issues/3191 >>>> <https://github.com/apache/camel-quarkus/issues/3191>> although I have a >>>> question about that since there seems to be a 2.3.x camel-quarkus so I >>>> wonder how many and which versions should be shown. Currently I’m showing >>>> latest, 2.3.x, and 2.2.x. >>>> >>>> I have no idea if I should open one or more issues to describe this work, >>>> or where such an issue should go. >>>> >>>> Thanks, >>>> David Jencks >>>> >>>> >>> >>> -- >>> Zoran Regvart >>> zo...@regvart.com <mailto:zo...@regvart.com> >>> >> >> >> -- >> Claus Ibsen >> - >> http://davsclaus.com <http://davsclaus.com/> @davsclaus >> Camel in Action 2: https://www.manning.com/ibsen2 >> <https://www.manning.com/ibsen2> >
Re: [Website and many projects] Please check the preview and review
Something is dreadfully wrong with the jenkins website build. It looks as if the jenkins build is not using the updated indexer and jenkins extensions. I don’t really understand how the yarn pnp stuff works, and I did not update any of the yarn caches…. I’m a bit scared off by stuff like this when I run yarn locally: ➤ YN: ┌ Link step ➤ YN0062: │ @netlify/traffic-mesh-agent-linux-x64@npm:0.27.0 The platform darwin is incompatible with this module, link skipped. ➤ YN0062: │ @netlify/traffic-mesh-agent-win32-x64@npm:0.27.0 The platform darwin is incompatible with this module, link skipped. Does yarn update the cache so it will work on any platform? It would be great if there was documentation in the README about what to do if you change the dependencies in package.json. The default values do appear in the preview. David Jencks > On Oct 23, 2021, at 2:09 AM, Claus Ibsen wrote: > > Hi > > The default value is not yet appearing. > > On Sat, Oct 23, 2021 at 10:03 AM Zoran Regvart wrote: >> >> Hi David, >> everything looks good, thank you for this effort! >> >> zoran >> >>> On 22 Oct 2021, at 02:45, David Jencks wrote: >>> >>> I’ve prepared a website update to the recent incompatible-syntax upgrade of >>> my indexer and jsonpath Antora extensions that build most of the tables and >>> a lot of content. The preview is at https://pr-648--camel.netlify.app >>> <https://pr-648--camel.netlify.app/>. I’d appreciate lots of eyes to see if >>> I broke anything >>> >>> I think this is ready to merge… there are 17 PRs to merge, some of which >>> need to be adjusted before merging (the website PR and 2 of the >>> camel-quarkus PRs). >>> >>> This also fixes some links to wrong versions and deals with Publish docs >>> for the last stable release apache/camel-quarkus#3191 >>> <https://github.com/apache/camel-quarkus/issues/3191> although I have a >>> question about that since there seems to be a 2.3.x camel-quarkus so I >>> wonder how many and which versions should be shown. Currently I’m showing >>> latest, 2.3.x, and 2.2.x. >>> >>> I have no idea if I should open one or more issues to describe this work, >>> or where such an issue should go. >>> >>> Thanks, >>> David Jencks >>> >>> >> >> -- >> Zoran Regvart >> zo...@regvart.com >> > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2
[Website and many projects] Please check the preview and review
I’ve prepared a website update to the recent incompatible-syntax upgrade of my indexer and jsonpath Antora extensions that build most of the tables and a lot of content. The preview is at https://pr-648--camel.netlify.app <https://pr-648--camel.netlify.app/>. I’d appreciate lots of eyes to see if I broke anything I think this is ready to merge… there are 17 PRs to merge, some of which need to be adjusted before merging (the website PR and 2 of the camel-quarkus PRs). This also fixes some links to wrong versions and deals with Publish docs for the last stable release apache/camel-quarkus#3191 <https://github.com/apache/camel-quarkus/issues/3191> although I have a question about that since there seems to be a 2.3.x camel-quarkus so I wonder how many and which versions should be shown. Currently I’m showing latest, 2.3.x, and 2.2.x. I have no idea if I should open one or more issues to describe this work, or where such an issue should go. Thanks, David Jencks
Re: [Website] Spring-boot errors or mistakes
Done, in my spring boot *-new-syntax branches. I did a full build on main-new-syntax which passed, I can do at least one more branch overnight and while I’m getting ready to create the other 13 (?) PRs. The site preview should be attached to https://github.com/apache/camel-website/pull/648. David Jencks > On Oct 19, 2021, at 10:12 PM, Claus Ibsen wrote: > > Hi David > > Yes you are correct, go ahead to remove/fix those tests > > On Tue, Oct 19, 2021 at 10:54 PM David Jencks > wrote: >> >> Tests in error: >> >> CamelDebeziumCommonTest.org.apache.camel.itest.springboot.CamelDebeziumCommonTest >> » Runtime >> org.apache.camel.itest.springboot.CamelHazelcastTest.componentTests(org.apache.camel.itest.springboot.CamelHazelcastTest) >> Run 1: CamelHazelcastTest>AbstractSpringBootTestSupport.startSpringBoot:44 >> » InvocationTarget >> Run 2: CamelHazelcastTest>AbstractSpringBootTestSupport.startSpringBoot:44 >> » InvocationTarget >> >> CamelHttpCommonTest.org.apache.camel.itest.springboot.CamelHttpCommonTest » >> Runtime >> CamelJettyCommonTest.org.apache.camel.itest.springboot.CamelJettyCommonTest >> » Runtime >> >> Tests run: 285, Failures: 0, Errors: 4, Skipped: 6 >> >> There actually are such classes under >> tests/camel-itest-spring-boot/src/test/java, and there’s no indication they >> are automatically generated. >> >> I removed CamelDebeziumCommonTest, CamelHttpCommonTest, and >> CamelJettyCommonTest which eliminated those 3 errors. The >> CamelHazelcastTest problem has a stack trace of >> >> Caused by: java.io.FileNotFoundException: class path resource >> [org/apache/camel/component/hazelcast/atomicnumber/springboot/customizer/HazelcastInstanceCustomizer.class] >> cannot be opened because it does not exist >>at >> org.springframework.core.io.ClassPathResource.getInputStream(ClassPathResource.java:187) >> ~[spring-core-5.3.10.jar!/:5.3.10] >>at >> org.springframework.core.type.classreading.SimpleMetadataReader.getClassReader(SimpleMetadataReader.java:55) >> ~[spring-core-5.3.10.jar!/:5.3.10] >> … >> and sure enough HazelcastInstanceCustomizer isn’t there any more… you >> removed it in >> >> commit a56ca8c09d5b6e50bb4891c623e09a470da0522e >> Author: Claus Ibsen >> Date: Sat Oct 9 09:54:37 2021 +0200 >> >>CAMEL-17056: camel-spring-boot-hazelcast-starter - Remove the old >> customer code as its standard in camel-core now >> but it’s still listed (with some others) in the spring.factories. >> >> Removing the HazelcastInstanceCustomizer lines from spring.factories has all >> the tests passing. >> >> I propose removing the excess projects in one commit in my upcoming PRs and >> then removing the HazelcastInstanceCustomizer lines in another commit, >> mentioning CAMEL-17056. >> >> David Jencks >> >>> On Oct 19, 2021, at 2:31 AM, Claus Ibsen wrote: >>> >>> And what integration test failures do you see when they are removed? >>> >>> eg maybe make sure that empty folders are removed etc, as the >>> integration test is some voodoo magic Nicola did that creates a spring >>> boot project per starter to see if it can install and startup spring >>> boot, and for some it also runs additional tests. >>> It's complex to maintain. >>> >>> On Tue, Oct 19, 2021 at 11:28 AM Claus Ibsen wrote: >>>> >>>> Hi >>>> >>>> Yeah these starters are not in use and should be removed >>>> >>>> camel-debezium-common-starter >>>> http-common-starter >>>> jetty-common-starter >>>> >>>> >>>> On Mon, Oct 18, 2021 at 8:34 PM David Jencks >>>> wrote: >>>>> >>>>> I can easily remove the empty starter json files, but if e.g. >>>>> camel-debezium-common-starter isn’t a starter, what is it? Removing the >>>>> 3 starter projects from camel-spring-boot results in integration test >>>>> failures. Does some other process install these as dependencies as >>>>> appropriate? Are they actually unnecessary but something needs more >>>>> tweaking? >>>>> >>>>> David Jencks >>>>> >>>>>> On Oct 18, 2021, at 12:17 AM, Claus Ibsen wrote: >>>>>> >>>>>> Hi >>>>>> >>>>>> The -common JARs are not starters, and they should not be lis
Re: [Website] Spring-boot errors or mistakes
Tests in error: CamelDebeziumCommonTest.org.apache.camel.itest.springboot.CamelDebeziumCommonTest » Runtime org.apache.camel.itest.springboot.CamelHazelcastTest.componentTests(org.apache.camel.itest.springboot.CamelHazelcastTest) Run 1: CamelHazelcastTest>AbstractSpringBootTestSupport.startSpringBoot:44 » InvocationTarget Run 2: CamelHazelcastTest>AbstractSpringBootTestSupport.startSpringBoot:44 » InvocationTarget CamelHttpCommonTest.org.apache.camel.itest.springboot.CamelHttpCommonTest » Runtime CamelJettyCommonTest.org.apache.camel.itest.springboot.CamelJettyCommonTest » Runtime Tests run: 285, Failures: 0, Errors: 4, Skipped: 6 There actually are such classes under tests/camel-itest-spring-boot/src/test/java, and there’s no indication they are automatically generated. I removed CamelDebeziumCommonTest, CamelHttpCommonTest, and CamelJettyCommonTest which eliminated those 3 errors. The CamelHazelcastTest problem has a stack trace of Caused by: java.io.FileNotFoundException: class path resource [org/apache/camel/component/hazelcast/atomicnumber/springboot/customizer/HazelcastInstanceCustomizer.class] cannot be opened because it does not exist at org.springframework.core.io.ClassPathResource.getInputStream(ClassPathResource.java:187) ~[spring-core-5.3.10.jar!/:5.3.10] at org.springframework.core.type.classreading.SimpleMetadataReader.getClassReader(SimpleMetadataReader.java:55) ~[spring-core-5.3.10.jar!/:5.3.10] … and sure enough HazelcastInstanceCustomizer isn’t there any more… you removed it in commit a56ca8c09d5b6e50bb4891c623e09a470da0522e Author: Claus Ibsen Date: Sat Oct 9 09:54:37 2021 +0200 CAMEL-17056: camel-spring-boot-hazelcast-starter - Remove the old customer code as its standard in camel-core now but it’s still listed (with some others) in the spring.factories. Removing the HazelcastInstanceCustomizer lines from spring.factories has all the tests passing. I propose removing the excess projects in one commit in my upcoming PRs and then removing the HazelcastInstanceCustomizer lines in another commit, mentioning CAMEL-17056. David Jencks > On Oct 19, 2021, at 2:31 AM, Claus Ibsen wrote: > > And what integration test failures do you see when they are removed? > > eg maybe make sure that empty folders are removed etc, as the > integration test is some voodoo magic Nicola did that creates a spring > boot project per starter to see if it can install and startup spring > boot, and for some it also runs additional tests. > It's complex to maintain. > > On Tue, Oct 19, 2021 at 11:28 AM Claus Ibsen wrote: >> >> Hi >> >> Yeah these starters are not in use and should be removed >> >> camel-debezium-common-starter >> http-common-starter >> jetty-common-starter >> >> >> On Mon, Oct 18, 2021 at 8:34 PM David Jencks >> wrote: >>> >>> I can easily remove the empty starter json files, but if e.g. >>> camel-debezium-common-starter isn’t a starter, what is it? Removing the 3 >>> starter projects from camel-spring-boot results in integration test >>> failures. Does some other process install these as dependencies as >>> appropriate? Are they actually unnecessary but something needs more >>> tweaking? >>> >>> David Jencks >>> >>>> On Oct 18, 2021, at 12:17 AM, Claus Ibsen wrote: >>>> >>>> Hi >>>> >>>> The -common JARs are not starters, and they should not be listed. >>>> >>>> On Mon, Oct 18, 2021 at 7:45 AM David Jencks >>> <mailto:david.a.jen...@gmail.com>> wrote: >>>>> >>>>> Locally I got the ability to show unused starter json files working, and >>>>> fixed the obvious problems: >>>>> >>>>> avro-rpc was using avro >>>>> paho-mqtt5 was using paho >>>>> in 3.7.x, grape and joor were set up wrong. >>>>> >>>>> There are still some “starters” i’m not sure about: >>>>> >>>>> debezium-common >>>>> http-common >>>>> jetty-common >>>>> >>>>> In a previous commit I added empty json files for these on the theory >>>>> that if they were starters they should have instructions on how to use >>>>> them, but they don’t have corresponding main camel doc pages. Are these >>>>> starters automatically installed by the corresponding non-common starter >>>>> artifacts such as camel-debezium-mongodb-starter? If so, I can remove the >>>>> empty json files. If not, how should their usage be documented? >>>>> >>>&g
Re: [Website] Spring-boot errors or mistakes
I can easily remove the empty starter json files, but if e.g. camel-debezium-common-starter isn’t a starter, what is it? Removing the 3 starter projects from camel-spring-boot results in integration test failures. Does some other process install these as dependencies as appropriate? Are they actually unnecessary but something needs more tweaking? David Jencks > On Oct 18, 2021, at 12:17 AM, Claus Ibsen wrote: > > Hi > > The -common JARs are not starters, and they should not be listed. > > On Mon, Oct 18, 2021 at 7:45 AM David Jencks <mailto:david.a.jen...@gmail.com>> wrote: >> >> Locally I got the ability to show unused starter json files working, and >> fixed the obvious problems: >> >> avro-rpc was using avro >> paho-mqtt5 was using paho >> in 3.7.x, grape and joor were set up wrong. >> >> There are still some “starters” i’m not sure about: >> >> debezium-common >> http-common >> jetty-common >> >> In a previous commit I added empty json files for these on the theory that >> if they were starters they should have instructions on how to use them, but >> they don’t have corresponding main camel doc pages. Are these starters >> automatically installed by the corresponding non-common starter artifacts >> such as camel-debezium-mongodb-starter? If so, I can remove the empty json >> files. If not, how should their usage be documented? >> >> I’m hoping to get my updated Antora extensions published in the next couple >> of days, at which point we can upgrade Camel to use them. >> >> David Jencks >> >>> On Oct 13, 2021, at 8:14 PM, David Jencks wrote: >>> >>> They are part of the camel-core-starter, although it’s hard to find the 2 >>> options per language among the 147 options for the starter. This suggests >>> that, for starters that encompass more than one component, it would be >>> useful to only show the options relevant to that component on the >>> components’ page. >>> >>> In any case I believe everything is working properly and I’ve merged all >>> the 9 PRs. >>> >>> The core and non-core languages are currently in different tables on the >>> spring-boot list page: I should be able to put them back into one table >>> after upgrading antora-indexer. >>> >>> David Jencks >>> >>>> On Oct 12, 2021, at 11:03 PM, David Jencks >>> <mailto:david.a.jen...@gmail.com> <mailto:david.a.jen...@gmail.com >>>> <mailto:david.a.jen...@gmail.com>>> wrote: >>>> >>>> There’s at least one problem…. (also asked on zulip) >>>> >>>> Are camel-core-languages actually part of camel-spring-boot? >>>> >>>> They are listed in the table in "list of starters" but with an artifactid >>>> of "camel-base" which isn't a starter. The current site doesn't have >>>> autoconfiguration info for them either. Since they are in the table, I >>>> added info to them for my PRs, but it's at least partly wrong, since it >>>> comes out as needing camel-core-languages-starter, which doesn't exist. >>>> cf. >>>> https://pr-644--camel.netlify.app/camel-spring-boot/latest/list.html#_camel_languages >>>> >>>> <https://pr-644--camel.netlify.app/camel-spring-boot/latest/list.html#_camel_languages> >>>> >>>> <https://pr-644--camel.netlify.app/camel-spring-boot/latest/list.html#_camel_languages >>>> >>>> <https://pr-644--camel.netlify.app/camel-spring-boot/latest/list.html#_camel_languages>>, >>>> >>>> https://pr-644--camel.netlify.app/components/latest/languages/header-language.html#_spring_boot_auto_configuration >>>> >>>> <https://pr-644--camel.netlify.app/components/latest/languages/header-language.html#_spring_boot_auto_configuration> >>>> >>>> <https://pr-644--camel.netlify.app/components/latest/languages/header-language.html#_spring_boot_auto_configuration >>>> >>>> <https://pr-644--camel.netlify.app/components/latest/languages/header-language.html#_spring_boot_auto_configuration>>. >>>> Since they say they are part of camel-core, I've used core.json to >>>> construct the autoconfig options table, but that doesn't look very >>>> relevant to me. >>>> >>>> Is there a starter needed for them? If so, which one? camel-core-starter >>>> seems like a possibility, is it cor
Re: [Website] Spring-boot errors or mistakes
Locally I got the ability to show unused starter json files working, and fixed the obvious problems: avro-rpc was using avro paho-mqtt5 was using paho in 3.7.x, grape and joor were set up wrong. There are still some “starters” i’m not sure about: debezium-common http-common jetty-common In a previous commit I added empty json files for these on the theory that if they were starters they should have instructions on how to use them, but they don’t have corresponding main camel doc pages. Are these starters automatically installed by the corresponding non-common starter artifacts such as camel-debezium-mongodb-starter? If so, I can remove the empty json files. If not, how should their usage be documented? I’m hoping to get my updated Antora extensions published in the next couple of days, at which point we can upgrade Camel to use them. David Jencks > On Oct 13, 2021, at 8:14 PM, David Jencks wrote: > > They are part of the camel-core-starter, although it’s hard to find the 2 > options per language among the 147 options for the starter. This suggests > that, for starters that encompass more than one component, it would be useful > to only show the options relevant to that component on the components’ page. > > In any case I believe everything is working properly and I’ve merged all the > 9 PRs. > > The core and non-core languages are currently in different tables on the > spring-boot list page: I should be able to put them back into one table after > upgrading antora-indexer. > > David Jencks > >> On Oct 12, 2021, at 11:03 PM, David Jencks > <mailto:david.a.jen...@gmail.com>> wrote: >> >> There’s at least one problem…. (also asked on zulip) >> >> Are camel-core-languages actually part of camel-spring-boot? >> >> They are listed in the table in "list of starters" but with an artifactid of >> "camel-base" which isn't a starter. The current site doesn't have >> autoconfiguration info for them either. Since they are in the table, I added >> info to them for my PRs, but it's at least partly wrong, since it comes out >> as needing camel-core-languages-starter, which doesn't exist. cf. >> https://pr-644--camel.netlify.app/camel-spring-boot/latest/list.html#_camel_languages >> >> <https://pr-644--camel.netlify.app/camel-spring-boot/latest/list.html#_camel_languages>, >> >> https://pr-644--camel.netlify.app/components/latest/languages/header-language.html#_spring_boot_auto_configuration >> >> <https://pr-644--camel.netlify.app/components/latest/languages/header-language.html#_spring_boot_auto_configuration>. >> Since they say they are part of camel-core, I've used core.json to construct >> the autoconfig options table, but that doesn't look very relevant to me. >> >> Is there a starter needed for them? If so, which one? camel-core-starter >> seems like a possibility, is it correct? >> >> If no starter is needed, why are they in the spring-boot starters table? >> >> I hope to be able to merge everything tomorrow, it got too late tonight. >> >> David Jencks >> >>> On Oct 12, 2021, at 7:41 PM, David Jencks >> <mailto:david.a.jen...@gmail.com>> wrote: >>> >>> The part of this work I’m ready for now is done, except for some >>> commit-squashing and dealing with check style errors :-) >>> >>> Preview at https://pr-644--camel.netlify.app >>> <https://pr-644--camel.netlify.app/> >>> >>> I seem to only be able to write AsciiDoc by now…. :-) Here’s a list of the >>> differences between table rows new/old: >>> >>> —— >>> = Differences between spring-boot list-old and list >>> >>> == latest (main) >>> >>> === In list, not list-old: >>> >>> * disruptor-vm-component >>> * splunk-hec-component >>> >>> * (3 bindy dataformats collapsed to one link, corresponding to the single >>> main bindy doc page) >>> >>> * csimple-language >>> >>> == 3.12.x >>> === In list, not list-old: >>> >>> * disruptor-vm-component >>> * splunk-hec-component >>> >>> * (3 bindy dataformats collapsed to one link, corresponding to the single >>> main bindy doc page) >>> >>> * csimple-language >>> >>> == 3.11.x >>> === In list, not list-old: >>> >>> * splunk-hec-component >>> * (3 bindy dataformats collapsed to one link, corresponding to the single >>> main bindy doc page) >>> * csimple-language >&g
Re: [Website] Spring-boot errors or mistakes
They are part of the camel-core-starter, although it’s hard to find the 2 options per language among the 147 options for the starter. This suggests that, for starters that encompass more than one component, it would be useful to only show the options relevant to that component on the components’ page. In any case I believe everything is working properly and I’ve merged all the 9 PRs. The core and non-core languages are currently in different tables on the spring-boot list page: I should be able to put them back into one table after upgrading antora-indexer. David Jencks > On Oct 12, 2021, at 11:03 PM, David Jencks wrote: > > There’s at least one problem…. (also asked on zulip) > > Are camel-core-languages actually part of camel-spring-boot? > > They are listed in the table in "list of starters" but with an artifactid of > "camel-base" which isn't a starter. The current site doesn't have > autoconfiguration info for them either. Since they are in the table, I added > info to them for my PRs, but it's at least partly wrong, since it comes out > as needing camel-core-languages-starter, which doesn't exist. cf. > https://pr-644--camel.netlify.app/camel-spring-boot/latest/list.html#_camel_languages > > <https://pr-644--camel.netlify.app/camel-spring-boot/latest/list.html#_camel_languages>, > > https://pr-644--camel.netlify.app/components/latest/languages/header-language.html#_spring_boot_auto_configuration > > <https://pr-644--camel.netlify.app/components/latest/languages/header-language.html#_spring_boot_auto_configuration>. > Since they say they are part of camel-core, I've used core.json to construct > the autoconfig options table, but that doesn't look very relevant to me. > > Is there a starter needed for them? If so, which one? camel-core-starter > seems like a possibility, is it correct? > > If no starter is needed, why are they in the spring-boot starters table? > > I hope to be able to merge everything tomorrow, it got too late tonight. > > David Jencks > >> On Oct 12, 2021, at 7:41 PM, David Jencks > <mailto:david.a.jen...@gmail.com>> wrote: >> >> The part of this work I’m ready for now is done, except for some >> commit-squashing and dealing with check style errors :-) >> >> Preview at https://pr-644--camel.netlify.app >> <https://pr-644--camel.netlify.app/> >> >> I seem to only be able to write AsciiDoc by now…. :-) Here’s a list of the >> differences between table rows new/old: >> >> —— >> = Differences between spring-boot list-old and list >> >> == latest (main) >> >> === In list, not list-old: >> >> * disruptor-vm-component >> * splunk-hec-component >> >> * (3 bindy dataformats collapsed to one link, corresponding to the single >> main bindy doc page) >> >> * csimple-language >> >> == 3.12.x >> === In list, not list-old: >> >> * disruptor-vm-component >> * splunk-hec-component >> >> * (3 bindy dataformats collapsed to one link, corresponding to the single >> main bindy doc page) >> >> * csimple-language >> >> == 3.11.x >> === In list, not list-old: >> >> * splunk-hec-component >> * (3 bindy dataformats collapsed to one link, corresponding to the single >> main bindy doc page) >> * csimple-language >> >> Moved to separate table: >> >> * camel-spring-cloud >> * camel-spring-cloud-consul >> * camel-spring-cloud-netflix >> * camel-spring-cloud-zookeeper >> >> == 3.7.x >> === In list, not list-old: >> * splunk-hec-component >> * (3 bindy dataformats collapsed to one link, corresponding to the single >> main bindy doc page) >> * csimple-language >> >> >> Moved to separate table: >> >> * camel-spring-cloud >> * camel-spring-cloud-consul >> * camel-spring-cloud-netflix >> * camel-spring-cloud-zookeeper >> >> —— >> >> There are also some presentation differences in the tables, the new ones >> follow the tables in main camel components more closely, e.g. >> >> > class="tableblock">Stable >> >> >> > class="tableblock">Stable-deprecated >> >> as the deprecation marker. >> >> Also quite a few core languages now list the starter as >> >> > class="tableblock">camel-core-languages-starter >> rather than >> > class="tableblock">camel-base >> >> Is this correct? >> >> There are 8 PRs: >> (Camel) >> Main spring
Re: [Website] Spring-boot errors or mistakes
There’s at least one problem…. (also asked on zulip) Are camel-core-languages actually part of camel-spring-boot? They are listed in the table in "list of starters" but with an artifactid of "camel-base" which isn't a starter. The current site doesn't have autoconfiguration info for them either. Since they are in the table, I added info to them for my PRs, but it's at least partly wrong, since it comes out as needing camel-core-languages-starter, which doesn't exist. cf. https://pr-644--camel.netlify.app/camel-spring-boot/latest/list.html#_camel_languages <https://pr-644--camel.netlify.app/camel-spring-boot/latest/list.html#_camel_languages>, https://pr-644--camel.netlify.app/components/latest/languages/header-language.html#_spring_boot_auto_configuration <https://pr-644--camel.netlify.app/components/latest/languages/header-language.html#_spring_boot_auto_configuration>. Since they say they are part of camel-core, I've used core.json to construct the autoconfig options table, but that doesn't look very relevant to me. Is there a starter needed for them? If so, which one? camel-core-starter seems like a possibility, is it correct? If no starter is needed, why are they in the spring-boot starters table? I hope to be able to merge everything tomorrow, it got too late tonight. David Jencks > On Oct 12, 2021, at 7:41 PM, David Jencks wrote: > > The part of this work I’m ready for now is done, except for some > commit-squashing and dealing with check style errors :-) > > Preview at https://pr-644--camel.netlify.app > <https://pr-644--camel.netlify.app/> > > I seem to only be able to write AsciiDoc by now…. :-) Here’s a list of the > differences between table rows new/old: > > —— > = Differences between spring-boot list-old and list > > == latest (main) > > === In list, not list-old: > > * disruptor-vm-component > * splunk-hec-component > > * (3 bindy dataformats collapsed to one link, corresponding to the single > main bindy doc page) > > * csimple-language > > == 3.12.x > === In list, not list-old: > > * disruptor-vm-component > * splunk-hec-component > > * (3 bindy dataformats collapsed to one link, corresponding to the single > main bindy doc page) > > * csimple-language > > == 3.11.x > === In list, not list-old: > > * splunk-hec-component > * (3 bindy dataformats collapsed to one link, corresponding to the single > main bindy doc page) > * csimple-language > > Moved to separate table: > > * camel-spring-cloud > * camel-spring-cloud-consul > * camel-spring-cloud-netflix > * camel-spring-cloud-zookeeper > > == 3.7.x > === In list, not list-old: > * splunk-hec-component > * (3 bindy dataformats collapsed to one link, corresponding to the single > main bindy doc page) > * csimple-language > > > Moved to separate table: > > * camel-spring-cloud > * camel-spring-cloud-consul > * camel-spring-cloud-netflix > * camel-spring-cloud-zookeeper > > —— > > There are also some presentation differences in the tables, the new ones > follow the tables in main camel components more closely, e.g. > > class="tableblock">Stable > >> > class="tableblock">Stable-deprecated > > as the deprecation marker. > > Also quite a few core languages now list the starter as > > class="tableblock">camel-core-languages-starter > rather than > class="tableblock">camel-base > > Is this correct? > > There are 8 PRs: > (Camel) > Main spring boot jsonpath (allow me to squash and merge) > <https://github.com/apache/camel/pull/6261> > Camel 3.12.x spring boot jsonpath (please let me squash and merge) > <https://github.com/apache/camel/pull/6263> > Camel 3.11.x spring boot jsonpath (please let me squash and merge) > <https://github.com/apache/camel/pull/6264> > Camel 3.7.x spring boot jsonpath (please let me squash and merge) > <https://github.com/apache/camel/pull/6262> > (Camel-spring-boot) > Main jsonpath (please let me squash and merge) > <https://github.com/apache/camel-spring-boot/pull/388> > Camel spring boot 3.12.x jsonpath (please let me squash and merge) > <https://github.com/apache/camel-spring-boot/pull/387> > Camel spring boot 3.11.x jsonpath (please let me squash and merge) > <https://github.com/apache/camel-spring-boot/pull/386> > Camel spring boot 3.7.x jsonpath (please let me squash and merge) > <https://github.com/apache/camel-spring-boot/pull/385> > > If everything looks good I plan to squash each PR to 2 or 3 commits (code > changes and generated changes) and rebase/merge. I don’t think any pla
Re: [Website] camel-spring-boot docs
I’ve implemented (3) and (4) on all active camel-spring-boot branches, I think they are pretty much ready to merge. Discussion is on another thread, "[Website] Spring-boot errors or mistakes” I don’t think I have anything else to say on this thread. Thanks David Jencks > On Oct 7, 2021, at 3:17 PM, David Jencks wrote: > > (1) and (2) are now completely implemented on all active camel-spring-boot > branches. I’m working on (3). > > David Jencks > >> On Oct 2, 2021, at 6:02 PM, David Jencks > <mailto:david.a.jen...@gmail.com>> wrote: >> >> I went ahead and implemented the first two of these in main “latest” >> branches. Other than removing 324 unreferenced and inaccessible pages, I >> don’t see any changes (There’s one slight change to the single reference to >> one of the removed pages: (diff is new to old) >> >> < Also add any component starters >> your Spring Boot application requires. For example this adds the > href="../../components/3.11.x/activemq-component.html#_spring_boot_auto_configuration" >> class="page">auto-configuration starter for the > href="../../components/3.11.x/activemq-component.html" class="page">ActiveMQ >> component. >> --- >> > And any component starters your >> > Spring Boot application requires. For example this adds the > > href="activemq-starter.html" class="page">starter for the > > href="../../components/3. >> >> (instead of linking to an otherwise inaccessible page, it links to the >> section where that content is included in the activemq component page.) >> >> PRs: >> https://github.com/apache/camel-website/pull/641 >> <https://github.com/apache/camel-website/pull/641> (incomplete, pending >> merging other two or 8, see below) >> https://github.com/apache/camel-spring-boot/pull/374 >> <https://github.com/apache/camel-spring-boot/pull/374> (moves generated >> content to "partials" in a separate directory tree) >> https://github.com/apache/camel/pull/6200 >> <https://github.com/apache/camel/pull/6200> (changes includes of this >> content) >> >> Since this removes a lot of pages, I’d like to apply this idea to all the >> active camel-spring-boot branches (latest, 3.12.x, 3.11.x, 3.7.x). >> >> I was hoping to replace copying the AsciiDoc files with symlinks, but I’d >> forgotten how cantankerous Ant is and couldn’t find a way to use it’s >> symlink task, and I’m hesitant to bring node/gulp in to the project just to >> make some symlinks. >> >> David Jencks >> >>> On Oct 2, 2021, at 10:17 AM, David Jencks >> <mailto:david.a.jen...@gmail.com>> wrote: >>> >>> I always forget that for Antora terms often have a specific meaning that >>> isn’t that common or obvious :-) >>> >>>> On Oct 2, 2021, at 12:45 AM, Claus Ibsen >>> <mailto:claus.ib...@gmail.com>> wrote: >>>> >>>> On Fri, Oct 1, 2021 at 11:30 PM David Jencks >>> <mailto:david.a.jen...@gmail.com>> wrote: >>>>> >>>>> I looked at camel-spring-boot a little bit and have several questions and >>>>> suggestions… if there’s agreement I’ll open some issues and work on them. >>>>> >>>>> 1. The AsciiDoc pages are only included in the main >>>>> component/dataformat/… pages, and not accessible through navigation >>>>> standalone. I think they should be partials, not standalone pages. >>>>> >>>> >>>> You can possible not do this as the ascii doc is generated with >>>> information from spring boot itself, there are some camel spring boot >>>> started components that have spring boot auto configuration, >>>> that information are not in the camel-catalog, but are stored in >>>> spring boot style (inside the JAR in META-INF there is a spring boot >>>> json file). >>>> >>>> >>> >>> This just involves moving the target location of the generated pages to >>> modules/ROOT/partials from modules/ROOT/pages, and in main camel components >>> changing the include::….page$... to include::….partial$… >>> This would not be a good idea if there was a firm plan to, at some point, >>> also have this information on standalone pages, or convert the current >>> include:: to a link to a standalone page. Otherwise, it’s simple and >>> shouldn’t disrupt anything. The generated
Re: [Website] Spring-boot errors or mistakes
The part of this work I’m ready for now is done, except for some commit-squashing and dealing with check style errors :-) Preview at https://pr-644--camel.netlify.app <https://pr-644--camel.netlify.app/> I seem to only be able to write AsciiDoc by now…. :-) Here’s a list of the differences between table rows new/old: —— = Differences between spring-boot list-old and list == latest (main) === In list, not list-old: * disruptor-vm-component * splunk-hec-component * (3 bindy dataformats collapsed to one link, corresponding to the single main bindy doc page) * csimple-language == 3.12.x === In list, not list-old: * disruptor-vm-component * splunk-hec-component * (3 bindy dataformats collapsed to one link, corresponding to the single main bindy doc page) * csimple-language == 3.11.x === In list, not list-old: * splunk-hec-component * (3 bindy dataformats collapsed to one link, corresponding to the single main bindy doc page) * csimple-language Moved to separate table: * camel-spring-cloud * camel-spring-cloud-consul * camel-spring-cloud-netflix * camel-spring-cloud-zookeeper == 3.7.x === In list, not list-old: * splunk-hec-component * (3 bindy dataformats collapsed to one link, corresponding to the single main bindy doc page) * csimple-language Moved to separate table: * camel-spring-cloud * camel-spring-cloud-consul * camel-spring-cloud-netflix * camel-spring-cloud-zookeeper —— There are also some presentation differences in the tables, the new ones follow the tables in main camel components more closely, e.g. Stable >> Stable-deprecated as the deprecation marker. Also quite a few core languages now list the starter as camel-core-languages-starter rather than camel-base Is this correct? There are 8 PRs: (Camel) Main spring boot jsonpath (allow me to squash and merge) <https://github.com/apache/camel/pull/6261> Camel 3.12.x spring boot jsonpath (please let me squash and merge) <https://github.com/apache/camel/pull/6263> Camel 3.11.x spring boot jsonpath (please let me squash and merge) <https://github.com/apache/camel/pull/6264> Camel 3.7.x spring boot jsonpath (please let me squash and merge) <https://github.com/apache/camel/pull/6262> (Camel-spring-boot) Main jsonpath (please let me squash and merge) <https://github.com/apache/camel-spring-boot/pull/388> Camel spring boot 3.12.x jsonpath (please let me squash and merge) <https://github.com/apache/camel-spring-boot/pull/387> Camel spring boot 3.11.x jsonpath (please let me squash and merge) <https://github.com/apache/camel-spring-boot/pull/386> Camel spring boot 3.7.x jsonpath (please let me squash and merge) <https://github.com/apache/camel-spring-boot/pull/385> If everything looks good I plan to squash each PR to 2 or 3 commits (code changes and generated changes) and rebase/merge. I don’t think any playbook changes are needed. I’m completing a big upgrade of the antora-indexer and jsonpath extensions, and plan to finish that next. It will involve changing the syntax of most uses of these extensions. After that I expect to be able to implement automated checking for starters that aren’t linked to by main camel components. I think there are about 5, but have no way to find them at this point. David Jencks > On Oct 12, 2021, at 12:01 AM, Claus Ibsen wrote: > > On Mon, Oct 11, 2021 at 8:07 AM David Jencks <mailto:david.a.jen...@gmail.com>> wrote: >> >> I’ve been working on generating the spring-boot autoconfig info from the >> json files generated during the spring-boot build, and the spring boot >> “list.adoc” tables using the indexer extension (like the components tables >> are generated). >> >> My understanding of the spring-boot stuff is that: >> >> - any Camel component can be used in Spring >> >> - Some camel components/dataformats… can participate in spring-boot auto >> configure. These components are identified as having a corresponding >> camel-*-starter project in the spring-boot repo. >> >> - There are 3 kinds of these: >> >> 1. There’s (generated) java code and a generated json file >> >> 2. There’s generated java code but no generated json file (e.g. jasypt) >> >> 3. There’s no java code and no json file, just some properties files etc. >> (e.g. aws-xray) >> >> I’m really not familiar with spring-boot or the philosophy behind it, but my >> guess is that if there’s a spring-boot starter jar then spring will create a >> singleton instance of something in the jar, and if there’s a json file that >> will guide or describe configuring this instance with data from somewhere. >> If this is roughly correct, then the 2nd and 3rd kinds correspond to >> non-configurable instances. >> >> I’ve discovered tha
[Website] Spring-boot errors or mistakes
I’ve been working on generating the spring-boot autoconfig info from the json files generated during the spring-boot build, and the spring boot “list.adoc” tables using the indexer extension (like the components tables are generated). My understanding of the spring-boot stuff is that: - any Camel component can be used in Spring - Some camel components/dataformats… can participate in spring-boot auto configure. These components are identified as having a corresponding camel-*-starter project in the spring-boot repo. - There are 3 kinds of these: 1. There’s (generated) java code and a generated json file 2. There’s generated java code but no generated json file (e.g. jasypt) 3. There’s no java code and no json file, just some properties files etc. (e.g. aws-xray) I’m really not familiar with spring-boot or the philosophy behind it, but my guess is that if there’s a spring-boot starter jar then spring will create a singleton instance of something in the jar, and if there’s a json file that will guide or describe configuring this instance with data from somewhere. If this is roughly correct, then the 2nd and 3rd kinds correspond to non-configurable instances. I’ve discovered that: - several components have starters that are not listed in the existing list - quite a few components main adoc documentation do not include the relevant autoconfig information (even if it’s “no options”) Most of these are linked to from the “list” page, so I’d imagine it would be pretty confusing to follow the link and find no information about spring-boot. Evidently, the current process is not maintainable. The (new, jsonpath based) autoconfig doc generation is based on putting the name of the appropriate json file in the main adoc file as a header attribute. I think we can produce something more resilient by: - for kinds (2) and (3), put a no-options appropriately named json file in the starter project under src/main/docs. This will not get added to the starter jar, unlike the properly generated json files. - put the corresponding header attribute in the appropriate main adoc file Now we should be able to do a basic check, that the number of unique json file names from header attributes is equal to the number of json files, which should be equal to the number of starters. This should be possible to do as part of the website build. This won’t check that for instance with a starter that serves many components, all the components will refer to the starter information, but it will detect most or all of the problems I’v found so far. The preview for https://github.com/apache/camel-website/pull/644 shows how far I’ve gotten with this: the appropriate page to look at is https://pr-644--camel.netlify.app/camel-spring-boot/latest/list-2.html This has, in addition to the tables of spring-boot autoconfigure-enabled components, counts of json files used and existing (there are 5 unused), and tables of non-spring-boot-enabled components/dataformats… All the dataformats and languages are set up properly. There are 2 components that aren’t spring-boot-enabled and a whole lot of “others" I think that actually failing the build when there are unused .json files and also finding out which json files are unused will have to wait for an upgrade to the antora-indexer… unfortunately this will require some syntax changes. I’d like to polish these changes, replace the list page with the new one (with the extra info removed), port this work to the other relevant branches, and then pursue releasing the upgraded antora-indexer extension and upgrading the syntax, and then work on making the checks work. Does this seem reasonable? —— Another feature of the preview is that I found out how to make the option names in all the jsonpath generated tables link to themselves and display a marker on hover, just like section titles, so you can now easily copy the link to an option and paste it somewhere. (the names have been links for a while, but there was no practical way to find out what the link was!) E.g. … https://pr-644--camel.netlify.app/components/latest/amqp-component.html#_endpoint_query_option_disableReplyTo David Jencks
[Website] camel-spring-boot options table generation -- Advice needed.
I’ve been working on converting the spring-boot auto configure documentation at the end of each component page to be generated by Antora from the json information source. At the moment after a quick look there seem to be 3 classes of change from the current docs, and I need advice about what to do with them: 1. These appear to me to fix likely copy-paste errors, e.g.: diff -r documentation-post-rearrange/components/latest/xquery-component.html documentation-jsonpath/components/latest/xquery-component.html 2784c2785 < When using saxon with Spring Boot make sure to use the following Maven dependency to have support for auto configuration: --- > When using xquery with Spring Boot make sure to use the following Maven > dependency to have support for auto configuration: I assume fixing these is a good idea. 2. When there are several components in the same spring boot starter, each component docs page is changed to specify the component rather than the collection of components, e.g.: diff -r documentation-post-rearrange/components/latest/xmlsecurity-verify-component.html documentation-jsonpath/components/latest/xmlsecurity-verify-component.html 3097c3098 < When using xmlsecurity with Spring Boot make sure to use the following Maven dependency to have support for auto configuration: --- > When using xmlsecurity-verify with Spring Boot make sure to use the > following Maven dependency to have support for auto configuration: I think this change is fine but it shouldn’t be too hard to produce the old behavior, although it will probably involve another manually maintained attribute in every component adoc. 3. http://… text in descriptions are now rendered as links, e.g.: diff -r documentation-post-rearrange/components/latest/xmlsecurity-verify-component.html documentation-jsonpath/components/latest/xmlsecurity-verify-component.html 3274c3275 < Signature algorithm. Default value is http://www.w3.org/2000/09/xmldsig#rsa-sha1. --- > Signature > algorithm. Default value is href="http://www.w3.org/2000/09/xmldsig#rsa-sha1; > class="bare">http://www.w3.org/2000/09/xmldsig#rsa-sha1. Some of these links, including, I think, this one, definitely make sense, and some are more questionable, such as turning an xml namespace URI into a link. The one I checked did lead to the schema for the namespace, but I didn’t check all of them. I haven’t yet investigated why the current code does not produce links. I’m not sure how easy it will be, but it makes sense to me to keep the non-broken links as actual links rather than text “https://…”. I guess the website PR preview build will tell us if (3) introduces broken links. So, there may be a preview associated with https://github.com/apache/camel-website/pull/644 soon. David Jencks
Re: [Website] camel-spring-boot docs
(1) and (2) are now completely implemented on all active camel-spring-boot branches. I’m working on (3). David Jencks > On Oct 2, 2021, at 6:02 PM, David Jencks wrote: > > I went ahead and implemented the first two of these in main “latest” > branches. Other than removing 324 unreferenced and inaccessible pages, I > don’t see any changes (There’s one slight change to the single reference to > one of the removed pages: (diff is new to old) > > < Also add any component starters > your Spring Boot application requires. For example this adds the href="../../components/3.11.x/activemq-component.html#_spring_boot_auto_configuration" > class="page">auto-configuration starter for the href="../../components/3.11.x/activemq-component.html" class="page">ActiveMQ > component. > --- > > And any component starters your > > Spring Boot application requires. For example this adds the > href="activemq-starter.html" class="page">starter for the > href="../../components/3. > > (instead of linking to an otherwise inaccessible page, it links to the > section where that content is included in the activemq component page.) > > PRs: > https://github.com/apache/camel-website/pull/641 > <https://github.com/apache/camel-website/pull/641> (incomplete, pending > merging other two or 8, see below) > https://github.com/apache/camel-spring-boot/pull/374 > <https://github.com/apache/camel-spring-boot/pull/374> (moves generated > content to "partials" in a separate directory tree) > https://github.com/apache/camel/pull/6200 > <https://github.com/apache/camel/pull/6200> (changes includes of this content) > > Since this removes a lot of pages, I’d like to apply this idea to all the > active camel-spring-boot branches (latest, 3.12.x, 3.11.x, 3.7.x). > > I was hoping to replace copying the AsciiDoc files with symlinks, but I’d > forgotten how cantankerous Ant is and couldn’t find a way to use it’s symlink > task, and I’m hesitant to bring node/gulp in to the project just to make some > symlinks. > > David Jencks > >> On Oct 2, 2021, at 10:17 AM, David Jencks > <mailto:david.a.jen...@gmail.com>> wrote: >> >> I always forget that for Antora terms often have a specific meaning that >> isn’t that common or obvious :-) >> >>> On Oct 2, 2021, at 12:45 AM, Claus Ibsen >> <mailto:claus.ib...@gmail.com>> wrote: >>> >>> On Fri, Oct 1, 2021 at 11:30 PM David Jencks >> <mailto:david.a.jen...@gmail.com>> wrote: >>>> >>>> I looked at camel-spring-boot a little bit and have several questions and >>>> suggestions… if there’s agreement I’ll open some issues and work on them. >>>> >>>> 1. The AsciiDoc pages are only included in the main component/dataformat/… >>>> pages, and not accessible through navigation standalone. I think they >>>> should be partials, not standalone pages. >>>> >>> >>> You can possible not do this as the ascii doc is generated with >>> information from spring boot itself, there are some camel spring boot >>> started components that have spring boot auto configuration, >>> that information are not in the camel-catalog, but are stored in >>> spring boot style (inside the JAR in META-INF there is a spring boot >>> json file). >>> >>> >> >> This just involves moving the target location of the generated pages to >> modules/ROOT/partials from modules/ROOT/pages, and in main camel components >> changing the include::….page$... to include::….partial$… >> This would not be a good idea if there was a firm plan to, at some point, >> also have this information on standalone pages, or convert the current >> include:: to a link to a standalone page. Otherwise, it’s simple and >> shouldn’t disrupt anything. The generated spring boot docs would still be in >> the camel-spring-boot repo. >> >>> >>>> 2. The individual generated pages are tied 1-1 with the ‘components’ >>>> component. I think, even though they are (at least currently) in a >>>> different repo having them in the ‘components’ component as part of a >>>> distributed component makes more sense than having them in a different >>>> component. >>>> >>> >>> Not sure what you mean? >> >> We’d need 2 directories in camel-spring-boot docs, say >> >> components/modules/spring-boot/partials where all the individual generated >> pages go, with a c
Re: Website build problem??? Apparently a transient failure
I tried again and the CI build seems to work, so perhaps this was a transient
network problem.
David Jencks
> On Oct 7, 2021, at 10:46 AM, David Jencks wrote:
>
> I’m not sure if I caused this problem or not… if I did I have no idea how.
>
> I apologize for merging my 2 PRs before checking the check status, but
> yesterday the similar PR was passing the check.
>
> The PR checks are failing the cache check step:
>
> https://github.com/apache/camel-website/runs/3829903801
> <https://github.com/apache/camel-website/runs/3829903801>
>
> 2021-10-07T16:52:25.6855704Z ##[group]Run yarn workspaces foreach install
> --check-cache
> 2021-10-07T16:52:25.6856563Z [36;1myarn workspaces foreach install
> --check-cache[0m
> 2021-10-07T16:52:25.6900044Z shell: /usr/bin/bash -e {0}
> 2021-10-07T16:52:25.6900448Z env:
> 2021-10-07T16:52:25.6900861Z CAMEL_ENV: production
> 2021-10-07T16:52:25.6901351Z HUGO_OPTIONS: --buildFuture
> 2021-10-07T16:52:25.6901817Z ##[endgroup]
> 2021-10-07T16:52:29.4750156Z [94m➤[39m [90mYN[39m: [94m➤[39m
> [90mYN[39m: ┌ Resolution step
> 2021-10-07T16:52:29.4752947Z [94m➤[39m [90mYN[39m:
> ::group::Resolution step
> 2021-10-07T16:52:29.8626667Z [94m➤[39m [90mYN[39m: [93m➤[39m
> YN0002: │
> [38;5;166m@octokit/[39m[38;5;173mrest[39m[38;5;111m@[39m[38;5;111mnpm:16.43.2[39m
> doesn't provide [38;5;166m@octokit/[39m[38;5;173mcore[39m
> ([38;5;111mpb8862[39m), requested by
> [38;5;166m@octokit/[39m[38;5;173mplugin-request-log[39m
> 2021-10-07T16:52:29.8630211Z [94m➤[39m [90mYN[39m: [93m➤[39m
> YN: │ Some peer dependencies are incorrectly met; run [38;5;111myarn
> explain peer-requirements [39m for details, where
> [38;5;111m[39m is the six-letter p-prefixed code
> 2021-10-07T16:52:29.8632409Z [94m➤[39m [90mYN[39m: ::endgroup::
> 2021-10-07T16:52:29.8633662Z [94m➤[39m [90mYN[39m: [94m➤[39m
> [90mYN[39m: └ Completed in 0s 387ms
> 2021-10-07T16:52:29.8637018Z [94m➤[39m [90mYN[39m: [94m➤[39m
> [90mYN[39m: ┌ Fetch step
> 2021-10-07T16:52:29.8638470Z [94m➤[39m [90mYN[39m: ::group::Fetch step
> 2021-10-07T16:52:57.9922578Z [94m➤[39m [90mYN[39m: [91m➤[39m
> YN0001: │ HTTPError: Response code 404 (Not Found)
> 2021-10-07T16:52:57.9924920Z [94m➤[39m [90mYN[39m: at
> se.
> (/home/runner/work/camel-website/camel-website/.yarn/releases/yarn-2.4.0.cjs:23:10082)
> 2021-10-07T16:52:57.9926599Z [94m➤[39m [90mYN[39m: at
> runMicrotasks ()
> 2021-10-07T16:52:57.9928614Z [94m➤[39m [90mYN[39m: at
> processTicksAndRejections (internal/process/task_queues.js:95:5)
> 2021-10-07T16:54:25.0530166Z [94m➤[39m [90mYN[39m: [94m➤[39m
> YN0013: │ 2058 packages were already cached
> 2021-10-07T16:54:25.0536341Z [94m➤[39m [90mYN[39m: ::endgroup::
> 2021-10-07T16:54:25.0537667Z [94m➤[39m [90mYN[39m: [94m➤[39m
> [90mYN[39m: └ Completed in 1m 56s
> 2021-10-07T16:54:25.0541475Z [94m➤[39m [90mYN[39m: [91m➤[39m
> YN: Failed with errors in 1m 56s
> 2021-10-07T16:54:25.2992839Z [94m➤[39m [90mYN[39m: [94m➤[39m
> [90mYN[39m: ┌ Resolution step
> 2021-10-07T16:54:25.2997310Z [94m➤[39m [90mYN[39m:
> ::group::Resolution step
> 2021-10-07T16:54:25.5036838Z [94m➤[39m [90mYN[39m: ::endgroup::
> 2021-10-07T16:54:25.5040763Z [94m➤[39m [90mYN[39m: [94m➤[39m
> [90mYN[39m: └ Completed in 0s 204ms
> 2021-10-07T16:54:25.5045422Z [94m➤[39m [90mYN[39m: [94m➤[39m
> [90mYN[39m: ┌ Fetch step
> 2021-10-07T16:54:25.5049174Z [94m➤[39m [90mYN[39m: ::group::Fetch step
> 2021-10-07T16:55:23.9082468Z [94m➤[39m [90mYN[39m: [94m➤[39m
> YN0013: │ 1258 packages were already cached
> 2021-10-07T16:55:23.9169053Z [94m➤[39m [90mYN[39m: ::endgroup::
> 2021-10-07T16:55:23.9172452Z [94m➤[39m [90mYN[39m: [94m➤[39m
> [90mYN[39m: └ Completed in 58s 412ms
> 2021-10-07T16:55:24.0775558Z [94m➤[39m [90mYN[39m: [94m➤[39m
> [90mYN[39m: ┌ Link step
> 2021-10-07T16:55:24.0780194Z [94m➤[39m [90mYN[39m: ::group::Link step
> 2021-10-07T16:56:09.9791888Z [94m➤[39m [90mYN[39m: [93m➤[39m
> YN0062: │
> [38;5;173mfsevents[39m[38;5;111m@[39m[38;5;111mpatch:fsevents@npm%3A1.2.13#builtin::version=1.2.13=11e9ea[39m
> The platform linux is incompatible with this module, build skipped.
> 2021-10-07T16:56:10.7333665Z [94m➤[39m [90mYN[39m: [94m➤[39m
> YN0008: │ [38;5;173mcore-js[39m[38;5;111m@[39m[38;5;111mnpm:3.1.4[39m
> must be rebuilt because its dependency tree changed
> 2021-10-07T16:56:10.7357165Z [94m➤[39m [90mYN[39m: [94
Website build problem???
: │ [38;5;173mjpegtran-bin[39m[38;5;111m@[39m[38;5;111mnpm:4.0.0[39m [31mSTDERR[39m ✔ jpegtran pre-build test passed successfully 2021-10-07T16:56:11.9618470Z [94m➤[39m [90mYN[39m: [94m➤[39m [90mYN[39m: │ [38;5;173moptipng-bin[39m[38;5;111m@[39m[38;5;111mnpm:6.0.0[39m [31mSTDERR[39m ✔ optipng pre-build test passed successfully 2021-10-07T16:56:12.0703207Z [94m➤[39m [90mYN[39m: [94m➤[39m [90mYN[39m: │ [38;5;173mgifsicle[39m[38;5;111m@[39m[38;5;111mnpm:4.0.1[39m [31mSTDERR[39m ✔ gifsicle pre-build test passed successfully 2021-10-07T16:56:12.0808189Z [94m➤[39m [90mYN[39m: ::endgroup:: 2021-10-07T16:56:12.0812310Z [94m➤[39m [90mYN[39m: [94m➤[39m [90mYN[39m: └ Completed in 48s 2ms 2021-10-07T16:56:12.2091388Z [94m➤[39m [90mYN[39m: [93m➤[39m YN: Done with warnings in 1m 47s 2021-10-07T16:56:12.2098889Z [94m➤[39m YN: Done in 3m 44s 2021-10-07T16:56:12.7550263Z ##[error]Process completed with exit code 1. I get a different set of HTTP errors running locally: ➤ YN: ➤ YN0001: │ HTTPError: @netlify/plugin-edge-handlers@npm:1.10.0: Response code 404 (Not Found) ➤ YN: at se. (/Users/david/projects/camel/camel-website/.yarn/releases/yarn-2.4.0.cjs:23:10082) ➤ YN: at runMicrotasks () ➤ YN: at processTicksAndRejections (internal/process/task_queues.js:97:5) ➤ YN: ➤ YN0001: │ HTTPError: @netlify/run-utils@npm:1.0.5: Response code 404 (Not Found) ➤ YN: at se. (/Users/david/projects/camel/camel-website/.yarn/releases/yarn-2.4.0.cjs:23:10082) ➤ YN: at runMicrotasks () ➤ YN: at processTicksAndRejections (internal/process/task_queues.js:97:5) ➤ YN: ➤ YN0001: │ HTTPError: @netlify/traffic-mesh-agent-linux-x64@npm:0.27.0: Response code 404 (Not Found) ➤ YN: at se. (/Users/david/projects/camel/camel-website/.yarn/releases/yarn-2.4.0.cjs:23:10082) ➤ YN: at runMicrotasks () ➤ YN: at processTicksAndRejections (internal/process/task_queues.js:97:5) The yarn cache hasn’t changed since Sept 8, so I’m pretty confused about what is going on. Any ideas? David Jencks
Re: [Website] Updating after a release
I’ve written a draft of instructions on how to update the docs/website after a camel/camel-karaf/camel-spring-boot release: https://github.com/apache/camel/pull/6215 I’ve also applied these instructions (retroactively in some cases) to all the active camel-karaf and camel-spring-boot branches: PRs listed in the doc PR above. Although in some cases this results in no link changes, I’d prefer to apply all the PRs so that the link generation code is as consistent as possible. I have a couple of questions as a TODO in the doc PR: NOTE: TODO: Is there a step of setting up CI or the regen bot? Should maven snapshots be deployed? I had to build (with maven) quite a few projects completely because there were no snapshots deployed. David Jencks > On Oct 4, 2021, at 10:47 PM, David Jencks wrote: > > Thanks Gregor, it is really useful to me to know that these are actually the > steps you are following, and that the branch instructions really are > missing…. I’m very good at misinterpreting docs! > > David Jencks > >> On Oct 4, 2021, at 10:39 PM, Gregor Zurowski >> wrote: >> >> I just wanted to confirm that the release guide >> (https://github.com/apache/camel/blob/main/docs/user-manual/modules/ROOT/pages/release-guide.adoc) >> is not outdated and describes the steps we are following when creating >> new releases. The step to branch off into a new camel-MAJOR.MINOR.x >> branch is missing though, I'll add it in. >> >> I'd be definitely interested in the missing steps that are needed >> preparing the branch for dependent projects. >> >> Thanks, >> Gregor >> >> On Mon, Oct 4, 2021 at 11:47 PM David Jencks >> wrote: >>> >>> 1. I got the camel-quarkus build to work with these PRs: >>> https://github.com/apache/camel/pull/6210 >>> https://github.com/apache/camel-spring-boot/pull/376 >>> both include changes that eventually need to go on other branches, so I >>> don’t want to squash the commits. >>> They also include very substantial changes from simply running the maven >>> build with the new version. >>> >>> (a) Evidently instructions to generate these changes is missing from the >>> existing release instructions, or they weren’t followed or I didn’t wait >>> long enough. I think it should be a step in the release process so that >>> it’s safe to use the branch as a dependency. >>> (b) I don’t see any harm in applying these manually generated changes, but >>> if there is an automated process that will eventually create them and that >>> is preferable please advise when it will occur. >>> >>> In any case the camel-quarkus build is currently broken due to the lack of >>> at least the non-generated changes in these PRs. >>> >>> >>> 2. Writing instructions about updating the website will take quite a while, >>> so we can discuss where they should go. For my convenience I’m going to >>> start with a separate page, and it’s easy enough to move the content if >>> that seems appropriate. >>> >>> David Jencks >>> >>> >>>> On Oct 4, 2021, at 1:59 PM, Andrea Cosentino wrote: >>>> >>>> If you want to work on one for the website it's fine, but you'll fragment >>>> the release docs more than the current situation. Fragmentation is bad now. >>>> We should try to reduce pages and focus on content. >>>> >>>> Il lun 4 ott 2021, 22:55 Andrea Cosentino ha scritto: >>>> >>>>> The release guide there it's not outdated. It's related only to camel >>>>> camel-karaf and camel-sb. All the other subprojects have their own release >>>>> guide. -1 to start from a new page. I would start from what we have, which >>>>> is exactly the release steps to release plain camel. >>>>> >>>>> Il lun 4 ott 2021, 22:47 David Jencks ha >>>>> scritto: >>>>> >>>>>> On a quick glance, that looks outdated (vintage camel 2), unaware of most >>>>>> sub-projects, and incomplete (e.g. creating the camel-3.12.x branch >>>>>> before >>>>>> releasing camel-3.12.0 from it). I don’t think I should try to complete >>>>>> and >>>>>> update those instructions. >>>>>> >>>>>> I think a new page on how to deal with the website would be appropriate, >>>>>> since that’s generally going to happen after the release explained on the >>>>>
Re: [Website] Move camel-karaf and camel-spring-boot content into "components" component?
Let’s try again… What I’m proposing here does not move any sources to different repos or change how e.g. the spring-boot tables of components etc are generated. I think it’s completely appropriate for the code generation etc for camel-spring-boot and camel-karat to be in separate repositories, and how tables are generated is an independent issue. I think that normally in an Antora component you want to present a cohesive set of information. I think that means that the vast majority of the links in the component will be to other pages in the same component. Otherwise, when you click a link, suddenly you’ll be in a completely different component environment, which I regard as extremely surprising. Right now, almost everything in camel-spring-boot and camel-karaf is a link to a different component, the “components” component. In particular, I find this disorienting. I click a link and think, “what happened, where did camel-spring-boot go?? I’m looking at the main activemq page!! How do I get back to spring-boot info??” I’m proposing that we consider presenting the camel-spring-boot and camel-karaf info as part of the “components” component, having sections for each in the components nav, under the existing stuff. I also think it’s a good idea to have items in the nav for components, dataformats, etc, so the presentation is parallel to the main components nav. These could either be simply links to the relevant tables, or expand to lists like the existing nav does (this would add another layer of nesting to the nav). Hopefully this explanation will make my text mockup of the proposed nav more comprehensible. Some subsidiary points: IIUC spring-boot and karaf are “extras” that come with most but not all components/dataformats etc. I haven’t thought about how this might be done, but I think it would be nice to have something like badges at the top of each component page, “spring-boot yes”, “karaf yes”. Possibly this would be appropriate for other subprojects: I’m pretty sure it would be for camel-quarkus. I’m not all that familiar with the other sub-projects, but I think they don’t have a 1-1 relationship with main camel components. For instance my understanding is that camel-quarkus bundles one or more components into a quarkus “thing”, and the camel-quarkus page explains this bundling. If there are any other subprojects that do have a necessary 1-1 relation to main camel components, they could have their documentation folded into the main components docs in the same way. David Jencks > On Oct 5, 2021, at 12:20 AM, Claus Ibsen wrote: > > Hi David > > Can you explain in more detail what you mean by moving "information" > to "components". > > If you refer to that there is a .adoc file that is generating a big > table such as > https://raw.githubusercontent.com/apache/camel-spring-boot/main/docs/modules/ROOT/pages/list.adoc > > Then that is generated via a maven plugin, that scans the source code > and builds up that table. > > The list is then used on the website to list all the supported Spring > Boot starters. > https://camel.apache.org/camel-spring-boot/latest/list.html > > That list used the xref links like we do in other places for Camel > components, languages, data formats, etc. > What is it that goes against the meaning of an Antora component? > > IMHO the system we have with the maven plugin that generates that > single file works well. > And the docs are in the src/main/docs folder for each of those spring > boot starters. > So we can maintain it where it belongs. > > > > On Tue, Oct 5, 2021 at 8:26 AM David Jencks wrote: >> >> IIUC the camel-karaf and camel-spring-boot subprojects are always released >> in sync with main camel. >> >> On the websites, for these subprojects, there is a little bit of general >> information, and some tables listing the parts of the “components” that are >> supported in the subproject (components, dataformats, etc). The table rows >> link to the docs in the “components” component, which is against the meaning >> of an Antora component. >> >> I wonder if it would make sense to include this subproject information in >> the “components” component. I’m thinking the nav might look like >> >> Components >> Dataformats >> Languages >> Other >> EIPS >> >> Camel-Spring-Boot >> - components >> - dataformats >> - languages >> - other >> >> Camel-Karaf >> - components >> - dataformats >> - languages >> - other >> >> I think this would be pretty easy to do, and it would definitely leave >> everything in the repo it is currently in. Does this idea seem interesting >> enough that I should prepare a PR/prototype? >> >> David Jencks > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2
Re: [Website] Move camel-karaf and camel-spring-boot content into "components" component?
This is another issue, but I’m thinking about it :-) David Jencks > On Oct 5, 2021, at 12:26 AM, Claus Ibsen wrote: > > Hi > > Btw on another note, then we have for camel-spring-boot and > camel-karaf a "catalog" which holds JSon files with information for > every JAR they support. > > https://github.com/apache/camel-spring-boot/tree/main/catalog/camel-catalog-provider-springboot > > The same for karaf at > https://github.com/apache/camel-karaf/tree/main/catalog/camel-catalog-provider-karaf > > If this is the case, then I think what you did for the core camel > project was to use this to generate the list of components there. > If this can be done the same way, then we can do it similarly, and we > can then remove that maven plugin that generates that list.adoc file. > > On Tue, Oct 5, 2021 at 9:20 AM Claus Ibsen wrote: >> >> Hi David >> >> Can you explain in more detail what you mean by moving "information" >> to "components". >> >> If you refer to that there is a .adoc file that is generating a big >> table such as >> https://raw.githubusercontent.com/apache/camel-spring-boot/main/docs/modules/ROOT/pages/list.adoc >> >> Then that is generated via a maven plugin, that scans the source code >> and builds up that table. >> >> The list is then used on the website to list all the supported Spring >> Boot starters. >> https://camel.apache.org/camel-spring-boot/latest/list.html >> >> That list used the xref links like we do in other places for Camel >> components, languages, data formats, etc. >> What is it that goes against the meaning of an Antora component? >> >> IMHO the system we have with the maven plugin that generates that >> single file works well. >> And the docs are in the src/main/docs folder for each of those spring >> boot starters. >> So we can maintain it where it belongs. >> >> >> >> On Tue, Oct 5, 2021 at 8:26 AM David Jencks wrote: >>> >>> IIUC the camel-karaf and camel-spring-boot subprojects are always released >>> in sync with main camel. >>> >>> On the websites, for these subprojects, there is a little bit of general >>> information, and some tables listing the parts of the “components” that are >>> supported in the subproject (components, dataformats, etc). The table rows >>> link to the docs in the “components” component, which is against the >>> meaning of an Antora component. >>> >>> I wonder if it would make sense to include this subproject information in >>> the “components” component. I’m thinking the nav might look like >>> >>> Components >>> Dataformats >>> Languages >>> Other >>> EIPS >>> >>> Camel-Spring-Boot >>> - components >>> - dataformats >>> - languages >>> - other >>> >>> Camel-Karaf >>> - components >>> - dataformats >>> - languages >>> - other >>> >>> I think this would be pretty easy to do, and it would definitely leave >>> everything in the repo it is currently in. Does this idea seem interesting >>> enough that I should prepare a PR/prototype? >>> >>> David Jencks >> >> >> >> -- >> Claus Ibsen >> - >> http://davsclaus.com @davsclaus >> Camel in Action 2: https://www.manning.com/ibsen2 > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2
[Website] Move camel-karaf and camel-spring-boot content into "components" component?
IIUC the camel-karaf and camel-spring-boot subprojects are always released in sync with main camel. On the websites, for these subprojects, there is a little bit of general information, and some tables listing the parts of the “components” that are supported in the subproject (components, dataformats, etc). The table rows link to the docs in the “components” component, which is against the meaning of an Antora component. I wonder if it would make sense to include this subproject information in the “components” component. I’m thinking the nav might look like Components Dataformats Languages Other EIPS Camel-Spring-Boot - components - dataformats - languages - other Camel-Karaf - components - dataformats - languages - other I think this would be pretty easy to do, and it would definitely leave everything in the repo it is currently in. Does this idea seem interesting enough that I should prepare a PR/prototype? David Jencks
Re: [Website] Updating after a release
Thanks Gregor, it is really useful to me to know that these are actually the steps you are following, and that the branch instructions really are missing…. I’m very good at misinterpreting docs! David Jencks > On Oct 4, 2021, at 10:39 PM, Gregor Zurowski wrote: > > I just wanted to confirm that the release guide > (https://github.com/apache/camel/blob/main/docs/user-manual/modules/ROOT/pages/release-guide.adoc) > is not outdated and describes the steps we are following when creating > new releases. The step to branch off into a new camel-MAJOR.MINOR.x > branch is missing though, I'll add it in. > > I'd be definitely interested in the missing steps that are needed > preparing the branch for dependent projects. > > Thanks, > Gregor > > On Mon, Oct 4, 2021 at 11:47 PM David Jencks wrote: >> >> 1. I got the camel-quarkus build to work with these PRs: >> https://github.com/apache/camel/pull/6210 >> https://github.com/apache/camel-spring-boot/pull/376 >> both include changes that eventually need to go on other branches, so I >> don’t want to squash the commits. >> They also include very substantial changes from simply running the maven >> build with the new version. >> >> (a) Evidently instructions to generate these changes is missing from the >> existing release instructions, or they weren’t followed or I didn’t wait >> long enough. I think it should be a step in the release process so that it’s >> safe to use the branch as a dependency. >> (b) I don’t see any harm in applying these manually generated changes, but >> if there is an automated process that will eventually create them and that >> is preferable please advise when it will occur. >> >> In any case the camel-quarkus build is currently broken due to the lack of >> at least the non-generated changes in these PRs. >> >> >> 2. Writing instructions about updating the website will take quite a while, >> so we can discuss where they should go. For my convenience I’m going to >> start with a separate page, and it’s easy enough to move the content if that >> seems appropriate. >> >> David Jencks >> >> >>> On Oct 4, 2021, at 1:59 PM, Andrea Cosentino wrote: >>> >>> If you want to work on one for the website it's fine, but you'll fragment >>> the release docs more than the current situation. Fragmentation is bad now. >>> We should try to reduce pages and focus on content. >>> >>> Il lun 4 ott 2021, 22:55 Andrea Cosentino ha scritto: >>> >>>> The release guide there it's not outdated. It's related only to camel >>>> camel-karaf and camel-sb. All the other subprojects have their own release >>>> guide. -1 to start from a new page. I would start from what we have, which >>>> is exactly the release steps to release plain camel. >>>> >>>> Il lun 4 ott 2021, 22:47 David Jencks ha >>>> scritto: >>>> >>>>> On a quick glance, that looks outdated (vintage camel 2), unaware of most >>>>> sub-projects, and incomplete (e.g. creating the camel-3.12.x branch before >>>>> releasing camel-3.12.0 from it). I don’t think I should try to complete >>>>> and >>>>> update those instructions. >>>>> >>>>> I think a new page on how to deal with the website would be appropriate, >>>>> since that’s generally going to happen after the release explained on the >>>>> existing page. >>>>> >>>>> Thoughts? >>>>> >>>>> David Jencks >>>>> >>>>>> On Oct 4, 2021, at 11:00 AM, Claus Ibsen wrote: >>>>>> >>>>>> Hi David >>>>>> >>>>>> Yes there is the release guide docs at >>>>>> >>>>> https://github.com/apache/camel/blob/main/docs/user-manual/modules/ROOT/pages/release-guide.adoc >>>>>> >>>>>> On Mon, Oct 4, 2021 at 6:46 PM David Jencks >>>>> wrote: >>>>>>> >>>>>>> I believe the Antora build has not yet been updated to include the >>>>> just released camel-3.12.x and related releases. However, camel-quarkus is >>>>> trying to use it, which breaks the camel-quarkus build. >>>>>>> >>>>>>> I’d like to do this and write up what to do when there’s a release, >>>>> and hopefully simplify the process a bit. >>>>>>> >>>>>>> Is there already such a document? >>>>>>> >>>>>>> I actually think it might be best to have a section explaining the >>>>> structure and interdependencies of the Antora site in the site, along with >>>>> instructions on what to do when there’s a new release. Alternatively >>>>> perhaps this could be an AsciiDoc file in camel-website. >>>>>>> >>>>>>> Is anyone else working on this? I don’t want to create conflicts or >>>>> duplicate work. >>>>>>> >>>>>>> David Jencks >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Claus Ibsen >>>>>> - >>>>>> http://davsclaus.com @davsclaus >>>>>> Camel in Action 2: https://www.manning.com/ibsen2 >>>>> >>>>> >>
Re: [Website] Updating after a release
1. I got the camel-quarkus build to work with these PRs: https://github.com/apache/camel/pull/6210 https://github.com/apache/camel-spring-boot/pull/376 both include changes that eventually need to go on other branches, so I don’t want to squash the commits. They also include very substantial changes from simply running the maven build with the new version. (a) Evidently instructions to generate these changes is missing from the existing release instructions, or they weren’t followed or I didn’t wait long enough. I think it should be a step in the release process so that it’s safe to use the branch as a dependency. (b) I don’t see any harm in applying these manually generated changes, but if there is an automated process that will eventually create them and that is preferable please advise when it will occur. In any case the camel-quarkus build is currently broken due to the lack of at least the non-generated changes in these PRs. 2. Writing instructions about updating the website will take quite a while, so we can discuss where they should go. For my convenience I’m going to start with a separate page, and it’s easy enough to move the content if that seems appropriate. David Jencks > On Oct 4, 2021, at 1:59 PM, Andrea Cosentino wrote: > > If you want to work on one for the website it's fine, but you'll fragment > the release docs more than the current situation. Fragmentation is bad now. > We should try to reduce pages and focus on content. > > Il lun 4 ott 2021, 22:55 Andrea Cosentino ha scritto: > >> The release guide there it's not outdated. It's related only to camel >> camel-karaf and camel-sb. All the other subprojects have their own release >> guide. -1 to start from a new page. I would start from what we have, which >> is exactly the release steps to release plain camel. >> >> Il lun 4 ott 2021, 22:47 David Jencks ha >> scritto: >> >>> On a quick glance, that looks outdated (vintage camel 2), unaware of most >>> sub-projects, and incomplete (e.g. creating the camel-3.12.x branch before >>> releasing camel-3.12.0 from it). I don’t think I should try to complete and >>> update those instructions. >>> >>> I think a new page on how to deal with the website would be appropriate, >>> since that’s generally going to happen after the release explained on the >>> existing page. >>> >>> Thoughts? >>> >>> David Jencks >>> >>>> On Oct 4, 2021, at 11:00 AM, Claus Ibsen wrote: >>>> >>>> Hi David >>>> >>>> Yes there is the release guide docs at >>>> >>> https://github.com/apache/camel/blob/main/docs/user-manual/modules/ROOT/pages/release-guide.adoc >>>> >>>> On Mon, Oct 4, 2021 at 6:46 PM David Jencks >>> wrote: >>>>> >>>>> I believe the Antora build has not yet been updated to include the >>> just released camel-3.12.x and related releases. However, camel-quarkus is >>> trying to use it, which breaks the camel-quarkus build. >>>>> >>>>> I’d like to do this and write up what to do when there’s a release, >>> and hopefully simplify the process a bit. >>>>> >>>>> Is there already such a document? >>>>> >>>>> I actually think it might be best to have a section explaining the >>> structure and interdependencies of the Antora site in the site, along with >>> instructions on what to do when there’s a new release. Alternatively >>> perhaps this could be an AsciiDoc file in camel-website. >>>>> >>>>> Is anyone else working on this? I don’t want to create conflicts or >>> duplicate work. >>>>> >>>>> David Jencks >>>> >>>> >>>> >>>> -- >>>> Claus Ibsen >>>> - >>>> http://davsclaus.com @davsclaus >>>> Camel in Action 2: https://www.manning.com/ibsen2 >>> >>>
Re: [Website] Updating after a release
On a quick glance, that looks outdated (vintage camel 2), unaware of most sub-projects, and incomplete (e.g. creating the camel-3.12.x branch before releasing camel-3.12.0 from it). I don’t think I should try to complete and update those instructions. I think a new page on how to deal with the website would be appropriate, since that’s generally going to happen after the release explained on the existing page. Thoughts? David Jencks > On Oct 4, 2021, at 11:00 AM, Claus Ibsen wrote: > > Hi David > > Yes there is the release guide docs at > https://github.com/apache/camel/blob/main/docs/user-manual/modules/ROOT/pages/release-guide.adoc > > On Mon, Oct 4, 2021 at 6:46 PM David Jencks wrote: >> >> I believe the Antora build has not yet been updated to include the just >> released camel-3.12.x and related releases. However, camel-quarkus is trying >> to use it, which breaks the camel-quarkus build. >> >> I’d like to do this and write up what to do when there’s a release, and >> hopefully simplify the process a bit. >> >> Is there already such a document? >> >> I actually think it might be best to have a section explaining the structure >> and interdependencies of the Antora site in the site, along with >> instructions on what to do when there’s a new release. Alternatively >> perhaps this could be an AsciiDoc file in camel-website. >> >> Is anyone else working on this? I don’t want to create conflicts or >> duplicate work. >> >> David Jencks > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2
[Website] Updating after a release
I believe the Antora build has not yet been updated to include the just released camel-3.12.x and related releases. However, camel-quarkus is trying to use it, which breaks the camel-quarkus build. I’d like to do this and write up what to do when there’s a release, and hopefully simplify the process a bit. Is there already such a document? I actually think it might be best to have a section explaining the structure and interdependencies of the Antora site in the site, along with instructions on what to do when there’s a new release. Alternatively perhaps this could be an AsciiDoc file in camel-website. Is anyone else working on this? I don’t want to create conflicts or duplicate work. David Jencks
Re: [Website] camel-spring-boot docs
I went ahead and implemented the first two of these in main “latest” branches. Other than removing 324 unreferenced and inaccessible pages, I don’t see any changes (There’s one slight change to the single reference to one of the removed pages: (diff is new to old) < Also add any component starters your Spring Boot application requires. For example this adds the auto-configuration starter for the ActiveMQ component. --- > And any component starters your > Spring Boot application requires. For example this adds the href="activemq-starter.html" class="page">starter for the href="../../components/3. (instead of linking to an otherwise inaccessible page, it links to the section where that content is included in the activemq component page.) PRs: https://github.com/apache/camel-website/pull/641 (incomplete, pending merging other two or 8, see below) https://github.com/apache/camel-spring-boot/pull/374 <https://github.com/apache/camel-spring-boot/pull/374> (moves generated content to "partials" in a separate directory tree) https://github.com/apache/camel/pull/6200 (changes includes of this content) Since this removes a lot of pages, I’d like to apply this idea to all the active camel-spring-boot branches (latest, 3.12.x, 3.11.x, 3.7.x). I was hoping to replace copying the AsciiDoc files with symlinks, but I’d forgotten how cantankerous Ant is and couldn’t find a way to use it’s symlink task, and I’m hesitant to bring node/gulp in to the project just to make some symlinks. David Jencks > On Oct 2, 2021, at 10:17 AM, David Jencks wrote: > > I always forget that for Antora terms often have a specific meaning that > isn’t that common or obvious :-) > >> On Oct 2, 2021, at 12:45 AM, Claus Ibsen wrote: >> >> On Fri, Oct 1, 2021 at 11:30 PM David Jencks >> wrote: >>> >>> I looked at camel-spring-boot a little bit and have several questions and >>> suggestions… if there’s agreement I’ll open some issues and work on them. >>> >>> 1. The AsciiDoc pages are only included in the main component/dataformat/… >>> pages, and not accessible through navigation standalone. I think they >>> should be partials, not standalone pages. >>> >> >> You can possible not do this as the ascii doc is generated with >> information from spring boot itself, there are some camel spring boot >> started components that have spring boot auto configuration, >> that information are not in the camel-catalog, but are stored in >> spring boot style (inside the JAR in META-INF there is a spring boot >> json file). >> >> > > This just involves moving the target location of the generated pages to > modules/ROOT/partials from modules/ROOT/pages, and in main camel components > changing the include::….page$... to include::….partial$… > This would not be a good idea if there was a firm plan to, at some point, > also have this information on standalone pages, or convert the current > include:: to a link to a standalone page. Otherwise, it’s simple and > shouldn’t disrupt anything. The generated spring boot docs would still be in > the camel-spring-boot repo. > >> >>> 2. The individual generated pages are tied 1-1 with the ‘components’ >>> component. I think, even though they are (at least currently) in a >>> different repo having them in the ‘components’ component as part of a >>> distributed component makes more sense than having them in a different >>> component. >>> >> >> Not sure what you mean? > > We’d need 2 directories in camel-spring-boot docs, say > > components/modules/spring-boot/partials where all the individual generated > pages go, with a components/antora.yml specifying name: components and > version: > > and > > spring-boot/modules/ROOT/pages where the 3 or 4 other non-generated pages go > with the current antora.yml. > > I think this would make it more clear that the individual generated docs > actually are shown as part of the “components” component. It won’t affect > Antora in any noticeable way. > >> >> >>> 3. IIUC the individual pages are completely generated from data in a json >>> file. I think this can be completely replaced with a partial and the >>> “jsonpath” stuff like we recently did for the ‘components' generated >>> content. >>> (There seem to be a few that aren’t generated under core, but I haven’t >>> found if or where they show up in the website. Perhaps some or all could be >>> removed?) >>> >> >> As first response, its from spring boot json data file. > > Thanks! > >> &g
Re: [Website] camel-spring-boot docs
I always forget that for Antora terms often have a specific meaning that isn’t that common or obvious :-) > On Oct 2, 2021, at 12:45 AM, Claus Ibsen wrote: > > On Fri, Oct 1, 2021 at 11:30 PM David Jencks wrote: >> >> I looked at camel-spring-boot a little bit and have several questions and >> suggestions… if there’s agreement I’ll open some issues and work on them. >> >> 1. The AsciiDoc pages are only included in the main component/dataformat/… >> pages, and not accessible through navigation standalone. I think they >> should be partials, not standalone pages. >> > > You can possible not do this as the ascii doc is generated with > information from spring boot itself, there are some camel spring boot > started components that have spring boot auto configuration, > that information are not in the camel-catalog, but are stored in > spring boot style (inside the JAR in META-INF there is a spring boot > json file). > > This just involves moving the target location of the generated pages to modules/ROOT/partials from modules/ROOT/pages, and in main camel components changing the include::….page$... to include::….partial$… This would not be a good idea if there was a firm plan to, at some point, also have this information on standalone pages, or convert the current include:: to a link to a standalone page. Otherwise, it’s simple and shouldn’t disrupt anything. The generated spring boot docs would still be in the camel-spring-boot repo. > >> 2. The individual generated pages are tied 1-1 with the ‘components’ >> component. I think, even though they are (at least currently) in a >> different repo having them in the ‘components’ component as part of a >> distributed component makes more sense than having them in a different >> component. >> > > Not sure what you mean? We’d need 2 directories in camel-spring-boot docs, say components/modules/spring-boot/partials where all the individual generated pages go, with a components/antora.yml specifying name: components and version: and spring-boot/modules/ROOT/pages where the 3 or 4 other non-generated pages go with the current antora.yml. I think this would make it more clear that the individual generated docs actually are shown as part of the “components” component. It won’t affect Antora in any noticeable way. > > >> 3. IIUC the individual pages are completely generated from data in a json >> file. I think this can be completely replaced with a partial and the >> “jsonpath” stuff like we recently did for the ‘components' generated content. >> (There seem to be a few that aren’t generated under core, but I haven’t >> found if or where they show up in the website. Perhaps some or all could be >> removed?) >> > > As first response, its from spring boot json data file. Thanks! > > >> 4. Are there any components/dataformats/… that don’t participate in spring >> boot? If not, what is the purpose of the table listing all the spring boots, >> which points to the components pages? >> https://camel.apache.org/camel-spring-boot/latest/list.html >> If this page serves a useful purpose perhaps the table can be generated >> using indexTable as in the ‘components’ component. >> > > The point is to list all the supported spring boot starters. When you > use Camel with Spring Boot then use only these JARs. > That is the "stuff" that works on Spring Boot. > > We have similar for Karaf (whats in the features.xml file), and for > Quarkus with the camel quarkus extensions. That makes sense! > > >> 5. These pages: >> https://camel.apache.org/camel-spring-boot/latest/index.html >> https://camel.apache.org/camel-spring-boot/latest/spring-boot.html >> seem to have a lot of overlapping content. I’m completely bewildered by the >> apparent duplication and don’t understand what the different choices on each >> page do or how they differ. I think it would be great if someone would make >> these docs clearer. >> > > Yes there is a JIRA ticket to overhaul and cleanup the docs. I am > slowing working my way through that. A never ending task :-) at least it seems that way to me sometimes… Many thanks! > > >> Maybe that’s enough for now… >> >> David Jencks > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2
[Website] camel-spring-boot docs
I looked at camel-spring-boot a little bit and have several questions and suggestions… if there’s agreement I’ll open some issues and work on them. 1. The AsciiDoc pages are only included in the main component/dataformat/… pages, and not accessible through navigation standalone. I think they should be partials, not standalone pages. 2. The individual generated pages are tied 1-1 with the ‘components’ component. I think, even though they are (at least currently) in a different repo having them in the ‘components’ component as part of a distributed component makes more sense than having them in a different component. 3. IIUC the individual pages are completely generated from data in a json file. I think this can be completely replaced with a partial and the “jsonpath” stuff like we recently did for the ‘components' generated content. (There seem to be a few that aren’t generated under core, but I haven’t found if or where they show up in the website. Perhaps some or all could be removed?) 4. Are there any components/dataformats/… that don’t participate in spring boot? If not, what is the purpose of the table listing all the spring boots, which points to the components pages? https://camel.apache.org/camel-spring-boot/latest/list.html If this page serves a useful purpose perhaps the table can be generated using indexTable as in the ‘components’ component. 5. These pages: https://camel.apache.org/camel-spring-boot/latest/index.html https://camel.apache.org/camel-spring-boot/latest/spring-boot.html seem to have a lot of overlapping content. I’m completely bewildered by the apparent duplication and don’t understand what the different choices on each page do or how they differ. I think it would be great if someone would make these docs clearer. Maybe that’s enough for now… David Jencks
Re: Apache Camel 3.12 Release
https://github.com/apache/camel/pull/6170
> On Sep 27, 2021, at 9:17 AM, David Jencks wrote:
>
> The work around "Polished EIPs and make their categorization simpler for
> tooling." appears to have broken the website build, by removing ‘eip’ from
> the labels in the json files.
> I’ve discovered where this is generated from and expect to have a PR in an
> hour or so.
>
> I think it would be nice to have the 3.12 branch have a working website
> build; otherwise this will have to be fixed in two places.
>
> The errors from the website build are (locally):
>
> [08:30:58.221] WARN (asciidoctor):
> file:
> core/camel-core-engine/src/main/docs/modules/eips/pages/batch-config-eip.adoc
> source: https://github.com/apache/camel.git
> <https://github.com/apache/camel.git> (refname: main, start path:
> core/camel-core-engine/src/main/docs)
> msg: {
> "msg": "Target example$json/batch-config.json not found",
> "useId": 0
> }
> [08:30:58.855] WARN (asciidoctor):
> file:
> core/camel-core-engine/src/main/docs/modules/eips/pages/faultToleranceConfiguration-eip.adoc
> source: https://github.com/apache/camel.git
> <https://github.com/apache/camel.git> (refname: main, start path:
> core/camel-core-engine/src/main/docs)
> msg: {
> "msg": "Target example$json/faultToleranceConfiguration.json not found",
> "useId": 0
> }
> [08:30:58.949] WARN (asciidoctor):
> file:
> core/camel-core-engine/src/main/docs/modules/eips/pages/hystrixConfiguration-eip.adoc
> source: https://github.com/apache/camel.git
> <https://github.com/apache/camel.git> (refname: main, start path:
> core/camel-core-engine/src/main/docs)
> msg: {
> "msg": "Target example$json/hystrixConfiguration.json not found",
> "useId": 0
> }
> [08:30:59.988] WARN (asciidoctor):
> file:
> core/camel-core-engine/src/main/docs/modules/eips/pages/resilience4jConfiguration-eip.adoc
> source: https://github.com/apache/camel.git
> <https://github.com/apache/camel.git> (refname: main, start path:
> core/camel-core-engine/src/main/docs)
> msg: {
> "msg": "Target example$json/resilience4jConfiguration.json not found",
> "useId": 0
> }
> [08:31:00.504] WARN (asciidoctor):
> file:
> core/camel-core-engine/src/main/docs/modules/eips/pages/stream-config-eip.adoc
> source: https://github.com/apache/camel.git
> <https://github.com/apache/camel.git> (refname: main, start path:
> core/camel-core-engine/src/main/docs)
> msg: {
> "msg": "Target example$json/stream-config.json not found",
> "useId": 0
> }
>
> David Jencks
>
>> On Sep 27, 2021, at 7:42 AM, Claus Ibsen > <mailto:claus.ib...@gmail.com>> wrote:
>>
>> On Mon, Sep 27, 2021 at 4:18 PM Karen Lease > <mailto:karenlease...@gmail.com>> wrote:
>>>
>>> Hi all,
>>>
>>> For camel-spring-boot, the issue CAMEL-16945 (upgrade to Junit5) was
>>> already merged except for the Arquillian integration tests and the
>>> related documentation update. I see it doesn't currently have a FixVersion.
>>>
>>
>> For documentation then its an ongoing effort for the rest of the year
>> to make it all up to date.
>> So its fine to do this work for 3.13.
>>
>>
>>> Karen Lease
>>
>>
>>
>> --
>> Claus Ibsen
>> -
>> http://davsclaus.com <http://davsclaus.com/> @davsclaus
>> Camel in Action 2: https://www.manning.com/ibsen2
>> <https://www.manning.com/ibsen2>
>
Re: Apache Camel 3.12 Release
The work around "Polished EIPs and make their categorization simpler for
tooling." appears to have broken the website build, by removing ‘eip’ from the
labels in the json files.
I’ve discovered where this is generated from and expect to have a PR in an hour
or so.
I think it would be nice to have the 3.12 branch have a working website build;
otherwise this will have to be fixed in two places.
The errors from the website build are (locally):
[08:30:58.221] WARN (asciidoctor):
file:
core/camel-core-engine/src/main/docs/modules/eips/pages/batch-config-eip.adoc
source: https://github.com/apache/camel.git (refname: main, start path:
core/camel-core-engine/src/main/docs)
msg: {
"msg": "Target example$json/batch-config.json not found",
"useId": 0
}
[08:30:58.855] WARN (asciidoctor):
file:
core/camel-core-engine/src/main/docs/modules/eips/pages/faultToleranceConfiguration-eip.adoc
source: https://github.com/apache/camel.git (refname: main, start path:
core/camel-core-engine/src/main/docs)
msg: {
"msg": "Target example$json/faultToleranceConfiguration.json not found",
"useId": 0
}
[08:30:58.949] WARN (asciidoctor):
file:
core/camel-core-engine/src/main/docs/modules/eips/pages/hystrixConfiguration-eip.adoc
source: https://github.com/apache/camel.git (refname: main, start path:
core/camel-core-engine/src/main/docs)
msg: {
"msg": "Target example$json/hystrixConfiguration.json not found",
"useId": 0
}
[08:30:59.988] WARN (asciidoctor):
file:
core/camel-core-engine/src/main/docs/modules/eips/pages/resilience4jConfiguration-eip.adoc
source: https://github.com/apache/camel.git (refname: main, start path:
core/camel-core-engine/src/main/docs)
msg: {
"msg": "Target example$json/resilience4jConfiguration.json not found",
"useId": 0
}
[08:31:00.504] WARN (asciidoctor):
file:
core/camel-core-engine/src/main/docs/modules/eips/pages/stream-config-eip.adoc
source: https://github.com/apache/camel.git (refname: main, start path:
core/camel-core-engine/src/main/docs)
msg: {
"msg": "Target example$json/stream-config.json not found",
"useId": 0
}
David Jencks
> On Sep 27, 2021, at 7:42 AM, Claus Ibsen wrote:
>
> On Mon, Sep 27, 2021 at 4:18 PM Karen Lease wrote:
>>
>> Hi all,
>>
>> For camel-spring-boot, the issue CAMEL-16945 (upgrade to Junit5) was
>> already merged except for the Arquillian integration tests and the
>> related documentation update. I see it doesn't currently have a FixVersion.
>>
>
> For documentation then its an ongoing effort for the rest of the year
> to make it all up to date.
> So its fine to do this work for 3.13.
>
>
>> Karen Lease
>
>
>
> --
> Claus Ibsen
> -
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
