Hi.
Le jeu. 14 nov. 2019 à 13:12, Alex Herbert <alex.d.herb...@gmail.com> a écrit :
>
>
> On 14/11/2019 01:39, Gilles Sadowski wrote:
> > 2019-11-14 1:35 UTC+01:00, Alex Herbert <alex.d.herb...@gmail.com>:
> >>
> >>> On 13 Nov 2019, at 23:53, Gilles Sadowski <gillese...@gmail.com> wrote:
> >>>
> >>> Hello.
> >>>
> >>> Le mer. 13 nov. 2019 à 18:29, Alex Herbert <alex.d.herb...@gmail.com
> >>> <mailto:alex.d.herb...@gmail.com>> a écrit :
> >>>>
> >>>> On 13/11/2019 13:09, Gilles Sadowski wrote:
> >>>>> Hi.
> >>>>>
> >>>>> Do you think that it would be feasible to have all those fixes applied
> >>>>> to other modular components (that were based on the [RNG] layout)
> >>>>> through a common layer (another POM) between those components
> >>>>> and "commons-parent"?
> >>>> Most of the fixes have been in the module site descriptors or items that
> >>>> should be moved up to commons parent. It may be possible to put the site
> >>>> descriptors into a parent. IIUC the site descriptors are merged together
> >>>> before the maven site plugin creates the site. So the fix for the top
> >>>> right banner could be moved into a common parent. Each child module
> >>>> would also want to enumerate the past releases of the javadocs. Thus
> >>>> they would require a site descriptor anyway and the banner fix would
> >>>> only save 5 lines per site.xml for the banner.
> >>> Well, I was thinking of whether a multi-layered POM could be more
> >>> flexible and robust in handling the different type of components (e.g.
> >>> "multi-module" or not).
> >> I think this would take a bit more reading on exactly how Maven thinks a
> >> multi-module project should work...
> > Probably. :-/
> >
> >>>> I did a big change to the use of svn to checkout the current site. This
> >>>> was required as the archived javadocs are not in a top level directory.
> >>>> So each module has to be checked out, the archived javadocs set to be
> >>>> excluded and then the rest of the site can be checked out.
> >>> Sorry, I don't follow.
> >>> Didn't links to the Javadoc of previous versions exist prior to those
> >>> changes?
> >> Yes. Take a look at the pom.xml on line 464 for prepare-checkout.
> >>
> >> This entire section was and is a mystery to me. I don’t know why it is
> >> required to create a copy of the site locally.
> > Oh, I seem to remember now that I was hit by this nonsense of
> > svn checking out the web site when the "site-content" did not
> > exist!
> >
> >> The previous code in this maven target was created on the assumption that
> >> it
> >> was a single module project and the archived javadocs for every previous
> >> version were in a javadocs directory under the top level directory. The
> >> same
> >> code is present in lang, io, text, etc.
> >>
> >> The way it works for those projects is incorrect for a multi-module site as
> >> the archived javadocs are in a different place. It also is a target run in
> >> every module and so for a full site build with the examples modules you
> >> ended up with 14 full checkout copies of the RNG site, including the old
> >> archived javadocs.
> > Not sure I get what you mean; but I don't think that anything related
> > to svn should be necessary in the POM file, excerpt perhaps to automate
> > the creation of an *empty* "site-content" directory (in every module)
> > in order to prevent downloading the web site.
> >
> >> Anyway I fixed it to work as it should. It checks out the site and ignores
> >> the old javadocs. I added a fix so child modules copy the parent site
> >> checkout. This only works if invoked in two stages on a clean git checkout:
> >>
> >> mvn -N pre-site && mvn pre-site
> >>
> >> This in itself is annoying as you cannot do:
> >>
> >> git clone …
> >> cd commons-rng && mvn site
> >>
> >> Without the checkout of the site in each module. I think I can fix this by
> >> using an external ant build.xml file where you can do if-else logic. But I
> >> was wondering if I even had to. Perhaps the goal only needs to run in the
> >> parent, or the dist-archive module for a release.
> > Not even there (IIUC whet you mean).
> > What I do is something along the lines:
> > $ mvn site site:stage
> > $ cd site-content
> > $ rm -rf *
> > $ cd target/staging
> > $ cp -r . ../../site-content
> > Then
> > $ cd ../../site-content
> > $ svn add ... the new files (shown with "?" by "svn status")
> > $ svn del ... the old files (shown with "!" by "svn status")
> > $ svn commit
> >
> >> What I would like to know is:
> >>
> >> 1. Do the child modules need this?
> > I don't think so.
> >
> >> 2. What is it used for? If it is only for a release then it should be in
> >> the
> >> release profile. Or maybe the commons-release plugin. Using ant to do this
> >> external execution is just poor (I spent a while fighting ant and it is not
> >> a programming language, so not suitable for the decisions required for the
> >> multi-module recursive processing).
> > I'd favour dropping anything which is not working properly. ;-)
> >
> >> 3. The top level checkout is used in the release process for manually
> >> updating the site. But why all the others?
> > Indeed.
> > I just created (locally) empty "site-content" and never put anything in
> > them. [Would be nice to automate; IIRC, Eric too had some mishaps
> > with "site-content"...]
> >
> >> If I go to for example commons-rng-client-api and empty the site-content
> >> folder ‘mvn clean package site’ still creates a site that looks fine.
> > Sure.
> > As you state above, why all the svn trickery appears in the POM is
> > a mistery...
>
> I have updated the rng parent pom to do a selective checkout of the site.
>
> Any child modules will just get an empty directory. The parent still
> gets the full checkout.
>
> I do not know if the full checkout is required.
IMO, it is not.
> It should at least be a
> svn versioned directory so that you can use it to copy a locally staged
> site back to the live site (as you describe above).
Sure, but I think that at setup (usually just after time the maven project
is "git clone"d), "site-content" in the top-level directory ("trunk") should
be initialized as "subversion" directory (using the info from the POM, I
guess) but without downloading the contents of the web site since a
developer will hardly ever need it (at least not until wanting to update
the live site).
> Since I do not fully understand what exactly is required of this
> directory I will leave it with the full checkout for now. This can
> always be changed by updating the 'prepare-checkout' execution in the
> 'setup-checkout' profile.
Does this behave as I propose above?
Also: I don't understand why the POM contains this sentence
"This includes the legacy javadocs for commons-rng-examples release 1.0."
> >
> >> Another bug with multi-module builds is that the following reports are in
> >> each child module and are duplicates of that in the top level page:
> >>
> >> Project information
> >> - Team
> >> - SCM
> >> - Issue Management
> >> - Mailing lists
> >> - CI management
> >> - Distribution management
> > That's probably because, lacking sufficient knowledge, I copied everything
> > to each module (using the non-modular Commons Math as a template).
> > OTOH, I don't think that having the above in each "sub-site" is bad.
> I've left these.
> >
> >> Project reports
> >> - jira
> >>
> >> The ones in the project information menu are harmless and fast to create.
> >> The jira report takes a long time to generate. I think at least this one
> >> should be disabled in child modules.
> > +1
> > [The more so that it refers to all the issues, not only those that pertain
> > to the module at hand.]
>
> I've updated the jira report to filter on the component id. This must be
> keyworded in the Jira ticket. I will go through Jira and label those
> which are missing component labels.
Do you mean "component" or "module"?
If the former, I don't understand.
If the latter, I had rather suggested that the JIRA report is not split into
a separate list for each module; rationale being more or less that issues
management are is a project level task (directly impacting e.g. a release).
Similarly, the "RAT" and "Changes" reports should probably only be visible
at the top-level:
http://commons.apache.org/proper/commons-rng/project-reports.html
while the "Checkstyle" one should only be visible in a module's "sub-site".
> Currently I have set examples to run a jira report using the 'examples'
> tag. This could be further sub-divisioned for each sub component of
> examples. Currently this is not done in Jira.
>
> >
> >> My motivation is to reduce bloat and and speed up the process of a site
> >> build. It was annoying when doing the release as you use clean checkouts
> >> and
> >> the download of 14 copies of the full site was slow.
> > Waiting for the upload is already painful enough: All files are transmitted
> > at
> > each web site update because of some tag, or date, has changed. :-(
> >
> >>>> Since this process is repeated for every module it can get very slow. I
> >>>> changed the site checkout to copy the directory from the parent if it
> >>>> was present. There was no solution I could find to fix this to run in a
> >>>> single maven command as profile activation for all modules is done by
> >>>> the reactor before any work is performed. Thus if the parent does the
> >>>> checkout there is no way for all the child modules to know the parent
> >>>> checkout exists in their profiles: when the profiles are initialised the
> >>>> parent checkout may not exist.
> >>>>
> >>>> There may be a better way to do this but it is all done in an antrun
> >>>> plugin. Ant has limited if-else capability in the antrun plugin as it
> >>>> only allows 1 <target> tag per execution. You require multiple <target>
> >>>> tags in the same ant build to have conditional dependencies between
> >>>> them, i.e. check for parent folder checkout and copy, otherwise do a svn
> >>>> checkout.
> >>>>
> >>>> I am going to try moving all this to a build.xml file and call that. It
> >>>> should allow more powerful use of ant. It also separates the ant config
> >>>> from the maven config.
> >>>>
> >>>> If this works the build.xml for ant to checkout the site recursively
> >>>> could be put into a parent.
> >>>>
> >>>> I am still unclear on why this site checkout is required for all the
> >>>> child modules. It is used in the maven-scm-publish-plugin but that
> >>>> should be used on the top level module during a release process. So a
> >>>> simpler fix is to not checkout the site in all the children.
> >>>>
> >>>> A simple test would be to set the poms to not copy the directory for all
> >>>> the child modules and do a dummy release.
> >>>>
> >>>> It's a work in progress...
> >>> I'm lost; what's the purpose?
> >> Removing all these copies of the live site. I think I need to look at the
> >> code for the commons-release plugin to understand what this folder it used
> >> for. It seems to me that it is not used for general development.
> > Sure, as mentioned above, an existing but empty "site-content" and
> > everything works fine.
>
> The site build process should now be a bit faster and cleaner.
>
Thanks,
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org
