I realized as I re-read this that one thing below may not be clear. When you use Maven to build your site, mvn site and mvn site:deploy are going to deploy the site to some location. In the new process this will be somewhere on your local file system. Once you have the site built you then have to get it into subversion - either in the main site via svnpubsub or by committing to the production area in subversion as I described below. Alternately, the CMS can run mvn site (it doesn't support building a multi-module site though).
Ralph On Jun 19, 2012, at 9:59 PM, Ralph Goers wrote: > Since I was totally confused by what the infra mandate meant I am going to > assume it is probably unclear to most everyone here. I've been working on > getting the logging site moved and I think I have a decent handle on it now. > Since Flume is about to become a TLP and is going to have to move its web > site this is the obvious time to conform to the requirements. > > The basic principal is that web sites are now backed by subversion. There > are actually 3 locations that matter 1) where your initial content (the > source) resides, 2) the staging site, 3) the production site. All of these > are locations in subversion. We get to determine the first, the second and > third are pretty much dictated. Projects have two choices for getting > content into their web site; a) use the CMS or b) use svnpubsub. In both of > these cases the web content is placed into your source area via svn commits. > With the CMS it automatically detects the changes and publishes them to the > staging site. I've not done it, but my understanding is that svnpubsub is a > manual process to do the same thing. > > Because you are using subversion, if you use Maven to build your full web > site it is possible that some files will be added and others deleted. So you > can't just do a build and then commit that to the source location as it will > leave old stuff lying around. That is where the Maven SCM Publish Plugin > comes in. It detects the difference between the current version and the new > version and performs the adds and deletes for you. Of course, committing a > whole site is going to generate a fair amount of commit logs. > > The CMS is much simpler. It uses the same subversion area and you can either > interact with it directly via normal editing or you can use the CMS web > interface. You can also customize the template engine the CMS uses if you > don't like the default. > > The last item is sort of a compromise that the logging project uses to mix > using the CMS with Maven builds. If you go to http://loggingtest.apache.org > you will see that logging is an umbrella project with a number of > sub-projects. What we did was use the CMS to host the main site. However, > each of the sub-projects is built with Maven or a similar tool. The CMS > allows you to have a file named extpaths.txt which identify paths in the > production site that the CMS should ignore. What we did was create a path > for each of the sub-projects. We then commit each site under a specific > version such as log4j-1.2.17 directly to the production site (you can't > upload it to staging or your source area). We then have a symbolic link from > 1.x to 1.2.17. The CMS then refers to 1.x. So at the next release we will > upload a full site build under the new release number and change the link. At > some point we will just remove the content for the old release. This will > make it much easier to deal with cleaning up. > > If it was totally up to me I would find a way to use the CMS for the things > that are fairly static and then use the extpath method to update per-release > content such as javadoc, RAT, findbugs, etc. However, I'm not sure I'd use > mvn site to do that. Some of the plugins, like findbugs, only generate their > reports per project, not as an aggregate. Thus you are forced to have the > full multi-module site to get them. Javadoc is actually one of the nice > plugins that has a configuration property to allow you to aggregate it (which > is what I did in the flume-1262 site). OTOH, you may not want the javadoc for > modules you consider to be optional to be included. IMO, including the > tutorials, guides etc in the release is actually not the best approach as it > prevents you from updating documentation bugs continuously. > > For reference, the loggingtest source site is at > https://svn.apache.org/repos/asf/logging/site/branches/cms/trunk/ and is > using something called Twig as the templating engine. The staging site is at > https://svn.apache.org/repos/infra/websites/staging/loggingtest/trunk/ and > the production site is at > https://svn.apache.org/repos/infra/websites/production/loggingtest/. You can > access the CMS at https://cms.apache.org/#bookmark. Also, the logging > project has documented our process at > http://wiki.apache.org/logging/ManagingTheWebSite. > > Finally, I'm going to reiterate that using this system is not optional. Infra > is going to stop publishing from people.apache.org by the end of the year so > everyone has to convert by then. As we graduate I expect they will want us to > do this now since we basically have to set it up from scratch anyway. > > Ralph > > > > On Jun 19, 2012, at 8:06 PM, Mike Percy wrote: > >> Having findbugs in the build would be great. >> >> I think most people would want the following docs: >> - Tutorial >> - Cookbook / examples >> - Configuration reference >> - Plugin dev guide >> >> I am sure there is room for more types of docs but I'm unsure what is being >> achieved by splitting everything up on a per-Maven-module basis (that's what >> Javadoc is for, right?). In cases like core, configuration, and node - what >> docs would someone expect from those? >> >> Regarding the CMS - I'm not sure we would want this stuff integrated with >> CMS. We should be able to check in a doc snapshot from the release tarball >> every time there is a release. CMS is for maintaining content that changes >> over time. >> >> Mike >> >> >> On Tuesday, June 19, 2012 at 1:43 AM, Ralph Goers wrote: >> >>> FWIW, I added the aggregate javadocs and each project should now have a >>> findbugs report in addition to the RAT report. Unfortunately, several of >>> the findbugs reports aren't empty. >>> >>> Ralph >>> >>> On Jun 19, 2012, at 1:09 AM, Ralph Goers wrote: >>> >>>> The issue with Sphinx appears to be that it is creating a new Python >>>> environment in every module. This is in service to generate exactly 2 >>>> documentation files. >>>> >>>> As far as I know xdoc is more powerful than APT and is the format I've >>>> always used. However, Maven supports quiet a few template formats - >>>> http://maven.apache.org/doxia/modules/index.html. >>>> >>>> From what I could see RST is very similar to APT. I only converted the RST >>>> to xdoc because a) I'm quite familiar with it and b) I found a tool to >>>> convert it from RST to xdoc so I didn't need to do much. >>>> >>>> The problem with Sphinx is that when it builds the user guide and dev >>>> guide it isn't integrated with the rest of the documentation built by the >>>> site plugin - assuming you want a multi-module site as I've done in >>>> flume-1262. If you don't want a multi-module site then I would suggest not >>>> using Maven to build the site at all. We are mandated to start using the >>>> Apache CMS by the end of the year. Using the site I've built will require >>>> updating it with the maven-scm-publish-plugin - >>>> http://maven.apache.org/sandbox/plugins/asf-svnpubsub-plugin/, which is >>>> going to be somewhat painful. If all you have are a bunch of static pages >>>> and don't really want the maven generated reports then the CMS is a much >>>> better way to go as you can directly edit the pages there. >>>> >>>> As for flume-1262, the only way for you to understand what a multi-module >>>> site would look like was to build it and then let you take a look. To do >>>> that check out the branch and then run >>>> >>>> mvn site >>>> mvn site:deploy -DsiteUrlDeployment=file:///Users/xxxx/flume >>>> >>>> where xxxx is your userid. Then browse to ~/flume/index.html to see the >>>> site. Note that it isn't actually generating many reports right now. >>>> Normally each project would include findbugs, pmd, clover or cobertura, >>>> etc, although some of these may also be on a CI server. >>>> >>>> Ralph >>>> >>>> >>>> On Jun 19, 2012, at 12:32 AM, Mike Percy wrote: >>>> >>>>> Hi Devs, >>>>> I noticed the work going on in branch "flume-1262", which appears to >>>>> involve porting the existing RST docs to xdoc. I guess this is in >>>>> response to a couple of concerns some people have raised: (1) Maven 2 >>>>> isn't compatible with the way Sphinx is integrated with the build >>>>> currently <https://issues.apache.org/jira/browse/FLUME-1282>, and (2) it >>>>> appears that invoking Jython to run Sphinx takes more than the default >>>>> amount of memory allocated to Maven (requires setting MAVEN_OPTS to >>>>> increase the memory for the build) >>>>> <https://issues.apache.org/jira/browse/FLUME-1256>. >>>>> >>>>> Since these issues have clear workarounds, I haven't set aside the time >>>>> to try to fix them myself. However, I thought it might be a good idea to >>>>> discuss openly on the dev list whether switching to another format makes >>>>> sense (and if so, which one). It seems that xdoc was mostly replaced by >>>>> APT (almost plain text) in Maven 2 (according to >>>>> https://maven.apache.org/doxia/references/xdoc-format.html ). I wonder if >>>>> there is a reason why Maven rewrote all their documentation in APT. >>>>> >>>>> While I'm sure RST and Jythonized Sphinx have some drawbacks, they also >>>>> have the following benefits: >>>>> - RST is meant to be very readable in source form >>>>> - Even tables are readable in source form and are easy to maintain >>>>> - Sphinx is skinnable / themeable and widely used in Python land >>>>> - Can output to HTML and PDF >>>>> >>>>> I'm certainly not against moving to something else, but it might be a >>>>> good idea to get consensus on a direction before going and independently >>>>> reformatting all the documentation. Since the existing docs are already >>>>> editable by the community, and people have been submitting patches to >>>>> them (while the previous HTML was totally unmaintainable), let's make >>>>> sure we end up with something better if we decide to replace it. >>>>> >>>>> Regards, >>>>> Mike >>>> >>> >> >> >> >
