On 02/01/2013, at 11:43 PM, Brett Porter <br...@apache.org> wrote: > > On 02/01/2013, at 10:45 AM, Hervé BOUTEMY <herve.bout...@free.fr> wrote: > >> Le lundi 24 décembre 2012 14:17:02 Brett Porter a écrit : >>> I had the same feeling pushing up Continuum's Maven site recently... >>> >>> On 23/12/2012, at 9:36 AM, Olivier Lamy <ol...@apache.org> wrote: >>>> 2012/12/22 Kristian Rosenvold <kristian.rosenv...@zenior.no>: >>>>> Svnsubpub is just ridiculously inefficient and we need to do something >>>>> about it... >>>> >>>> That remember me old time when pushing maven sites to sourceforge tru >>>> wagon ftp ... :-) >>> >>> Is the main problem the file-by-file nature, or something else (such as the >>> size)? >> I tried to measure and find facts about this: my conclusion is that it is >> the >> file-by-file nature >> exactly like webdav >> notice that we have really a bunch of generated files: did you realize that >> maven-scm site has more files than maven core? that it represents 131 MB, >> tens >> of thousand of files? > > Yes, similar size in Continuum which I've been working on migrating to > svnpubsub recently. > >> >> IMHO, if we stored zip files of documentations (or tar, the archive format >> can >> be discussed) and httpd could serve their content on the fly like if they >> were >> unzipped, this would be awesome >> Apparently, coding an lua script could do the job: just need to find >> somebody >> able to do it :) > > I've raised this topic and that thread on #asfinfra and will try and follow > up - I know this was flagged by them some time ago.
See my response to your mail over there today - this may already be possible: http://www.apache.org/dev/cmsref.html#generated-docs But aside from that... > >> >> any other idea? > > For now I'm just focusing on reducing the number of changes each site > deployment to make it a smaller checkin. > > See: http://continuum.apache.org/development/publishing-site.html > > What I'm doing there is deploying to /docs/latest each time, then copying > that to the versioned location on the server-side. That way, the next version > is just the diff against the last, not a whole new one. > > There are a few things that cause changes across generation runs: > - the javadocs generate a timestamp, and this is the biggest culprit as it is > the largest number of files. I'm going to add -notimestamp tomorrow and see > if I can get sequential deploys to remain unmodified > - most skins incorporated a timestamp that changes each build (I've removed > that in Continuum's skin). The publish date is still there, but it doesn't > seem to be a big problem. > - the dependencies report seems to change between runs By addressing those 3 things, I got down to either no, or very few, changes between commits (it seems Javadoc doesn't consistently order some elements on the page), e.g.: http://svn.apache.org/r1428378 The dependencies issue was fixed in MPIR-266 (which may need a release). Making the skin timestamp configurable is another task. I added a few tips to the ASF site: http://www.staging.apache.org/dev/project-site.html#generated > > For other reports the change more frequently, I've stopped publishing them to > the main site. There is Sonar for most of it at http://analysis.apache.org/, > or you can publish them to a filesystem staging in CI and view them through > the workspace. > > With these sorts of improvements, incremental deployments will actually be > better than shipping up a large ZIP that gets unpacked, even though it is > mostly unchanged. Does this help? Cheers, Brett -- Brett Porter br...@apache.org http://brettporter.wordpress.com/ http://au.linkedin.com/in/brettporter http://twitter.com/brettporter --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
