Hi All,
This is an update into my investigations on automating the release
process for
FOP. As we're nearing release it looks as though version 2.0 will
remain a
manual process for the time being. That's not to say that it will
forever
remain like that but at present unless a breakthrough occurs or I
receive some
feedback from the infrastructure team, I don't currently have the
necessary
knowledge on the Apache infrastructure (or Perl know how) to achieve the
desired result despite my efforts.
During the time since my last message I found a solution using a
markdown
extension. This appeared to fulfil all of our requirements and after
writing
and testing one, it seemed to simply be a case of installing it. Due
to the
nature of Apache's websites this was not something I could do myself
as we
don't have control over the CMS. After raising a ticket with the
infrastructure team about doing this, I was pointed to another
project called
Thrift. Their site appeared to provide tag replacement using preexisting
functionality found in the perl modules of the Apache CMS.
After reading the documentation and experimenting I've reached
somewhat of an
impasse due to a number of reasons. Firstly the documentation on
customizing
these patterns is limited and covers only basic patterns. Second, my own
experience with Perl is lacking and as such makes it hard to debug and
understand some of the more complicated required modules and sections
of the
CMS. Finally during my testing the errors I was getting were extremely
unhelpful and provide next to no clues as to where the problem lay in
my own
code. Instead they point to the Perl CMS libraries highlighting missing
expected characters and at a guess incompatibilities between the
markdown
we're using and what's expected by the pattern's own subroutine.
I have tried to follow and utilize the code found in the Thrift
project with
little luck. I have posted on the infrastructure mailing list for
help but as
of yet have not had any responses. I am guessing this is not a
commonly used
feature and as such knowledge on the subject may be in short supply.
As such
and without possibility of using the markdown extension we're left
with the
manual process for the time being. I will keep an eye out on the
infrastructure page and prod them occasionally to see if I can move
things along.
Apologies for the long e-mail but just wanted to keep you all updated.
Robert Meyer
> Date: Mon, 2 Jun 2014 14:44:58 +0100
> From: [email protected]
> To: [email protected]
> Subject: Re: FOP Release Automation
>
> Hi All,
>
> I certainly use the web interface when making small tweaks to the
docs.
> As you know users occasionally report small mistakes that require
minor
> tweaks. I'd like to streamline the updating of the website for
release
> purposes but I don't want to disable/prevent the current web
> interface which works well when all you want to do is make a minor
> adjustment in response to a user e-mail.
>
> Rob is away this week, but he will continue the investigation into
> scripting the website updates when he returns.
>
> Thanks for the ideas so far, a few promising leads.
>
> Thanks,
>
> Chris
>
> On 30/05/2014 17:23, Clay Leeds wrote:
> > Agreed, ‘some’ people wouldn’t be happy with that. ;-)
> >
> > I wonder if the CMS Web interface could be extended to allow for
a few
> > keywords like FOP_VERSION, FOP_REVISION, FOP_BRANCH, etc.
> >
> > The CMS tool's WYSIWYG interface indicates it uses the Wysiwym
> > MarkDown Editor, which is extensible:
> >
> > https://web.archive.org/web/20110121060932/http://wmd-editor.com/
> >
> > (site’s down & hasn’t been updated since 2011)...
> >
> > or
> >
> > https://code.google.com/p/wmd/
> >
> > We might still need to do some ANT hanky panky, but at least if we
> > could leverage WMD’s extensibility it would help us get where we’re
> > trying to go?
> >
> > Clay
> >
> > On May 30, 2014, at 7:19 AM, Robert Meyer <[email protected]
> > <mailto:[email protected]>> wrote:
> >> Hi,
> >>
> >> I like the simplicity of your idea, but the web interface is
not so
> >> easy to dismiss unfortunately.
> >>
> >> If you do have a copy with those tags in, if any changes are
made on
> >> the web interface then that copy would become out of date.
> >>
> >> We could always shutdown the web interface, but I don't think too
> >> many people would be happy with that ;-)
> >>
> >> Regards,
> >>
> >> Robert
> >>
> >>
------------------------------------------------------------------------
> >> From: [email protected]
<mailto:[email protected]>
> >> To: [email protected]
> >> <mailto:[email protected]>
> >> Subject: RE: FOP Release Automation
> >> Date: Fri, 30 May 2014 14:48:15 +0100
> >>
> >> Hi,
> >>
> >> Simple way is to store docs inside fop repo:
> >>
> >> Fop/docs/index.markdown
> >>
> >> Inside markdown file you reference ant properties eg:
> >> ${version}
> >>
> >> Then you call which does ant expandproperties and calls
markdown to
> >> html tool:
> >> ant docs
> >>
> >> Then you call which does a zip, scp and unzip of html files to web
> >> server:
> >> ant upload-docs
> >>
> >> This method doesn’t support web interface of editing files but I
> >> don’t see how this is really needed.
> >> If I submit a patch to fop it should also contain doc changes
rather
> >> than having separate repo and patch for that.
> >>
> >> Thanks
> >>
> >> *From:*Robert Meyer [mailto:[email protected]]
> >> *Sent:*30 May 2014 14:05
> >> *To:*[email protected]
> >> <mailto:[email protected]>
> >> *Subject:*RE: FOP Release Automation
> >>
> >> Hi,
> >>
> >> After investigating your suggestions Clay I have found that
svn-hooks
> >> can't be used for the purpose we require unfortunately as it
may lead
> >> to problems with how SVN operates and also may have some
unexpected
> >> results with files being committed. This is stated in the
> >> documentation under "Creating Repository Hooks" highlighted in the
> >> warning red box further down:
> >>
> >>
http://www-inst.eecs.berkeley.edu/~cs61b/fa09/docs/svn-book-html-chunk/svn.reposadmin.create.html
> >>
<http://www-inst.eecs.berkeley.edu/%7Ecs61b/fa09/docs/svn-book-html-chunk/svn.reposadmin.create.html>
> >>
> >> Pascal, I agree that the process is fairly straightforward, but I
> >> have been asked to automate this further so am just looking into
> >> ideas presently.
> >>
> >> I think a possible way forward then would be to use your
suggestion
> >> Pascal of placing the versioned docs for the site inside the FOP
> >> repository for their associated version. These can then be
referenced
> >> using the svn-externals from the main site repository.
> >>
> >> In addition to this, the main site files (which would need to be
> >> updated) could contain tags for the last three versions which
would
> >> be replaced using Clay's markdown pre-processor suggestion. The
> >> pre-processor would replace the tags with values stored in a
> >> properties file in the main site repository.
> >>
> >> To create a release, the user would need to update the
svn-external
> >> references to account for the new version and update the
properties
> >> file for tag replacement. When the properties file is pushed it
will
> >> be read by the custom markdown pre-processor and display the new
> >> version when rendered.
> >>
> >> Those two stages could be done using a single script to simplify
> >> things further, but the main complication is getting the markdown
> >> pre-processor working. From looking at this page:
> >>
> >> http://www.apache.org/dev/cmsref.html#markdown
> >>
> >> I am guessing we use PyPy (Python Markdown) which supports
> >> extensions, so I will look at this shortly to try out a small
example
> >> and investigate the feasibility of doing this. There is also the
> >> matter of updating the versioned documents for each FOP when a new
> >> version is released, but maybe this could be done with the
> >> pre-processor as well.
> >>
> >> Anyway, let me know what you think.
> >>
> >> Regards,
> >>
> >> Robert
> >
>
