If someone's looking for a way to help with the porting
effort, consider tackling the creation of an index.html
page in each subdir of the content/ tree (other than the
base which has an /index.md). The contents of the file
should be just
{% include 'default' %}
(see the docs dir for an example page that can be copied)
and the build system will take care of generating the page
in a consistent fashion with the rest of the site. Once
that is done there are a lot of pages that function like
index pages but are written in markdown that can be deleted,
like docs.md and its ilk. Nanoc transforms that file into
/docs/index.html but that's kinda crufty and not supported
by the CMS, which only lets you alter file extensions not paths.
> On Wednesday, March 19, 2014 8:54 AM, Joe Schaefer <[email protected]>
> wrote:
> > I just pencilled in snippet preprocessing support into
> the thrift builds. Once Jake and I have settled on the
> feature set of ASF::Value::Snippet, and we have ported
> the snippet calling text within the markdown pages
> to the new format, things will just work (knock on wood).
>
>
>
>
>> On Wednesday, March 19, 2014 6:48 AM, Joseph Schaefer
> <[email protected]> wrote:
>> > Sure I understand that need. If there's a way of pulling the
> document
>> sources directly from the web I can parse the content to look for snippet
>> markers and make those available to the template system. That's pretty
> much
>> how www.apache.org is generated.
>>
>> Sent from my iPhone
>>
>>
>>> On Mar 19, 2014, at 5:28 AM, Henrique Mendonça
> <[email protected]>
>> wrote:
>>>
>>> Hi Joe,
>>>
>>> Thanks for your effort. I just wanted to add that the code snippets
> are a
>>> crucial feature for this project, which has way more target languages
> than
>>> active contributors. In our case, reducing manual maintenance costs is
> much
>>> more important than the total site build time. Perhaps we can also
> solve
>>> this in the CMS, though.
>>>
>>> - Henrique
>>>
>>>
>>>> On 19 March 2014 03:17, Joe Schaefer
> <[email protected]>
>> wrote:
>>>>
>>>> A couple more thoughts regarding the background on the CMS...
>>>> the bulk of the focus of the CMS up until now has been to
>>>> support really big sites like maven and openoffice, both of
>>>> which have websites well over 5GB worth of text files. The
>>>> build system was hollowed out and revamped to support external
>>>> builds in other programming languages, as well as scaling up
>>>> the perl builds to provide about 8 child processes that build
>>>> the site in parallel by default. Full site build times for
>>>> openoffice went from over 30 minutes to between 5 to 10 minutes
>>>> depending on disk activity, which made experimentation with
> site-wide
>>>> changes feasible on a reasonable timeframe. At first the project
> was
>> in
>>>> the habit of running the builds locally before checking in their
>> changes
>>>> because the delay in feedback post-commit was unreasonable, but
> with
>> smart
>>>> use of SSI the were able to cut down on the number of changes that
>
>> required
>>>> full site builds and hence no longer bother with the whole
> pre-build
>> mumbo
>>>> jumbo before checking in changes. The CMS is designed to only
> build
>> pages
>>>> that were altered, and if you just edit a page or two only those
> pages
>> and
>>>> the pages that depend on them will be built.
>>>>
>>>> Now that I've turned my attention to thrift, there are several
>
>> aspects
>>>> that are better off being retuned based on the modest size of the
> site.
>>>> First of all you have a sitemap url that lists all of your
>> markdown-based
>>>> pages, which would be a nonstarter for a site like openoffice.
> The
>>>> implications of the sitemap url are that every page needs to be
> built
>> in
>>>> memory just to generate that page, which leads to a lot of
> redundancy
>> in a
>>>> build system designed to run as forked child processes. We can do
>
>> better...
>>>>
>>>> I've done two things to make this work well- first I made the
>> number of
>>>> child processes "aka runners" tunable on a per-project
> basis,
>> and since the
>>>> sitemap is going to build every page anyway as the slowest page to
>
>> build, I
>>>> memoized the main view that builds the markdown pages for thrift.
> The
>> best
>>>> results are when there's only one runner so we get the full
> benefit
>> of
>>>> memoization: typical site build times are consistently under 2
> seconds
>> now.
>>>>
>>>> You'll appreciate why I am such a performance stickler for the
> CMS
>> when
>>>> you start using the bookmarklet, since builds are the only
> asynchronous
>>>> part of the process from making a change to a page and publishing
> the
>>>> results. Ideally when the system is working well the builds
> happen
>> before
>>>> you have time to go looking for them, but you do need to be
> mindful of
>> not
>>>> publishing before the built pages have been committed back to svn
> by
>>>> buildbot.
>>>>
>>>>
>>>>
>>>>> On Monday, March 17, 2014 11:59 PM, Joe Schaefer
>> <[email protected]>
>>>> wrote:
>>>>>> Ok there's enough of an idea about how to port the
> rest of
>> the site's
>>>>> pages over based on the porting work I've already done
> that
>> it's time
>>>>> for me to step back and let you guys catch up. Basically the
> idea
>> is
>>>> that while
>>>>> there is templating support in the markdown sources, it's
>> fairly limited
>>>>> compared to what nanoc offers, but instead of writing a lot of
>
>> complex
>>>> template
>>>>> code into your sources, with the cms you just add code to
>> lib/view.pmand/or
>>>>> lib/path.pm to keep the document sources simple and just add
> more
>>>> template
>>>>> variables. What I just finished in lib/view.pm and
> lib/path.pm is
>>>> directory
>>>>> listing support for /docs.html but it can be carried over to
>> similar
>>>> pages.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> On Monday, March 17, 2014 9:16 PM, Joe Schaefer
>>>>> <[email protected]> wrote:
>>>>>>> Interesting ideas, thanks. FWIW I think that sort
>>>>>> of thing might be best supported as a periodic cron
>>>>>> if we can write something stable enough, because pulling
>>>>>> stuff out of the git repo is something the CMS isn't
> going
>>>>>> to do all that well being an svn application.
>>>>>>
>>>>>> The CMS build results are available at
>>>> http://thrift.staging.apache.org/
>>>>>> feel free to play with the sources in
>> repos/asf/thrift/cms-site/trunk.
>>>>>> It's very fast, probably an order of magnitude faster
> than
>> your current
>>>>>
>>>>>> tech.
>>>>>> Whole site builds in two seconds; fractions of a second
> for
>> typical
>>>> mods to
>>>>>
>>>>>> content/ markdown pages.
>>>>>>
>>>>>> The CMS bookmarklet is compatible with the staging site
> FWIW.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Monday, March 17, 2014 7:40 PM, Roger Meier
>>>>>> <[email protected]> wrote:
>>>>>>>> Hi Joe & Co
>>>>>>>
>>>>>>> thanks for this test run with Apache CMS!
>>>>>>>
>>>>>>> What I would love to see is this:
>>>>>>>
>>>>>>> Source file: => Web Site URL:
>>>>>>> test/README.md => thrift.apache.org/test
>>>>>>> test/STATUS.md => thrift.apache.org/test/status
>>>>>>> test/keys/README.md => thrift.apache.org/test/keys
>>>>>>> lib/<lang>/README.md =>
>> thrift.apache.org/lib/<lang>
>>>>>>> lib/<lang>/test/README.md =>
>>>>>> thrift.apache.org/lib/<lang>/test
>>>>>>> lib/<lang>/examples/README.md =>
>>>>>>> thrift.apache.org/lib/<lang>/examples
>>>>>>> compiler/cpp/README.md =>
> thrift.apache.org/compiler
>>>>>>> debian/README.md => thrift.apache.org/debian
>>>>>>> contrib/<topic>/README.md =>
>> thrift.apache.org/<topic>
>>>>>>>
>>>>>>> just made the following patch to rename all file to
> .md
>> within our
>>>>> source
>>>>>>> tree:
> https://issues.apache.org/jira/browse/THRIFT-2407
>>>>>>> ... we have *41* md's within the source tree and
>> that's the
>>>>> place
>>>>>> where
>>>>>>> people look for documentation first. => simple to
>> manage
>>>>>>> Would be user friendly to have all of this online
> with
>> each release
>>>>>>> with the URL layout mentioned above.
>>>>>>>
>>>>>>> just received the buildboot...
>>>>>>> Do you have kind of a preview of your prototype?
>>>>>>>
>>>>>>> all the best!
>>>>>>> -roger
>>>>>>> ;-r
>>>>>>>
>>>>>>>
>>>>>>> Quoting Jake Farrell <[email protected]>:
>>>>>>>
>>>>>>>> No objections from me
>>>>>>>>
>>>>>>>> -Jake
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Mar 17, 2014 at 12:04 PM, Joe Schaefer
>>>>>>> <[email protected]>wrote:
>>>>>>>>
>>>>>>>>> Any objections to me making a copy of the
> site
>>>>>>>>> tree alongside it called cms-site? I'd
> like
>> to
>>>>>>>>> get cracking on knocking up the supporting
> perl
>>>>>>>>> for this so we can continue to brainstorm...
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Sunday, March 16, 2014 1:53 PM, Joe
> Schaefer
>>>>>>> <[email protected]>
>>>>>>>>> wrote:
>>>>>>>>>>> On Sunday, March 16, 2014 1:40 PM,
> Jake
>> Farrell
>>>>>>> <[email protected]>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hey Joe
>>>>>>>>>>> Thanks for reaching out, we chose to
> go
>> with a site
>>>>>> generator
>>>>>>> over the
>>>>>>>>>>> Apache CMS due to it initially not
> meeting
>> all of
>>>>> our
>>>>>> needs.
>>>>>>> Our current
>>>>>>>>>>> needs for the site are
>>>>>>>>>>>
>>>>>>>>>>> - Markdown to html conversion
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> That one's easy- its the default for
> the
>> cms.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> - Global variables/config usable
> throughout
>> in a
>>>>> template
>>>>>>
>>>>>>> based layout
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Easy too- just create a perl hash
> somewhere and
>> make it
>>>>>> available
>>>>>>> to
>>>>>>>>> your views.
>>>>>>>>>> Code your views to pass that hash along to
> your
>>>>> templates
>>>>>> and you
>>>>>>> are
>>>>>>>>> all set.
>>>>>>>>>> Note if your hash contains objects you can
> make
>> method
>>>>> calls
>>>>>> on
>>>>>>> them in
>>>>>>>>> your
>>>>>>>>>> templates. Note: while I wouldn't
>> recommend this in
>>>>>
>>>>>> general,
>>>>>>> for smaller
>>>>>>>>>> projects like thrift that prefer
> convenience
>> over
>>>>> separation
>>>>>> of
>>>>>>> concerns
>>>>>>>>> you can
>>>>>>>>>> have the django template engine do a pass
> over
>> your
>>>>> markdown
>>>>>>> before
>>>>>>>>> passing it
>>>>>>>>>> along to your actual template page, just
> as is
>> seemingly
>>>>>
>>>>>> common to
>>>>>>> other
>>>>>>>>> popular
>>>>>>>>>> site generation software.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> - Syntax highlighting
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Comes with python's markdown support,
> but
>> some
>>>>> people
>>>>>> prefer
>>>>>>> the look of
>>>>>>>>>> javascript highlighters.
>>>>>>>>>>
>>>>>>>>>>> - Easily include code snippets from
> our
>> source code
>>>>> base
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Statically I hope. No idea how to pull
>> snippets out of
>>>>> your
>>>>>> git
>>>>>>> repo
>>>>>>>>> via the
>>>>>>>>>> cms.
>>>>>>>>>>
>>>>>>>>>>> - Ability to locally test before
> committing
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Belt and suspenders with the cms- you can
> build
>> your
>>>>> changes
>>>>>>> before
>>>>>>>>> committing
>>>>>>>>>> with the build scripts, browse your
> changes
>> before
>>>>> committing
>>>>>> via
>>>>>>> the cms
>>>>>>>>>> webgui, or simply commit your changes and
> view
>> the
>>>>> staging
>>>>>> site
>>>>>>> prior to
>>>>>>>>> live
>>>>>>>>>> publication (which is a separate step).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> -Jake
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Sun, Mar 16, 2014 at 12:44 PM, Joe
>> Schaefer
>>>>>>>>>> <[email protected]>wrote:
>>>>>>>>>>>
>>>>>>>>>>>> As it so happens I am interested
> in
>> working on
>>>>>> supporting
>>>>>>>>>>>> thrift's use case as a
> potential
>> user of
>>>>> the
>>>>>> Apache
>>>>>>> CMS.
>>>>>>>>>>>> While the CMS works well for
> massive
>> projects
>>>>> there
>>>>>> are
>>>>>>>>>>>> things it could do better at
> supporting
>> for
>>>>> more
>>>>>> moderate
>>>>>>>>>>>> sized sites like thrift's.
>>>>>>>>>>>>
>>>>>>>>>>>> I'd therefore like to engage
> you
>> folks in a
>>>>>
>>>>>>> brainstorming session
>>>>>>>>>>>> about what sorts of features you
> want
>> for your
>>>>> site
>>>>>> and
>>>>>>> to find
>>>>>>>>>>>> simple ways of supporting those
>> features for
>>>>> you.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks.
>>>>
>>
>
