Here’s a screen shot, attached :-)
On 1/19/17, 1:04 AM, "Matt Foley" <[email protected]> wrote:
Hi all,
I’ve put together a prototype doc book, along the lines we discussed, and
it looks pretty worthwhile.
Many thanks to Mike M. who whipped the pom.xml file into shape, and helped
me find the right site.xml file to imitate.
If you’re interested, please do a single-branch clone as follows:
git clone --single-branch -b METRON-660
https://github.com/mattf-horton/incubator-metron.git [clonename]
(or whatever git command pleases you :-)
In this branch, there’s a new top-level subdirectory named site-book/.
This is not necessarily how we want to integrate stuff, it was just convenient
to do it separate from the existing site/ directory for now. To build the
book, do these three commands:
cd site-book/
bin/generate-md.sh #This gathers all the *.md files into
site-book/src/site/markdown/**, and generates the menu tree into
site-book/src/site/site.xml
mvn clean site:site #This builds the book with the maven site
plugin and doxia-markdown plugin
If both those steps are successful, you can then go to a browser and open
file:///Users/yourname/yourpath/clonename/site-book/target/site/index.html
and see the book, with the nav menu on the LHS.
It’s important to note that a very usable (not perfect) nav hierarchy has
been auto-generated; this is not hardwired nor hand-edited.
I do plan to add some overrides that allow individual items in the menu to
be tweaked.
While it already looks fairly nice, and clearly illustrates the value of
building a book, there are two glaring issues.
• Doxia-markdown doesn’t process the triple back-tick (```) the same way as
Github Markdown. It seems to color-code it as <code>, but doesn’t preserve
line breaks, which is really bad.
• Similarly, it only processes bullet lists in isolation, and it doesn’t
correctly combine bullet lists subordinate to a numbered list.
The upshot is that
• both code and bullet lists often lose their linebreaks, and get mushed
into run-on paragraphs, usually combined with the preceding paragraph, and
• bullet lists interrupt numbered lists and make them start over at #1.
Perhaps 80-90% of these issues can be fixed by editing the markdown files
to put blank lines around the list formats. I started doing this, but I didn’t
want to obscure the proto by editing tons of .md files. As of this proto, only
the half dozen actually broken files (that caused maven site build errors) have
been fixed.
The other 10-20% will just require simplification of the markdown used,
unless we can get an updated version of the plugins.
Anyway, please take a look and share your thoughts.
Thanks,
--Matt
On 1/16/17, 1:02 PM, "Michael Miklavcic" <[email protected]>
wrote:
Hey Matt, feel free to ping me.
On Mon, Jan 16, 2017 at 1:39 PM, Matt Foley <[email protected]> wrote:
> I looked into the Falcon website and doxia over the weekend, and I’m
> convinced that using the doxia-markdown plugin should make it dirt
simple
> to do what’s been discussed in this thread, with no overhead on the
part of
> people writing the README.md files.
>
> I fiddled with trying to do a POC, and unfortunately concluded (again)
> that I don’t really know maven very well :-)
> Are there any maven experts out there who would be willing to give me
some
> pointers (offline) on how to make use of this apparently simple maven
> plug-in?
>
> I can do the bit of scripting needed to gather the docs. I’ve opened
> https://issues.apache.org/jira/browse/METRON-660 with some sub-tasks
for
> this work.
> --Matt
>
> On 1/13/17, 12:04 PM, "[email protected]" <[email protected]> wrote:
>
> +1 on any improvement to documentation and more consistency. At
this
> point, I think getting rid of or hiding some of the pages on the
wiki
> (at
> least for the short term) would be better than leaving them around
> because
> there's a lot of misinformation.
>
> Jon
>
> On Fri, Jan 13, 2017 at 10:13 AM Nick Allen <[email protected]>
> wrote:
>
> > +1 I think it is sorely needed.
> >
> > If we can come up with a really slick solution like Spark, then
> great. I am
> > also not against a half-baked solution that can later evolve
into
> something
> > else. For example, create an index README.md that links
together
> all the
> > existing READMEs and run Pandoc on it. Not ideal, but way
better
> than what
> > we have.
> >
> >
> >
> > On Fri, Jan 13, 2017 at 9:53 AM, Otto Fowler <
> [email protected]>
> > wrote:
> >
> > > I think something that does what you have laid out here, no
matter
> the
> > > implementation details would be ideal
> > >
> > >
> > > On January 12, 2017 at 18:05:24, Matt Foley ([email protected])
> wrote:
> > >
> > > We currently have three forms of documentation, with the
following
> > > advantages and disadvantages:
> > >
> > > || Docs || Pro || Con ||
> > > | CWiki |
> > > Easy to edit, no special tools required, don't have to be a
> developer to
> > > contribute, google and wiki search |
> > > Not versioned, no review process, distant from the code,
obsolete
> content
> > > tends to accumulate |
> > > | Site |
> > > Versioned and reviewed, only committers can edit, google
search |
> > > Yet another arcane toolset must be learned, only web
programmers
> feel
> > > comfortable contributing, "asf-site" branch not related to
code
> versions,
> > > distant from the code, tends to go obsolete due to
non-maintenance
> |
> > > | README.md |
> > > Versioned and reviewed, only committers can edit, tied to code
> versions,
> > > highly local to the code being documented |
> > > Non-developers don't know about them, may be scared by
github, poor
> > scoring
> > > in google search, no high-level presentation |
> > >
> > > Various discussion threads indicate the developer community
likes
> > > README-based docs, and it's easy to see why from the above. I
> propose
> > this
> > > extension to the README-based documentation, to address their
> > > disadvantages:
> > >
> > > 1. Produce a script that gathers the README.md files from all
code
> > > subdirectories into a hierarchical list. The script would
have an
> > exclusion
> > > list for non-user-content, which at this point would consist
of
> [site/*,
> > > build_utils/*]. The hierarchy would be sorted depth-first. The
> resulting
> > > hierarchical list at this time (with six added README files to
> complete
> > the
> > > hierarchy) would be:
> > >
> > > ./README.md
> > > ./metron-analytics/README.md <== (need file here)
> > > ./metron-analytics/metron-maas-service/README.md
> > > ./metron-analytics/metron-profiler/README.md
> > > ./metron-analytics/metron-profiler-client/README.md
> > > ./metron-analytics/metron-statistics/README.md
> > > ./metron-deployment/README.md
> > > ./metron-deployment/amazon-ec2/README.md
> > > ./metron-deployment/packaging/README.md <== (need file here)
> > > ./metron-deployment/packaging/ambari/README.md <== (need file
> here)
> > > ./metron-deployment/packaging/docker/ansible-docker/README.md
> > > ./metron-deployment/packaging/docker/rpm-docker/README.md
> > > ./metron-deployment/packer-build/README.md
> > > ./metron-deployment/roles/ <== (need file here)
> > > ./metron-deployment/roles/kibana/README.md
> > > ./metron-deployment/roles/monit/README.md
> > > ./metron-deployment/roles/opentaxii/README.md
> > > ./metron-deployment/roles/pcap_replay/README.md
> > > ./metron-deployment/roles/sensor-test-mode/README.md
> > > ./metron-deployment/vagrant/README.md <== (need file here)
> > > ./metron-deployment/vagrant/codelab-platform/README.md
> > > ./metron-deployment/vagrant/fastcapa-test-platform/README.md
> > > ./metron-deployment/vagrant/full-dev-platform/README.md
> > > ./metron-deployment/vagrant/quick-dev-platform/README.md
> > > ./metron-platform/README.md
> > > ./metron-platform/metron-api/README.md
> > > ./metron-platform/metron-common/README.md
> > > ./metron-platform/metron-data-management/README.md
> > > ./metron-platform/metron-enrichment/README.md
> > > ./metron-platform/metron-indexing/README.md
> > > ./metron-platform/metron-management/README.md
> > > ./metron-platform/metron-parsers/README.md
> > > ./metron-platform/metron-pcap-backend/README.md
> > > ./metron-sensors/README.md <== (need file here)
> > > ./metron-sensors/fastcapa/README.md
> > > ./metron-sensors/pycapa/README.md
> > >
> > > 2. Arrange to run this script as part of the build process,
and
> commit
> > the
> > > resulting hierarchy list to someplace in the versioned and
branched
> > ./site/
> > > subdirectory.
> > >
> > > 3. Produce a "doc reader" web page that takes in this
hierarchy of
> .md
> > > pages, and presents a LHS doc tree of links, and a main
display
> area for
> > a
> > > currently selected file. If we want to get fancy, this page
would
> also
> > > provide: (a) telescoping (collapse/expand) of the doc tree;
(b)
> floating
> > > next/prev/up/home buttons in the display area.
> > >
> > > #4. Add to this web page a pull-down menu that selects among
all
> the
> > > release versions of Metron, and (if not running in the Apache
> site) a
> > > SNAPSHOT version for the current filesystem version (for
developer
> > > preview). Let it re-write the file paths per release version
to the
> > proper
> > > release tag in github. This web page will therefore be
> > version-independent.
> > > Put it in the asf-site branch of the Apache site, as the new
"docs"
> > > sub-site from the home web page. Update the list of releases
at
> each
> > > release, or if we want to get fancy, teach it to read the
release
> tags
> > from
> > > github.
> > >
> > > 5. As part of the release process, the release manager (a)
assures
> the
> > > release is tagged in github with a consistent naming
convention,
> and (b)
> > > submits the new hierarchy of links to google search (there's
an
> api for
> > > that).
> > >
> > > 6. Deprecate the use of cwiki for anything but long-lived
> > > demonstrations/tutorials that are unlikely to go obsolete.
> > >
> > >
> > > Do folks feel this would be a good contribution to the
visibility,
> > > timeliness, and usability of our docs?
> > > Is this an adequate solution for the current problems?
> > >
> > > Thanks,
> > > --Matt
> > >
> >
> >
> >
> > --
> > Nick Allen <[email protected]>
> >
> --
>
> Jon
>
> Sent from my mobile device
>
>
>
>
>
