Hey Matt, feel free to ping me. On Mon, Jan 16, 2017 at 1:39 PM, Matt Foley <[email protected]> wrote:
> I looked into the Falcon website and doxia over the weekend, and I’m > convinced that using the doxia-markdown plugin should make it dirt simple > to do what’s been discussed in this thread, with no overhead on the part of > people writing the README.md files. > > I fiddled with trying to do a POC, and unfortunately concluded (again) > that I don’t really know maven very well :-) > Are there any maven experts out there who would be willing to give me some > pointers (offline) on how to make use of this apparently simple maven > plug-in? > > I can do the bit of scripting needed to gather the docs. I’ve opened > https://issues.apache.org/jira/browse/METRON-660 with some sub-tasks for > this work. > --Matt > > On 1/13/17, 12:04 PM, "[email protected]" <[email protected]> wrote: > > +1 on any improvement to documentation and more consistency. At this > point, I think getting rid of or hiding some of the pages on the wiki > (at > least for the short term) would be better than leaving them around > because > there's a lot of misinformation. > > Jon > > On Fri, Jan 13, 2017 at 10:13 AM Nick Allen <[email protected]> > wrote: > > > +1 I think it is sorely needed. > > > > If we can come up with a really slick solution like Spark, then > great. I am > > also not against a half-baked solution that can later evolve into > something > > else. For example, create an index README.md that links together > all the > > existing READMEs and run Pandoc on it. Not ideal, but way better > than what > > we have. > > > > > > > > On Fri, Jan 13, 2017 at 9:53 AM, Otto Fowler < > [email protected]> > > wrote: > > > > > I think something that does what you have laid out here, no matter > the > > > implementation details would be ideal > > > > > > > > > On January 12, 2017 at 18:05:24, Matt Foley ([email protected]) > wrote: > > > > > > We currently have three forms of documentation, with the following > > > advantages and disadvantages: > > > > > > || Docs || Pro || Con || > > > | CWiki | > > > Easy to edit, no special tools required, don't have to be a > developer to > > > contribute, google and wiki search | > > > Not versioned, no review process, distant from the code, obsolete > content > > > tends to accumulate | > > > | Site | > > > Versioned and reviewed, only committers can edit, google search | > > > Yet another arcane toolset must be learned, only web programmers > feel > > > comfortable contributing, "asf-site" branch not related to code > versions, > > > distant from the code, tends to go obsolete due to non-maintenance > | > > > | README.md | > > > Versioned and reviewed, only committers can edit, tied to code > versions, > > > highly local to the code being documented | > > > Non-developers don't know about them, may be scared by github, poor > > scoring > > > in google search, no high-level presentation | > > > > > > Various discussion threads indicate the developer community likes > > > README-based docs, and it's easy to see why from the above. I > propose > > this > > > extension to the README-based documentation, to address their > > > disadvantages: > > > > > > 1. Produce a script that gathers the README.md files from all code > > > subdirectories into a hierarchical list. The script would have an > > exclusion > > > list for non-user-content, which at this point would consist of > [site/*, > > > build_utils/*]. The hierarchy would be sorted depth-first. The > resulting > > > hierarchical list at this time (with six added README files to > complete > > the > > > hierarchy) would be: > > > > > > ./README.md > > > ./metron-analytics/README.md <== (need file here) > > > ./metron-analytics/metron-maas-service/README.md > > > ./metron-analytics/metron-profiler/README.md > > > ./metron-analytics/metron-profiler-client/README.md > > > ./metron-analytics/metron-statistics/README.md > > > ./metron-deployment/README.md > > > ./metron-deployment/amazon-ec2/README.md > > > ./metron-deployment/packaging/README.md <== (need file here) > > > ./metron-deployment/packaging/ambari/README.md <== (need file > here) > > > ./metron-deployment/packaging/docker/ansible-docker/README.md > > > ./metron-deployment/packaging/docker/rpm-docker/README.md > > > ./metron-deployment/packer-build/README.md > > > ./metron-deployment/roles/ <== (need file here) > > > ./metron-deployment/roles/kibana/README.md > > > ./metron-deployment/roles/monit/README.md > > > ./metron-deployment/roles/opentaxii/README.md > > > ./metron-deployment/roles/pcap_replay/README.md > > > ./metron-deployment/roles/sensor-test-mode/README.md > > > ./metron-deployment/vagrant/README.md <== (need file here) > > > ./metron-deployment/vagrant/codelab-platform/README.md > > > ./metron-deployment/vagrant/fastcapa-test-platform/README.md > > > ./metron-deployment/vagrant/full-dev-platform/README.md > > > ./metron-deployment/vagrant/quick-dev-platform/README.md > > > ./metron-platform/README.md > > > ./metron-platform/metron-api/README.md > > > ./metron-platform/metron-common/README.md > > > ./metron-platform/metron-data-management/README.md > > > ./metron-platform/metron-enrichment/README.md > > > ./metron-platform/metron-indexing/README.md > > > ./metron-platform/metron-management/README.md > > > ./metron-platform/metron-parsers/README.md > > > ./metron-platform/metron-pcap-backend/README.md > > > ./metron-sensors/README.md <== (need file here) > > > ./metron-sensors/fastcapa/README.md > > > ./metron-sensors/pycapa/README.md > > > > > > 2. Arrange to run this script as part of the build process, and > commit > > the > > > resulting hierarchy list to someplace in the versioned and branched > > ./site/ > > > subdirectory. > > > > > > 3. Produce a "doc reader" web page that takes in this hierarchy of > .md > > > pages, and presents a LHS doc tree of links, and a main display > area for > > a > > > currently selected file. If we want to get fancy, this page would > also > > > provide: (a) telescoping (collapse/expand) of the doc tree; (b) > floating > > > next/prev/up/home buttons in the display area. > > > > > > #4. Add to this web page a pull-down menu that selects among all > the > > > release versions of Metron, and (if not running in the Apache > site) a > > > SNAPSHOT version for the current filesystem version (for developer > > > preview). Let it re-write the file paths per release version to the > > proper > > > release tag in github. This web page will therefore be > > version-independent. > > > Put it in the asf-site branch of the Apache site, as the new "docs" > > > sub-site from the home web page. Update the list of releases at > each > > > release, or if we want to get fancy, teach it to read the release > tags > > from > > > github. > > > > > > 5. As part of the release process, the release manager (a) assures > the > > > release is tagged in github with a consistent naming convention, > and (b) > > > submits the new hierarchy of links to google search (there's an > api for > > > that). > > > > > > 6. Deprecate the use of cwiki for anything but long-lived > > > demonstrations/tutorials that are unlikely to go obsolete. > > > > > > > > > Do folks feel this would be a good contribution to the visibility, > > > timeliness, and usability of our docs? > > > Is this an adequate solution for the current problems? > > > > > > Thanks, > > > --Matt > > > > > > > > > > > -- > > Nick Allen <[email protected]> > > > -- > > Jon > > Sent from my mobile device > > > > >
