Just a followup thought that's a bit more constructive, maybe we could migrate the README.md's into a site directory and use doxia markdown (example here <https://github.com/larrycai/doxia-markdown-demo>) to generate the site as part of the build to resolve 1 through 3?
On Thu, Jan 12, 2017 at 6:19 PM, Casey Stella <[email protected]> wrote: > So, I do think this would be better than what we currently do. I like a > few things in particular: > > - I don't like the wiki one bit. > - We have a LOT of documentation in the README.md's and it's sometimes > poorly organized > - I like a documentation preprocessing pipeline to be present. For > instance, a major ask is all of the stellar functions in one place. That's > solved by updating an index manually in the READMEs and keeping it in sync > with the annotation. I'd like to make a stellar annotation -> markdown > generator as part of the build and this would be nice for such a task. > > My only concern is that the html generation/viewer seems like a fair > amount of engineering. Are you sure there isn't something easier that we > could conform to? I'm sure we aren't the only project in the world that > has this particular issue. Is there something like a maven site plugin or > something? Just a thought. I'll come back with more :) > > Great ideas! Keep them coming! > > Casey > > On Thu, Jan 12, 2017 at 6:05 PM, Matt Foley <[email protected]> wrote: > >> We currently have three forms of documentation, with the following >> advantages and disadvantages: >> >> || Docs || Pro || Con || >> | CWiki | >> Easy to edit, no special tools required, don't have to be a >> developer to contribute, google and wiki search | >> Not versioned, no review process, distant from the code, obsolete content >> tends to accumulate | >> | Site | >> Versioned and reviewed, only committers can edit, google search | >> Yet another arcane toolset must be learned, only web programmers >> feel comfortable contributing, "asf-site" branch not related to code >> versions, distant from the code, tends to go obsolete due to >> non-maintenance | >> | README.md | >> Versioned and reviewed, only committers can edit, tied to code >> versions, highly local to the code being documented | >> Non-developers don't know about them, may be scared by github, poor >> scoring in google search, no high-level presentation | >> >> Various discussion threads indicate the developer community likes >> README-based docs, and it's easy to see why from the above. I propose this >> extension to the README-based documentation, to address their disadvantages: >> >> 1. Produce a script that gathers the README.md files from all code >> subdirectories into a hierarchical list. The script would have an >> exclusion list for non-user-content, which at this point would consist of >> [site/*, build_utils/*]. The hierarchy would be sorted depth-first. The >> resulting hierarchical list at this time (with six added README files to >> complete the hierarchy) would be: >> >> ./README.md >> ./metron-analytics/README.md <== (need file here) >> ./metron-analytics/metron-maas-service/README.md >> ./metron-analytics/metron-profiler/README.md >> ./metron-analytics/metron-profiler-client/README.md >> ./metron-analytics/metron-statistics/README.md >> ./metron-deployment/README.md >> ./metron-deployment/amazon-ec2/README.md >> ./metron-deployment/packaging/README.md <== (need file here) >> ./metron-deployment/packaging/ambari/README.md <== (need file here) >> ./metron-deployment/packaging/docker/ansible-docker/README.md >> ./metron-deployment/packaging/docker/rpm-docker/README.md >> ./metron-deployment/packer-build/README.md >> ./metron-deployment/roles/ <== (need file here) >> ./metron-deployment/roles/kibana/README.md >> ./metron-deployment/roles/monit/README.md >> ./metron-deployment/roles/opentaxii/README.md >> ./metron-deployment/roles/pcap_replay/README.md >> ./metron-deployment/roles/sensor-test-mode/README.md >> ./metron-deployment/vagrant/README.md <== (need file here) >> ./metron-deployment/vagrant/codelab-platform/README.md >> ./metron-deployment/vagrant/fastcapa-test-platform/README.md >> ./metron-deployment/vagrant/full-dev-platform/README.md >> ./metron-deployment/vagrant/quick-dev-platform/README.md >> ./metron-platform/README.md >> ./metron-platform/metron-api/README.md >> ./metron-platform/metron-common/README.md >> ./metron-platform/metron-data-management/README.md >> ./metron-platform/metron-enrichment/README.md >> ./metron-platform/metron-indexing/README.md >> ./metron-platform/metron-management/README.md >> ./metron-platform/metron-parsers/README.md >> ./metron-platform/metron-pcap-backend/README.md >> ./metron-sensors/README.md <== (need file here) >> ./metron-sensors/fastcapa/README.md >> ./metron-sensors/pycapa/README.md >> >> 2. Arrange to run this script as part of the build process, and commit >> the resulting hierarchy list to someplace in the versioned and branched >> ./site/ subdirectory. >> >> 3. Produce a "doc reader" web page that takes in this hierarchy of .md >> pages, and presents a LHS doc tree of links, and a main display area for a >> currently selected file. If we want to get fancy, this page would also >> provide: (a) telescoping (collapse/expand) of the doc tree; (b) floating >> next/prev/up/home buttons in the display area. >> >> #4. Add to this web page a pull-down menu that selects among all the >> release versions of Metron, and (if not running in the Apache site) a >> SNAPSHOT version for the current filesystem version (for developer >> preview). Let it re-write the file paths per release version to the proper >> release tag in github. This web page will therefore be >> version-independent. Put it in the asf-site branch of the Apache site, as >> the new "docs" sub-site from the home web page. Update the list of >> releases at each release, or if we want to get fancy, teach it to read the >> release tags from github. >> >> 5. As part of the release process, the release manager (a) assures the >> release is tagged in github with a consistent naming convention, and (b) >> submits the new hierarchy of links to google search (there's an api for >> that). >> >> 6. Deprecate the use of cwiki for anything but long-lived >> demonstrations/tutorials that are unlikely to go obsolete. >> >> >> Do folks feel this would be a good contribution to the visibility, >> timeliness, and usability of our docs? >> Is this an adequate solution for the current problems? >> >> Thanks, >> --Matt >> >> >> >
