This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch afd
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
The following commit(s) were added to refs/heads/afd by this push:
new c2d5262add Document agent-friendly docs support and the tinkeradoc
build step
c2d5262add is described below
commit c2d5262add6139b45509d2b8eab3a51bf87c9f5b
Author: Stephen Mallette <[email protected]>
AuthorDate: Mon Jul 27 13:22:08 2026 -0400
Document agent-friendly docs support and the tinkeradoc build step
Add an Agent-Friendly Documentation subsection to the developer docs
covering
the Markdown mirror, llms.txt indexes, the llms-summary page model, the size
budget with allow-oversize, and validation via bin/validate-llms-txt.sh.
Note
the required 'mvn clean install -f docs/tinkeradoc-extension/pom.xml' step,
which a root build does not cover and whose absence silently skips the
Markdown
split. Extend the tinkeradoc README to cover
MarkdownSplitter/LlmsTxtGenerator,
and point doc authors to the llms-summary rules from the contributing guide.
Assisted-by: Claude Code:claude-opus-4-8
---
docs/src/dev/developer/contributing.asciidoc | 4 ++
.../dev/developer/development-environment.asciidoc | 63 +++++++++++++++++++++-
docs/tinkeradoc-extension/README.md | 14 +++++
3 files changed, 80 insertions(+), 1 deletion(-)
diff --git a/docs/src/dev/developer/contributing.asciidoc
b/docs/src/dev/developer/contributing.asciidoc
index 4aa8376a6b..f3c01520ab 100644
--- a/docs/src/dev/developer/contributing.asciidoc
+++ b/docs/src/dev/developer/contributing.asciidoc
@@ -82,6 +82,10 @@ the appropriate branch on which to submit the pull request
so that the documenta
tied to. To view generated documentation locally, read more about environment
configurations in the
<<documentation-environment,Documentation Environment>> and <<documentation,
Contributor Documentation>> sections.
+The reference documentation is also rendered in an agent-friendly form
(Markdown plus an `llms.txt` index). When adding
+a new section that should be discoverable as its own page, give it an
`llms-summary` attribute and keep the resulting
+page within the size budget. See <<agent-friendly-documentation,Agent-Friendly
Documentation>> for the authoring rules.
+
For web site changes, the process is largely the same except that the
documentation system is HTML based instead of
Asciidoc. The content can be found in the source control tree at
link:https://github.com/apache/tinkerpop/tree/master/docs/site[docs/site].
The web site is always published from the `master` branch as it is not bound
to a version, so there is no need to
diff --git a/docs/src/dev/developer/development-environment.asciidoc
b/docs/src/dev/developer/development-environment.asciidoc
index 7081ba0f2f..59a9d3394e 100644
--- a/docs/src/dev/developer/development-environment.asciidoc
+++ b/docs/src/dev/developer/development-environment.asciidoc
@@ -130,7 +130,17 @@ Gremlin Console subprocess, and renders the console output
as tabbed HTML. The o
distributions, installs the required plugins into the console, starts a
Gremlin Server (for the `:remote` examples) and
a Gephi mock, then invokes Maven to run the extension. Javadocs should be
built using Java 11.
-Before generating documentation, build the build all of the dependencies that
the process consumes:
+The `docs/tinkeradoc-extension` project is a standalone Maven project rather
than a module of the root reactor, so a
+`mvn clean install` from the project root does not build it. It must be built
and installed into the local repository
+before generating documentation -- both because the `asciidoctor-maven-plugin`
consumes it as a plugin dependency and
+because `bin/process-docs.sh` runs its compiled classes to produce the
Markdown mirror and `llms.txt` (see
+<<agent-friendly-documentation>>). If the extension has not been built, the
Markdown and `llms.txt` step is silently
+skipped with a `skipping Markdown split` warning. Build it with:
+
+[source,text]
+mvn clean install -f docs/tinkeradoc-extension/pom.xml
+
+Next, build all of the dependencies that the documentation process consumes:
[source,text]
mvn clean install -pl
:gremlin-console,:gremlin-server:spark-gremlin,:neo4j-gremlin -am -DskipTests
-DincludeNeo4j
@@ -182,6 +192,57 @@ section declares a different set, so each conflicting
chapter declares its compl
old preprocessor's per-file plugin juggling; `bin/process-docs.sh` installs
each plugin's dependencies into
`ext/<plugin>/plugin/` (not the shared `lib/`) precisely so they can be
toggled this way.
+[[agent-friendly-documentation]]
+==== Agent-Friendly Documentation
+
+Alongside the HTML, the documentation build produces an agent-friendly
rendering of the reference documentation that
+follows the link:https://agentdocsspec.com/[Agent Friendly Documentation
Specification] (which builds on
+link:https://llmstxt.org/[llms.txt]). This lets AI agents and other automated
consumers discover and read the docs as
+plain Markdown. It is generated by two standalone tools that live in the
`docs/tinkeradoc-extension` project alongside
+the AsciidoctorJ extension: `MarkdownSplitter` and `LlmsTxtGenerator`. Because
these run from the extension's compiled
+classes, the extension must already be built (see the build step at the start
of this section) or `bin/process-docs.sh`
+prints `skipping Markdown split` and produces no Markdown output.
+
+The build emits three kinds of artifact under `target/docs`:
+
+* a *Markdown mirror* -- a `.md` copy of each page beside its `.html`, so
every page is reachable at both
+`.../<path>.html` and `.../<path>.md`;
+* a per-version *discovery index* at `docs/<version>/llms.txt`; and
+* a site-root *discovery index* at `/llms.txt`.
+
+Pages are summary-driven: a section becomes its own Markdown page if and only
if it carries an `llms-summary` attribute,
+whose value is the one-line description used in `llms.txt`. Authors add these
directly in the AsciiDoc source:
+
+[source,asciidoc]
+----
+[[looping]]
+[llms-summary="Recipe: complex looping statements using the repeat() step."]
+== Looping
+----
+
+Each generated page must stay under a 50,000-character budget.
`bin/process-docs.sh` runs the splitter in `--strict`
+mode, which fails the build when a page exceeds the budget and is not
explicitly allowed to. Resolve a violation either
+by adding an `llms-summary` to a subsection so the page is split into smaller
pages, or -- when a page must remain whole
+-- by marking it `allow-oversize="true"`:
+
+[source,asciidoc]
+----
+[llms-summary="A collection of practical Gremlin traversal patterns and
solutions to common graph problems.",allow-oversize="true"]
+----
+
+When the docs are published, `bin/publish-docs.sh` regenerates both `llms.txt`
files with absolute URLs (the spec's
+link-resolution checks only recognize full `http(s)://` links); the local
build intentionally uses relative links for
+offline inspection.
+
+The result can be validated against the specification's own tooling with
`bin/validate-llms-txt.sh`, which runs
+`npx afdocs check` (requires Node.js >= 22). It can check the published site
or a local build:
+
+[source,text]
+----
+bin/validate-llms-txt.sh # check the published current docs
+bin/validate-llms-txt.sh -v local # check the local build under target/docs
+----
+
To generate the web site locally, there is no need for any of the above
infrastructure. Site generation is a simple
shell script:
diff --git a/docs/tinkeradoc-extension/README.md
b/docs/tinkeradoc-extension/README.md
index 6903e2bfe7..f54b121cdc 100644
--- a/docs/tinkeradoc-extension/README.md
+++ b/docs/tinkeradoc-extension/README.md
@@ -39,6 +39,20 @@ The extension registers with AsciidoctorJ through the SPI
(`GremlinDocsExtension
and dismissing `Display stack trace?` prompts. `ConsoleRestartHandler` and
`PluginDirectoryRestartHandler` restart
that subprocess when a book needs a different plugin set.
+The project also houses two standalone command-line tools that produce the
agent-friendly rendering of the docs (the
+[Agent Friendly Documentation Specification](https://agentdocsspec.com/),
built on [llms.txt](https://llmstxt.org/)):
+
+- **`MarkdownSplitter`** splits each book's Markdown output into agent-sized
pages. A section becomes its own page iff
+ it carries an `llms-summary` attribute; its `--strict` mode fails the build
when a page exceeds the 50,000-character
+ budget and is not marked `allow-oversize="true"`.
+- **`LlmsTxtGenerator`** scans the split pages and writes the `llms.txt`
discovery index over them, optionally with an
+ absolute-URL prefix for publishing.
+
+These run from the extension's compiled classes and are driven by
`bin/process-docs.sh` (and `bin/publish-docs.sh` for
+publishing); they are not invoked directly. See the
+[Agent-Friendly
Documentation](../src/dev/developer/development-environment.asciidoc) section
of the developer docs for
+the authoring rules (`llms-summary`, the size budget, `allow-oversize`) and
validation via `bin/validate-llms-txt.sh`.
+
## Block Syntax
A `gremlin-groovy` block takes an optional graph name as its second positional
attribute, which seeds `graph` and `g`
