xiazcy commented on code in PR #3576: URL: https://github.com/apache/tinkerpop/pull/3576#discussion_r3670125797
########## docs/src/dev/provider/gremlin-semantics.asciidoc: ########## @@ -519,13 +520,15 @@ The following table maps the notions proposed above to the various `P` operators link:https://github.com/apache/tinkerpop/tree/x.y.z/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/test/features/semantics/Equality.feature[Equality Tests], link:https://github.com/apache/tinkerpop/tree/x.y.z/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/test/features/semantics/Comparability.feature[Comparability Tests] +[llms-summary="Per-step semantic definitions: the precise input/output behavior each Gremlin step must exhibit, used to verify provider implementations for correctness.""] Review Comment: ```suggestion [llms-summary="Per-step semantic definitions: the precise input/output behavior each Gremlin step must exhibit, used to verify provider implementations for correctness."] ``` ########## docs/src/dev/provider/gremlin-semantics.asciidoc: ########## @@ -1617,6 +1647,7 @@ See: link:https://github.com/apache/tinkerpop/tree/x.y.z/gremlin-core/src/main/j link:https://github.com/apache/tinkerpop/tree/x.y.z/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/SubstringLocalStep.java[source (local)], link:https://tinkerpop.apache.org/docs/x.y.z/reference/#substring-step[reference] +[llms-summary="The formal semantics of the toLower() step: returns the lowercase representation of incoming string traverser, or if Scope.local is specified, returns the lowercase representation of each string elemen"] Review Comment: ```suggestion [llms-summary="The formal semantics of the toLower() step: returns the lowercase representation of incoming string traverser, or if Scope.local is specified, returns the lowercase representation of each string element inside the incoming list traverser."] ``` ########## docs/src/recipes/duplicate-edge.asciidoc: ########## @@ -16,6 +16,7 @@ KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. //// +[llms-summary="Recipe: Whether part of a graph maintenance process or for some other analysis need, it is sometimes necessary to detect if there is more than one edge between two vert"] Review Comment: ```suggestion [llms-summary="Recipe: Whether part of a graph maintenance process or for some other analysis need, it is sometimes necessary to detect if there is more than one edge between two vertices."] ``` ########## docs/src/reference/the-traversal.asciidoc: ########## @@ -389,6 +390,7 @@ Spawn steps, which actually yield a traversal, typically match the names of exis * `V()` - Reads vertices from the graph to start the traversal (<<graph-step, example>>). [[graph-traversal-steps]] +[llms-summary="Reference catalog of the individual Gremlin steps (map, filter, sideEffect, branch, and more) with syntax and examples for each.""] Review Comment: ```suggestion [llms-summary="Reference catalog of the individual Gremlin steps (map, filter, sideEffect, branch, and more) with syntax and examples for each."] ``` ########## docs/src/reference/the-traversal.asciidoc: ########## @@ -3283,6 +3358,7 @@ An option to a <<general-steps,`branch()`>> or <<choose-step,`choose()`>>. link:++https://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.html#option(M,org.apache.tinkerpop.gremlin.process.traversal.Traversal)++[`option(Object,Traversal)`], link:++https://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.html#option(org.apache.tinkerpop.gremlin.process.traversal.Traversal)++[`option(Traversal)`] +[llms-summary="The optional()-step (branch/flatMap) returns the result of the specified traversal if it yields a result else it returns the calling element, i.e. the identity("] Review Comment: ```suggestion [llms-summary="The optional()-step (branch/flatMap) returns the result of the specified traversal if it yields a result, else it returns the calling element (i.e. the identity)."] ``` ########## docs/src/reference/the-traversal.asciidoc: ########## @@ -3821,6 +3908,7 @@ link:++https://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gre link:++https://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.html#property(org.apache.tinkerpop.gremlin.structure.VertexProperty.Cardinality,java.lang.Object,java.lang.Object,java.lang.Object...)++[`property(Cardinality, Object, Object, Object...)`], link:++https://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gremlin/structure/VertexProperty.Cardinality.html++[`Cardinality`] +[llms-summary="The propertiesMap()-step yields a Map representation of the properties of an element."] Review Comment: ```suggestion [llms-summary="The propertyMap()-step yields a Map representation of the properties of an element."] ``` ########## docs/src/dev/provider/gremlin-semantics.asciidoc: ########## @@ -1496,6 +1522,7 @@ See: link:https://github.com/apache/tinkerpop/tree/x.y.z/gremlin-core/src/main/j link:https://github.com/apache/tinkerpop/tree/x.y.z/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/ReplaceLocalStep.java[source (local)], link:https://tinkerpop.apache.org/docs/x.y.z/reference/#replace-step[reference] +[llms-summary="The formal semantics of the reverse() step: returns the reverse of the incoming traverser"] Review Comment: ```suggestion [llms-summary="The formal semantics of the reverse() step: returns the reverse of the incoming traverser."] ``` ########## docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/MarkdownConverter.java: ########## @@ -0,0 +1,499 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.tinkerpop.tinkeradoc; + +import org.asciidoctor.ast.Block; +import org.asciidoctor.ast.ContentNode; +import org.asciidoctor.ast.Document; +import org.asciidoctor.ast.List; +import org.asciidoctor.ast.ListItem; +import org.asciidoctor.ast.Section; +import org.asciidoctor.ast.StructuralNode; +import org.asciidoctor.ast.Table; +import org.asciidoctor.ast.Row; +import org.asciidoctor.ast.Cell; +import org.asciidoctor.converter.ConverterFor; +import org.asciidoctor.converter.StringConverter; + +import java.util.Map; + +/** + * A GitHub-Flavored-Markdown backend for the TinkerPop docs. Registered for the {@code tpmarkdown} + * backend, it walks the resolved AST and emits Markdown, so the same executed document renders to + * both HTML (built-in backend) and Markdown from a single Gremlin execution (see + * {@link GremlinExecutionCache}). + * <p> + * Gremlin tab groups arrive as {@code pass} blocks whose content the treeprocessor has already + * rendered to Markdown (via {@link MarkdownTabRenderer}); those pass through verbatim. Prose, + * headings, lists, admonitions, and simple tables are converted here. Complex tables that GFM pipe + * syntax cannot represent fall back to a fenced text rendering. + */ +@ConverterFor("tpmarkdown") +public class MarkdownConverter extends StringConverter { + + /** + * Section attribute holding an author-curated one-line summary for the llms.txt index, e.g. + * {@code [llms-summary="..."]} on a section header. Emitted only as a hidden Markdown comment; + * never rendered into the page body of either backend. + */ + static final String LLMS_SUMMARY_ATTR = "llms-summary"; + + /** + * Section attribute ({@code allow-oversize="true"}) marking a summarized page that may exceed the + * size budget without failing the build's size lint. Emitted as a hidden + * {@code <!-- llms-allow-oversize -->} marker for {@link MarkdownSplitter}; never rendered into + * the page body. Splitting itself is driven solely by {@code llms-summary}. + */ + static final String LLMS_ALLOW_OVERSIZE_ATTR = "allow-oversize"; + + public MarkdownConverter(final String backend, final Map<String, Object> opts) { + super(backend, opts); + } + + @Override + public String convert(final ContentNode node, final String transform, final Map<Object, Object> opts) { + if (node == null) return ""; + final String name = transform != null ? transform : node.getNodeName(); + + // Debug hook: print every (transform,nodeName) the converter is asked to render. + if (System.getProperty("tinkeradoc.md.debug") != null) { + System.out.println("[MD-CONV] transform=" + transform + " nodeName=" + node.getNodeName()); + } + + switch (name) { + case "document": + case "embedded": + return convertDocument((Document) node); + case "section": + return convertSection((Section) node); + case "paragraph": + return blockContent((StructuralNode) node) + "\n"; + case "ulist": + return convertList((List) node, false); + case "olist": + return convertList((List) node, true); + case "dlist": + return convertDescriptionList((org.asciidoctor.ast.DescriptionList) node); + case "colist": + // Callout list: the tab renderers already inline callouts as // (n); a stray colist + // elsewhere renders as a simple numbered list of its item texts. + return convertList((List) node, true); + case "listing": + case "literal": + return convertListing((Block) node); + case "pass": + // Treeprocessor-produced Markdown (e.g. a gremlin tab group) — emit verbatim. + return ((Block) node).getContent() + "\n"; + case "admonition": + return convertAdmonition((Block) node); + case "table": + return convertTable((Table) node); + case "image": + return convertImage((StructuralNode) node); + case "inline_quoted": + return convertInlineQuoted((org.asciidoctor.ast.PhraseNode) node); + case "inline_anchor": + return convertInlineAnchor((org.asciidoctor.ast.PhraseNode) node); + case "inline_break": + return ((org.asciidoctor.ast.PhraseNode) node).getText() + "\n"; + case "inline_callout": + // Callouts inside gremlin tab groups are handled by the tab renderers; a stray + // inline callout elsewhere renders as its number in parentheses. + return "(" + ((org.asciidoctor.ast.PhraseNode) node).getText() + ")"; + case "inline_image": + return convertInlineImage((org.asciidoctor.ast.PhraseNode) node); + case "inline_footnote": + case "inline_indexterm": + return inlineText(node); + default: + return convertFallback(node); + } + } + + /** Renders an {@code inline_quoted} phrase (bold/italic/monospace/etc.) as Markdown. */ + private String convertInlineQuoted(final org.asciidoctor.ast.PhraseNode node) { + final String text = node.getText() == null ? "" : node.getText(); + final String type = node.getType(); + if (type == null) return text; + switch (type) { + case "strong": return "**" + text + "**"; + case "emphasis": return "_" + text + "_"; + case "monospaced": return "`" + text + "`"; + case "mark": return text; + case "superscript": return "<sup>" + text + "</sup>"; + case "subscript": return "<sub>" + text + "</sub>"; + default: return text; + } + } + + /** Renders an {@code inline_anchor} (link/xref) as a Markdown link, or bare text if no target. */ + private String convertInlineAnchor(final org.asciidoctor.ast.PhraseNode node) { + final String text = node.getText(); + final Object target = node.getTarget(); + if (target == null) return text == null ? "" : text; + final String label = (text == null || text.isEmpty()) ? target.toString() : text; + return "[" + label + "](" + target + ")"; + } + + private String inlineText(final ContentNode node) { + if (node instanceof org.asciidoctor.ast.PhraseNode) { + final String t = ((org.asciidoctor.ast.PhraseNode) node).getText(); + return t == null ? "" : t; + } + return ""; + } + + private String convertDocument(final Document doc) { + final String content = doc.getContent() == null ? "" : doc.getContent().toString(); + final String title = doc.getDoctitle(); + final String assembled; + // In a book, the document title is realized as a level-0 section inside the content, so it + // already appears as an H1 there (matching the HTML backend's single <h1 class="sect0">). + // In an article, the title is not part of the content, so emit it as the leading H1. + if (title != null && !title.isEmpty() && !isBook(doc)) { Review Comment: Looks like the llms.txt lists every tutorial twice, with identical descriptions pointing at two different pages: ``` ## tutorials/getting-started - [Getting Started](https://tinkerpop.apache.org/docs/3.7.7-SNAPSHOT/tutorials/getting-started/index.md): A beginner's tutorial introducing graphs, vertices, edges, and basic Gremlin traversals to get started with Apache TinkerPop. - [Getting Started](https://tinkerpop.apache.org/docs/3.7.7-SNAPSHOT/tutorials/getting-started/_getting_started.md): A beginner's tutorial introducing graphs, vertices, edges, and basic Gremlin traversals to get started with Apache TinkerPop. ``` Same pattern for `gremlin-language-variants`, `gremlins-anatomy`, and `the-gremlin-console` — each has both an `index.md` and a `_<title>.md`. The 2nd link looks like a semi-duplication with some weird rendering and content cut-off, are these intentional? If not, Kiro is pointing a fix to here, to have the leading DocTitle/the article path in MarkdownConverter fold the promoted title section into index.md the same way books do, so each tutorial yields a single page. ########## docs/src/dev/provider/gremlin-semantics.asciidoc: ########## @@ -1647,6 +1678,7 @@ See: link:https://github.com/apache/tinkerpop/tree/x.y.z/gremlin-core/src/main/j link:https://github.com/apache/tinkerpop/tree/x.y.z/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/ToLowerLocalStep.java[source (local)], link:https://tinkerpop.apache.org/docs/x.y.z/reference/#toLower-step[reference] +[llms-summary="The formal semantics of the toUpper() step: returns the uppercase representation of incoming string traverser, or if Scope.local is specified, returns the uppercase representation of each string elemen"] Review Comment: ```suggestion [llms-summary="The formal semantics of the toUpper() step: returns the uppercase representation of incoming string traverser, or if Scope.local is specified, returns the uppercase representation of each string element inside the incoming list traverser."] ``` ########## docs/src/recipes/element-existence.asciidoc: ########## @@ -16,6 +16,7 @@ KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. //// +[llms-summary="Recipe: Checking for whether or not a graph element is present in the graph is simple:"] Review Comment: ```suggestion [llms-summary="Recipe: Checking whether a graph element (a vertex or edge) is already present in the graph."] ``` ########## docs/src/reference/the-traversal.asciidoc: ########## @@ -1690,6 +1730,7 @@ continue to surface this condition as their standard remote exception type. link:++https://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.html#fail()++[`fail()`], link:++https://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.html#fail(java.lang.String)++[`fail(String)`] +[llms-summary="The filter() step maps the traverser from the current object to either true or false where the latter will not pass the traverser to the next step in the proces"] Review Comment: ```suggestion [llms-summary="The filter() step maps the traverser from the current object to either true or false where the latter will not pass the traverser to the next step in the proces."] ``` ########## docs/src/recipes/pagination.asciidoc: ########## @@ -16,6 +16,7 @@ KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. //// +[llms-summary="Recipe: In most database applications, it is oftentimes desirable to return discrete blocks of data for a query rather than all of the data that the total results would"] Review Comment: ```suggestion [llms-summary="Recipe: In most database applications, it is oftentimes desirable to return discrete blocks of data for a query rather than all of the data that the total results would contain."] ``` ########## docs/src/reference/the-traversal.asciidoc: ########## @@ -4877,6 +4989,7 @@ g.V().has('name','josh').out('created').values('name').tree('x').cap('x') link:++https://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.html#tree()++[`tree()`], link:++https://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.html#tree(java.lang.String)++[`tree(String)`] +[llms-summary="The trim()-step (map) returns a string with leading and leading whitespace removed."] Review Comment: ```suggestion [llms-summary="The trim()-step (map) returns a string with leading and trailing whitespace removed."] ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
