I'd like to share some general documentation nitpicks in the hopes that we can address them as a community.
None of these issues are a big problem -- again, they are nitpicks! -- but they do have a notable impact on our documentation when considered all together. I will submit some patches to chip away at these problems. Hopefully, some of you reading this will join me. Agent instructions should not compensate for documentation deficiencies We should instead correct the deficiencies "upstream" in the main documentation so that both humans and AI agents can benefit. Some instructions in AGENTS.md <https://github.com/apache/spark/blob/1226c5ccc1cf87362136d4e10332e35e80640cd9/AGENTS.md> look like documentation smells <https://c2.com/xp/CodeSmell.html> to me. For example, the guidance on how to select a test base class or how to run tests seems like it should already be captured well for humans, no? And if it isn't, then shouldn't we fix the documentation "for humans" first and just point agents to it? Likewise, some of the guidance under "Pull Request Workflow" seems like it should instead live in our general contributing guide <https://spark.apache.org/contributing.html>. The instructions on how to label PRs with components, in particular, seem more up-to-date for agents than they are for humans! A page should only have one title That means the rendered HTML on any given page should only contain one h1 tag <https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/Heading_Elements#avoid_using_multiple_h1_elements_on_one_page>. This is something we get wrong on many pages. We often use # as a section heading, which translates to h1. We should instead use ##. Another way we get this wrong is by adding a title to pages that already have a title specified in the YAML frontmatter <https://github.com/apache/spark/blob/1226c5ccc1cf87362136d4e10332e35e80640cd9/docs/hardware-provisioning.md?plain=1#L3>. Jekyll generates a title automatically using this information, so we don't need to manually add a title on top of that. I don't plan to try fixing all of our headings, but I would like us to be aware of this issue so we stop adding to it. I am looking into adding a Markdown linter that can help us with this. Don't use Liquid tags for syntax highlighting Instead, use the Markdown code fences that we're all already familiar with. For some reason, perhaps simply due to inertia, we have many uses of Liquid tags for syntax highlighting <https://github.com/apache/spark/blob/1226c5ccc1cf87362136d4e10332e35e80640cd9/docs/mllib-data-types.md?plain=1#L61-L74>. In addition to being more verbose than Markdown code fences, Liquid tags confuse most editors and IDEs. Note how GitHub interprets the Python code comments in that example <https://github.com/apache/spark/blob/1226c5ccc1cf87362136d4e10332e35e80640cd9/docs/mllib-data-types.md?plain=1#L61-L74> as headings because it doesn't recognize Liquid tags. My IDE does the same thing.
