Thanks for working on this! +1 to adding a linter for markdown files.
Ideally we can reuse the tooling in the java/spotless ecosystem, but it looks like `flexmark` only supports a limited set of features as shown by your PR #13908. https://github.com/diffplug/spotless/tree/main/plugin-gradle#flexmark I like pre commit. It's in Python and does not support running hooks in java (yet), https://pre-commit.com/#overriding-language-version We currently use Python for docs/, sites/, and openapi/. We can consider using Python for pre commit and linting. I did find this pre-commit-java repo ( https://github.com/gherynos/pre-commit-java) but note that it's using the GPLv3 license, which is not allowed for Apache projects, https://www.apache.org/licenses/GPL-compatibility.html But we can take inspiration from this project; it uses Docker to run the hooks. We can do the same if we really want to avoid taking on the Python dependency. Best, Kevin Liu On Sat, Aug 23, 2025 at 8:40 AM Manu Zhang <owenzhang1...@gmail.com> wrote: > Not sure I've configured correctly but the spotless flexmark plugin is > only able to fix one markdown file[1]. Meanwhile, this plugin doesn't > support any options provided by flexmark. > > Hi Fokko, does pre-commit require Python and we need a gradle task to > integrate it? > > 1. https://github.com/apache/iceberg/pull/13908 > > On Fri, Aug 22, 2025 at 12:22 PM Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: > >> Hi, >> >> Great suggestion Manu ! Indeed, if spotless can support it, for >> consistency, it's probably better to use it. >> >> Regards >> JB >> >> On Thu, Aug 21, 2025 at 6:12 PM Fokko Driesprong <fo...@apache.org> >> wrote: >> > >> > Hey Manu, >> > >> > Thanks for suggesting this, and I strongly support using a linter. >> Recently I noticed that we use different flavors of Markdown in the table, >> and the linter would take care of that. >> > >> > I do have a similar remark as Eduard. If Spotless supports this, I >> think that would be the easiest. Otherwise, I think pre-commit would also >> be a good option within the Java repo as this is also easy to run locally. >> Using pre-commit we can also add other linters (shell, end-of-line, >> detecting debug statements, credential detection, spell-checker, etc). >> > >> > The biggest downside is that we might lose some version history due to >> just reformatting. For example, if you widen a column in a table, I think >> the linter will realign the whole table. However, through GitHub we can >> easily track down the lineage. >> > >> > Kind regards, >> > Fokko >> > >> > Off-topic: At some point, we can replace pre-commit by prek when it >> gets mature enough. As Atwood's law states; Any application that can be >> written in Rust, will eventually be written in Rust (slightly adapted). >> > >> > >> > Op do 21 aug 2025 om 17:59 schreef Eduard Tudenhöfner < >> etudenhoef...@apache.org>: >> >> >> >> We're already using spotless to format Java code and spotless also >> supports markdown files so maybe worth exploring how we could achieve this >> through spotless? >> >> The main advantage would be that people would be able to catch linting >> errors already locally before CI runs. >> >> >> >> On Thu, Aug 21, 2025 at 5:38 PM Manu Zhang <owenzhang1...@gmail.com> >> wrote: >> >>> >> >>> Hi all, >> >>> >> >>> What do you think of adding a GitHub action to lint markdown files? >> It can catch markdown rendering issues early and ensure a consistent style >> across markdown files. iceberg-python has already included markdown lint[1] >> in pre-commit hook. (Thanks Fokko for the suggestion!) >> >>> >> >>> I've a draft PR[2] that adds a Docs CI triggered on changes to any >> markdown files. The lint rules are highly customizable via a config >> file[3]. While fixing existing issues spotted by the CI, I'd like to get >> early feedback from the community. >> >>> >> >>> 1. >> https://github.com/apache/iceberg-python/blob/main/.pre-commit-config.yaml#L41 >> >>> 2. https://github.com/apache/iceberg/pull/13826 >> >>> 3. >> https://github.com/manuzhang/iceberg/blob/markdownlint/.markdownlint.jsonc >> >>> >> >>> Regards, >> >>> Manu >> >
