Awesome to see that spotless works here! Looks like the `flink` and `kafka-connect` folders are missing from `settings.gradle`, there are markdown files in those folders too. We should spot check a few of these changes to make sure the docs are rendered correctly. For example, this change <https://github.com/apache/iceberg/pull/13908/files#diff-691d93e2f9eb8070c1170b38a401e6005a6fd7957f2e67aa3a2a89922046848fR76-R81> might mess up the formatting, I've seen this issue when dealing with pyiceberg's docs.
Best, Kevin Liu On Mon, Aug 25, 2025 at 10:28 AM Fokko Driesprong <fo...@apache.org> wrote: > Hey Manu, > > Thanks for creating the draft PR. If we go for it, we should add the > command to auto-fix the voilations to the top README.md > <https://github.com/apache/iceberg?tab=readme-ov-file#building>. I like > it a lot, curious to learn what others' think. > > Kind regards, > Fokko > > Op ma 25 aug 2025 om 18:20 schreef Manu Zhang <owenzhang1...@gmail.com>: > >> Thanks @Eduard, it's working now after including docker/docs/format/site >> in settings.gradle[1]. All style issues in markdown files under these >> folders can be spotted and fixed by gradlew commands. I agree it's the best >> approach. Please help double check. >> >> >> 1. >> https://github.com/apache/iceberg/pull/13908/files#diff-7f825392aa37acd1cee0c2e7b9bb7366ad6eac64f3e6cdd816e156bcb69d30de >> >> On Mon, Aug 25, 2025 at 3:12 PM Eduard Tudenhöfner < >> etudenhoef...@apache.org> wrote: >> >>> @Manu my guess is that it only found the markdown file that is inside a >>> gradle project folder, whereas other markdown files under *site* or >>> *format* haven't been found. Maybe check whether there's a way to apply >>> the formatting to folders like *site* or *format*. >>> >>> On Sat, Aug 23, 2025 at 5:41 PM Manu Zhang <owenzhang1...@gmail.com> >>> wrote: >>> >>>> Not sure I've configured correctly but the spotless flexmark plugin is >>>> only able to fix one markdown file[1]. Meanwhile, this plugin doesn't >>>> support any options provided by flexmark. >>>> >>>> Hi Fokko, does pre-commit require Python and we need a gradle task to >>>> integrate it? >>>> >>>> 1. https://github.com/apache/iceberg/pull/13908 >>>> >>>> On Fri, Aug 22, 2025 at 12:22 PM Jean-Baptiste Onofré <j...@nanthrax.net> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> Great suggestion Manu ! Indeed, if spotless can support it, for >>>>> consistency, it's probably better to use it. >>>>> >>>>> Regards >>>>> JB >>>>> >>>>> On Thu, Aug 21, 2025 at 6:12 PM Fokko Driesprong <fo...@apache.org> >>>>> wrote: >>>>> > >>>>> > Hey Manu, >>>>> > >>>>> > Thanks for suggesting this, and I strongly support using a linter. >>>>> Recently I noticed that we use different flavors of Markdown in the table, >>>>> and the linter would take care of that. >>>>> > >>>>> > I do have a similar remark as Eduard. If Spotless supports this, I >>>>> think that would be the easiest. Otherwise, I think pre-commit would also >>>>> be a good option within the Java repo as this is also easy to run locally. >>>>> Using pre-commit we can also add other linters (shell, end-of-line, >>>>> detecting debug statements, credential detection, spell-checker, etc). >>>>> > >>>>> > The biggest downside is that we might lose some version history due >>>>> to just reformatting. For example, if you widen a column in a table, I >>>>> think the linter will realign the whole table. However, through GitHub we >>>>> can easily track down the lineage. >>>>> > >>>>> > Kind regards, >>>>> > Fokko >>>>> > >>>>> > Off-topic: At some point, we can replace pre-commit by prek when it >>>>> gets mature enough. As Atwood's law states; Any application that can be >>>>> written in Rust, will eventually be written in Rust (slightly adapted). >>>>> > >>>>> > >>>>> > Op do 21 aug 2025 om 17:59 schreef Eduard Tudenhöfner < >>>>> etudenhoef...@apache.org>: >>>>> >> >>>>> >> We're already using spotless to format Java code and spotless also >>>>> supports markdown files so maybe worth exploring how we could achieve this >>>>> through spotless? >>>>> >> The main advantage would be that people would be able to catch >>>>> linting errors already locally before CI runs. >>>>> >> >>>>> >> On Thu, Aug 21, 2025 at 5:38 PM Manu Zhang <owenzhang1...@gmail.com> >>>>> wrote: >>>>> >>> >>>>> >>> Hi all, >>>>> >>> >>>>> >>> What do you think of adding a GitHub action to lint markdown >>>>> files? It can catch markdown rendering issues early and ensure a >>>>> consistent >>>>> style across markdown files. iceberg-python has already included markdown >>>>> lint[1] in pre-commit hook. (Thanks Fokko for the suggestion!) >>>>> >>> >>>>> >>> I've a draft PR[2] that adds a Docs CI triggered on changes to any >>>>> markdown files. The lint rules are highly customizable via a config >>>>> file[3]. While fixing existing issues spotted by the CI, I'd like to get >>>>> early feedback from the community. >>>>> >>> >>>>> >>> 1. >>>>> https://github.com/apache/iceberg-python/blob/main/.pre-commit-config.yaml#L41 >>>>> >>> 2. https://github.com/apache/iceberg/pull/13826 >>>>> >>> 3. >>>>> https://github.com/manuzhang/iceberg/blob/markdownlint/.markdownlint.jsonc >>>>> >>> >>>>> >>> Regards, >>>>> >>> Manu >>>>> >>>>
