TaiJuWu,

Hugo has a Docker image which makes it very easy to run locally, and I
expect many developers have Docker installed already. I'm not totally sure,
but I don't think this would require any build support (Gradle or Hugo/Go).
I believe you would simply mount the docs directory in the docker container
and it would work. This is the workflow I expect to use since I don't plan
on managing a Go installation locally.

-David

On Tue, Mar 11, 2025 at 2:36 AM TaiJu Wu <tjwu1...@gmail.com> wrote:

> Hi Harish,
>
> Thanks for this KIP.
>
> Is it possible to add dev website from trunk for developers?
> It could help developer to browser the web without local build and check
> recent documentation change quickly.
> I think it can improve our documentation a lot since developers can easy to
> check it.
>
> Best,
> TaiJuWu
>
>
> Harish Vishwanath <harish.shas...@gmail.com> 於 2025年3月11日 週二 上午11:53寫道:
>
> > Thanks Chia-Ping. I have updated
> > <
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1133%3A+AK+Documentation+and+Website+in+Markdown#KIP1133:AKDocumentationandWebsiteinMarkdown-DevelopmentWorkflow
> > >
> > the KIP to include the above information.
> >
> > Regards,
> > Harish
> >
> >
> > On Sun, Mar 9, 2025 at 10:09 PM Chia-Ping Tsai <chia7...@gmail.com>
> wrote:
> >
> > > hi Harish
> > >
> > > > Hugo supports live-reload. The docsy theme has dependencies on golang
> > and
> > > a
> > > couple of nodeJS libraries (autoprefixer, postcss etc.,) and as such it
> > > will be simpler to use docker based workflows to avoid having to
> install
> > > any dependencies locally for development (ref:
> > >
> > >
> >
> https://github.com/hvishwanath/kafka-site-md/blob/main/README.md#local-development-server
> > > ). We can set up CI pipelines to build and push production images
> serving
> > > the site with nginx.
> > >
> > > Thank you for the explanation. Would you please consider including this
> > > information in the KIP? Documenting the new workflow for reviewing
> > > documentation would be very beneficial for future reference. In fact,
> we
> > > recently resolved a bug where the local site's behavior differed from
> the
> > > deployed version on Apache ...
> > >
> > > Best,
> > > Chia-Ping
> > >
> > > Harish Vishwanath <harish.shas...@gmail.com> 於 2025年3月10日 週一
> 下午12:51寫道:
> > >
> > > > Thank you Chia-Ping.
> > > >
> > > > Conversion is mostly automated (
> https://github.com/hvishwanath/ak2md
> > )
> > > > and
> > > > as such the prototype
> > > > <https://kafka-site-md-711970345036.us-central1.run.app/> supports
> all
> > > > previous versions. However, not having to support all previous
> versions
> > > > will definitely be simpler.
> > > >
> > > > Hugo supports live-reload. The docsy theme has dependencies on golang
> > > and a
> > > > couple of nodeJS libraries (autoprefixer, postcss etc.,) and as such
> it
> > > > will be simpler to use docker based workflows to avoid having to
> > install
> > > > any dependencies locally for development (ref:
> > > >
> > > >
> > >
> >
> https://github.com/hvishwanath/kafka-site-md/blob/main/README.md#local-development-server
> > > > ). We can set up CI pipelines to build and push production images
> > serving
> > > > the site with nginx.
> > > >
> > > > Regards,
> > > > Harish
> > > >
> > > >
> > > > On Sun, Mar 9, 2025 at 9:08 PM Chia-Ping Tsai <chia7...@gmail.com>
> > > wrote:
> > > >
> > > > > hi Harish
> > > > >
> > > > > I have to say, this KIP is one of the best ideas I've seen
> recently,
> > > and
> > > > I
> > > > > agree with Matthias that we don't need to translate any older
> > > versions. I
> > > > > truly hope this KIP can be included in 4.0.
> > > > >
> > > > > I have a quick question: what is the recommended procedure for
> > testing
> > > > the
> > > > > site locally after migrating to Markdown? Previously, we used a
> > > container
> > > > > to host the site locally for previewing changes. Could you please
> > > provide
> > > > > guidance on the updated workflow?
> > > > >
> > > > > Best,
> > > > > Chia-Ping
> > > > >
> > > > > Harish Vishwanath <harish.shas...@gmail.com> 於 2025年3月10日 週一
> > > 上午11:53寫道:
> > > > >
> > > > > > Hello all
> > > > > >
> > > > > > Sorry for my delayed response. Thanks for your feedback and
> > comments.
> > > > > > Please see below:
> > > > > >
> > > > > > > Do you plan to have a development version of the website online
> > in
> > > > > > parallel to the old HTML version?
> > > > > > Yes. The current prototype which supports all versions until AK
> 3.9
> > > is
> > > > > > running at
> https://kafka-site-md-711970345036.us-central1.run.app/
> > ,
> > > > the
> > > > > > prototype
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1133%3A+AK+Documentation+and+Website+in+Markdown#KIP1133:AKDocumentationandWebsiteinMarkdown-Prototype
> > > > > > >
> > > > > > section of the KIP has more details. We should be able to host
> this
> > > in
> > > > > ASF
> > > > > > infra. Further, as described in the KIP
> > > > > >
> > > > > > >I understand that locally with the markdown files and an
> markdown
> > > > > editor,
> > > > > > we see the rough structure of the docs (which is already an
> > > > improvement),
> > > > > > but we will not see the end product, because for that we again
> > need a
> > > > web
> > > > > > server.
> > > > > > Hugo supports live reload making local development easier (ref:
> > > > > >
> > https://github.com/hvishwanath/kafka-site-md/blob/main/Makefile#L25)
> > > > > >
> > > > > >
> > > > > > [MM1] Yes., I have ensured that all existing auto generated
> > > > > documentations
> > > > > > (html files under javadoc, generated folders) are available at
> the
> > > same
> > > > > > relative location as they are in the current website. You can see
> > > this
> > > > in
> > > > > > the prototype.
> > > > > > For example:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://kafka.apache.org/39/javadoc/org/apache/kafka/clients/consumer/ConsumerConfig.html#main(java.lang.String%5B%5D)
> > > > > > is
> > > > > > available at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://kafka-site-md-711970345036.us-central1.run.app/39/javadoc/org/apache/kafka/clients/consumer/ConsumerConfig.html#main(java.lang.String%5B%5D)
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://kafka-site-md-711970345036.us-central1.run.app/39/javadoc/org/apache/kafka/clients/consumer/ConsumerConfig.html#main(java.lang.String%5B%5D)
> > > > > > >
> > > > > > Furthermore, for production, we can serve this with a full blown
> > web
> > > > > server
> > > > > > such as nginx allowing us to set up any required redirects.
> > > > > >
> > > > > > [MM2] I recommend that we leave any auto generated documentation
> to
> > > > > > continue to be in HTML format. We can serve them as static assets
> > > (ref:
> > > > > > https://github.com/hvishwanath/kafka-site-md/tree/main/static)
> > > > > >
> > > > > > >I am also in favor of having easy access to `trunk / SNAPSHOT`
> > > version
> > > > > of
> > > > > > the docs if possible.
> > > > > > Yes, this is definitely possible.
> > > > > >
> > > > > >
> > > > > > Regards,
> > > > > > Harish
> > > > > >
> > > > > >
> > > > > > On Sat, Mar 1, 2025 at 3:01 PM Matthias J. Sax <mj...@apache.org
> >
> > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I am totally in favor to make this happen, and I was
> disappointed
> > > > each
> > > > > > > time in the past, when it was proposed but we did not do it.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Given that it is a huge effort, I am wondering if it would be
> > > > > sufficient
> > > > > > > to just to this change for 4.0 release though, and not
> translate
> > > any
> > > > > > > older versions at all?
> > > > > > >
> > > > > > >
> > > > > > > For including the docs in the release, I am actually wondering
> > what
> > > > the
> > > > > > > value is? Does anybody actually use these artifact? Of course,
> we
> > > > need
> > > > > > > to publish JavaDocs, but the content of the web page seems to
> > have
> > > > very
> > > > > > > low value to the community? -- I did also check ASF guidelines
> > > about
> > > > it
> > > > > > > and the webpage [1] say:
> > > > > > >
> > > > > > > > This material may include documentation concerning the
> release
> > > > [...]
> > > > > > >
> > > > > > > So it is optional, but not required to publish docs artifacts
> > from
> > > my
> > > > > > > understanding. I have no objections to keep publishing the web
> > page
> > > > > > > docs, just saying, if the effort is too large and the value to
> > low,
> > > > we
> > > > > > > could consider just dropping it.
> > > > > > >
> > > > > > >
> > > > > > > I am also in favor of having easy access to `trunk / SNAPSHOT`
> > > > version
> > > > > > > of the docs if possible. Some other ASF project do this
> already,
> > > for
> > > > > > > example Flink [2]. It's much easier to work with, and allows to
> > > spot
> > > > > > > issues during development. While it does not help on a PR
> > directly,
> > > > it
> > > > > > > is still a good way to verify a PR after it was merged, so any
> > > errors
> > > > > > > can be detected and corrected more easily.
> > > > > > >
> > > > > > >
> > > > > > > Also agree to the other raised points about generated content
> and
> > > > > > > permalinks. For permalinks, if effort is reasonable, I would
> > rather
> > > > try
> > > > > > > to redirect as many as possible.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > -Matthias
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://www.apache.org/legal/release-policy.html#distribute-other-artifacts
> > > > > > > [2] https://nightlies.apache.org/flink/flink-docs-master/
> > > > > > >
> > > > > > >
> > > > > > > On 2/25/25 8:43 AM, David Arthur wrote:
> > > > > > > > MM1: I suspect Hugo has some way of doing url redirection,
> but
> > at
> > > > the
> > > > > > > very
> > > > > > > > least we can add redirects in our htaccess file
> > > > > > > > https://github.com/apache/kafka-site/blob/asf-site/.htaccess
> > > > > > > >
> > > > > > > > Excluding the "front matter" (intro, quickstart, powered by,
> > > etc) I
> > > > > > think
> > > > > > > > the main links we need to worry about are the configuration
> > docs
> > > > > pages.
> > > > > > > In
> > > > > > > > the demo, it looks like we are using the same anchors for
> > > specific
> > > > > > config
> > > > > > > > sections.
> > > > > > > >
> > > > > > > > E.g.,
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://kafka-site-md-711970345036.us-central1.run.app/39/configuration/configuration/#topicconfigs_compression.gzip.level
> > > > > > > > is the same content as
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://kafka.apache.org/documentation/#topicconfigs_compression.zstd.level
> > > > > > > >
> > > > > > > > So maybe it shouldn't be too hard?
> > > > > > > >
> > > > > > > > Personally, I think the only links we should consider to be
> > > > > > "permalinks"
> > > > > > > > are the front matter and the documentation pages with the
> > version
> > > > in
> > > > > > the
> > > > > > > > URL. The unversioned documentation pages (like
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://kafka.apache.org/documentation/#topicconfigs_compression.zstd.level
> > > > > > > )
> > > > > > > > can change from release to release, so they are not expected
> to
> > > be
> > > > > > > > permanent.
> > > > > > > >
> > > > > > > > Do we have any data on inbound links to the site? Maybe from
> > > Google
> > > > > > > > Analytics or similar?
> > > > > > > >
> > > > > > > > -David A
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, Feb 24, 2025 at 10:17 AM Mickael Maison <
> > > > > > > mickael.mai...@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Hi Harish,
> > > > > > > >>
> > > > > > > >> This is definitively something we want to do!
> > > > > > > >>
> > > > > > > >> MM1. One aspect to keep in mind in terms of compatibility is
> > > > whether
> > > > > > > >> existing links will still work. Thousands of online
> resources
> > > link
> > > > > to
> > > > > > > >> the Apache Kafka website. Is it possible to keep all links
> > > > working?
> > > > > > > >>
> > > > > > > >> MM2. A bunch of sections are generated. To do so we have a
> > > number
> > > > of
> > > > > > > >> classes in the public API (for example 0, 1) with toHtml()
> or
> > > > main()
> > > > > > > >> methods that output HTML. Should we update these to output
> > > > Markdown
> > > > > > > >> directly instead?
> > > > > > > >>
> > > > > > > >> Converting the website has been discussed several times, I'd
> > > > suggest
> > > > > > > >> checking at least
> > > > https://issues.apache.org/jira/browse/KAFKA-2967
> > > > > to
> > > > > > > >> see what was previously said.
> > > > > > > >>
> > > > > > > >> 0:
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://kafka.apache.org/39/javadoc/org/apache/kafka/common/config/ConfigDef.html#toHtml()
> > > > > > > >> 1:
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://kafka.apache.org/39/javadoc/org/apache/kafka/clients/consumer/ConsumerConfig.html#main(java.lang.String%5B%5D)
> > > > > > > >>
> > > > > > > >> Thanks,
> > > > > > > >> Mickael
> > > > > > > >>
> > > > > > > >> On Mon, Feb 24, 2025 at 3:33 PM Bruno Cadonna <
> > > cado...@apache.org
> > > > >
> > > > > > > wrote:
> > > > > > > >>>
> > > > > > > >>> Hi Harish,
> > > > > > > >>>
> > > > > > > >>> Thanks for the KIP! Great initiative!
> > > > > > > >>>
> > > > > > > >>> Do you plan to have a development version of the website
> > online
> > > > in
> > > > > > > >>> parallel to the old HTML version?
> > > > > > > >>>
> > > > > > > >>> I understand that locally with the markdown files and an
> > > markdown
> > > > > > > >>> editor, we see the rough structure of the docs (which is
> > > already
> > > > an
> > > > > > > >>> improvement), but we will not see the end product, because
> > for
> > > > that
> > > > > > we
> > > > > > > >>> again need a web server.
> > > > > > > >>>
> > > > > > > >>> I assume that after you migrated all docs to markdown, we
> > want
> > > to
> > > > > see
> > > > > > > >>> the complete website, review the content, and make some
> > > > > refinements.
> > > > > > > >>>
> > > > > > > >>> Best,
> > > > > > > >>> Bruno
> > > > > > > >>>
> > > > > > > >>> On 23.02.25 07:52, Harish Vishwanath wrote:
> > > > > > > >>>> Thank you David. Appreciate your feedback. My comments
> > below:
> > > > > > > >>>>
> > > > > > > >>>> DA1) I think this is a very good suggestion. While
> building
> > > the
> > > > > > > >> prototype,
> > > > > > > >>>> I did spend additional effort for older versions of the
> doc
> > > > since
> > > > > > > >> there are
> > > > > > > >>>> fairly significant differences in the structure and format
> > of
> > > > the
> > > > > > > >> documents
> > > > > > > >>>> as compared to 3.x versions. I have written the automation
> > > which
> > > > > can
> > > > > > > do
> > > > > > > >>>> this, but I do think hosting the older versions (<3.x) in
> > its
> > > > > > current
> > > > > > > >>>> format and moving the rest to markdown will reduce the
> > overall
> > > > > > > >>>> testing/validation effort. That said, if we foresee making
> > > > > > relatively
> > > > > > > >>>> frequent updates to older versions, it might be worth
> > > converting
> > > > > > > >> everything
> > > > > > > >>>> to markdown.
> > > > > > > >>>>
> > > > > > > >>>> DA2) My hunch is that this can be quite involved.
> Rendering
> > a
> > > > > usable
> > > > > > > >> HTML
> > > > > > > >>>> version requires all the supporting artifacts (css, js,
> > > images,
> > > > > hugo
> > > > > > > >>>> shortcodes, partials etc.,) which will only be present in
> > the
> > > > > > website
> > > > > > > >>>> documentation. Further, the hyper links themselves will
> not
> > > work
> > > > > > > >> without
> > > > > > > >>>> a webserver. To serve a directory structure based HTML, it
> > > will
> > > > > > > >> require us
> > > > > > > >>>> to have different templates for these docs where
> > dependencies
> > > on
> > > > > the
> > > > > > > >>>> artifacts I mentioned above will need to be removed, links
> > > will
> > > > > need
> > > > > > > >> to be
> > > > > > > >>>> converted to use relative directory locations etc.,
> > > > > > > >>>>
> > > > > > > >>>> DA3) This is definitely possible. Using tools such as
> > `pandoc`
> > > > we
> > > > > > > >> should be
> > > > > > > >>>> able to render plain text versions of markdown files. For
> > ex,
> > > > the
> > > > > > > below
> > > > > > > >>>> would create a plain text version of all docs for 39/
> > branch.,
> > > > > which
> > > > > > > >> can be
> > > > > > > >>>> packaged with the binary distribution.
> > > > > > > >>>>
> > > > > > > >>>> find docs/ -name "*.md" -type f | while read -r file; do
> > > > > > > >>>>       dir=$(dirname "$file")
> > > > > > > >>>>       filename=$(basename "$file" .md)
> > > > > > > >>>>       mkdir -p "39-ascii/$dir"
> > > > > > > >>>>       pandoc -f markdown -t plain "$file" -o
> > > > > > > >> "39-ascii/$dir/$filename.txt"
> > > > > > > >>>> done
> > > > > > > >>>>
> > > > > > > >>>> DA4) Makes sense. I updated the wiki to reflect this.
> > > > > > > >>>>
> > > > > > > >>>>
> > > > > > > >>>> Regards,
> > > > > > > >>>> Harish
> > > > > > > >>>>
> > > > > > > >>>>
> > > > > > > >>>> On Fri, Feb 21, 2025 at 8:56 AM David Arthur <
> > > mum...@gmail.com>
> > > > > > > wrote:
> > > > > > > >>>>
> > > > > > > >>>>> Thanks for the KIP, Harish!
> > > > > > > >>>>>
> > > > > > > >>>>> I am +100 just on the title alone :)
> > > > > > > >>>>>
> > > > > > > >>>>> The motivations in the KIP are sound. All of these are
> very
> > > > real
> > > > > > > >> annoyances
> > > > > > > >>>>> when it comes to maintaining our docs. I suspect we have
> > not
> > > > done
> > > > > > > >> anything
> > > > > > > >>>>> to improve them since the system we have worked well
> enough
> > > in
> > > > > the
> > > > > > > >> past. A
> > > > > > > >>>>> better way to say that is: it hasn't been bad enough to
> > > > motivate
> > > > > us
> > > > > > > to
> > > > > > > >>>>> change it.
> > > > > > > >>>>>
> > > > > > > >>>>> With efforts like this, there is always some question
> about
> > > > which
> > > > > > > >> tool or
> > > > > > > >>>>> method we should use. I like that your proposal includes
> > > these
> > > > > > > >> specifics as
> > > > > > > >>>>> well as a complete and functional demo.
> > > > > > > >>>>>
> > > > > > > >>>>> Few questions:
> > > > > > > >>>>>
> > > > > > > >>>>> DA1) Do we really need to convert all of our docs to the
> > new
> > > > > thing?
> > > > > > > It
> > > > > > > >>>>> might be reasonable to draw a line somewhere (maybe 3.0?)
> > and
> > > > > have
> > > > > > > >> docs
> > > > > > > >>>>> prior to that be hosted elsewhere (kafka.apache.org/old
> or
> > > > > > > >> something).
> > > > > > > >>>>> This
> > > > > > > >>>>> may or may not be more work than just converting
> > everything.
> > > > > WDYT?
> > > > > > > >>>>>
> > > > > > > >>>>> DA2) For our packaged docs (those that go in the binaries
> > we
> > > > > > > >> distribute),
> > > > > > > >>>>> do you think it's possible to render HTML that can be
> > viewed
> > > > > > without
> > > > > > > a
> > > > > > > >>>>> webserver? Currently our packaged docs are kind of
> useless
> > > IMO.
> > > > > It
> > > > > > > >> would be
> > > > > > > >>>>> better if they could be viewed without any extra effort
> by
> > an
> > > > > > > >> operator.
> > > > > > > >>>>>
> > > > > > > >>>>> DA3) As a possible alternative to DA2, could we render
> the
> > > > > markdown
> > > > > > > >> docs as
> > > > > > > >>>>> ascii text files? This is arguably better for operators
> > since
> > > > > they
> > > > > > > >> don't
> > > > > > > >>>>> always have a GUI or web browser handy.
> > > > > > > >>>>>
> > > > > > > >>>>> DA4) Just a nitpick, could you reduce the "Sample
> directory
> > > > > layout"
> > > > > > > >> down to
> > > > > > > >>>>> one or two examples? It's difficult to see the versioned
> vs
> > > > > > > >> non-versioned
> > > > > > > >>>>> docs in the current block
> > > > > > > >>>>>
> > > > > > > >>>>> Thanks!
> > > > > > > >>>>> David A
> > > > > > > >>>>>
> > > > > > > >>>>>
> > > > > > > >>>>> On Sun, Feb 16, 2025 at 2:10 PM Harish Vishwanath <
> > > > > > > >>>>> harish.shas...@gmail.com>
> > > > > > > >>>>> wrote:
> > > > > > > >>>>>
> > > > > > > >>>>>> Hello,
> > > > > > > >>>>>>
> > > > > > > >>>>>> Following up from my previous email regarding moving AK
> > > > > > > >> documentation to
> > > > > > > >>>>>> markdown, I wrote KIP-1133
> > > > > > > >>>>>> <
> > > > > > > >>>>>>
> > > > > > > >>>>>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1133%3A+AK+Documentation+and+Website+in+Markdown
> > > > > > > >>>>>>>
> > > > > > > >>>>>> detailing
> > > > > > > >>>>>> the proposal and would like to start discussions. Please
> > > take
> > > > a
> > > > > > > look.
> > > > > > > >>>>>>
> > > > > > > >>>>>> Regards,
> > > > > > > >>>>>> Harish
> > > > > > > >>>>>>
> > > > > > > >>>>>
> > > > > > > >>>>>
> > > > > > > >>>>> --
> > > > > > > >>>>> David Arthur
> > > > > > > >>>>>
> > > > > > > >>>>
> > > > > > > >>>
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>


-- 
David Arthur

Reply via email to