Hi,
I am totally in favor to make this happen, and I was disappointed each
time in the past, when it was proposed but we did not do it.
Given that it is a huge effort, I am wondering if it would be sufficient
to just to this change for 4.0 release though, and not translate any
older versions at all?
For including the docs in the release, I am actually wondering what the
value is? Does anybody actually use these artifact? Of course, we need
to publish JavaDocs, but the content of the web page seems to have very
low value to the community? -- I did also check ASF guidelines about it
and the webpage [1] say:
This material may include documentation concerning the release [...]
So it is optional, but not required to publish docs artifacts from my
understanding. I have no objections to keep publishing the web page
docs, just saying, if the effort is too large and the value to low, we
could consider just dropping it.
I am also in favor of having easy access to `trunk / SNAPSHOT` version
of the docs if possible. Some other ASF project do this already, for
example Flink [2]. It's much easier to work with, and allows to spot
issues during development. While it does not help on a PR directly, it
is still a good way to verify a PR after it was merged, so any errors
can be detected and corrected more easily.
Also agree to the other raised points about generated content and
permalinks. For permalinks, if effort is reasonable, I would rather try
to redirect as many as possible.
-Matthias
[1]
https://www.apache.org/legal/release-policy.html#distribute-other-artifacts
[2] https://nightlies.apache.org/flink/flink-docs-master/
On 2/25/25 8:43 AM, David Arthur wrote:
MM1: I suspect Hugo has some way of doing url redirection, but at the very
least we can add redirects in our htaccess file
https://github.com/apache/kafka-site/blob/asf-site/.htaccess
Excluding the "front matter" (intro, quickstart, powered by, etc) I think
the main links we need to worry about are the configuration docs pages. In
the demo, it looks like we are using the same anchors for specific config
sections.
E.g.,
https://kafka-site-md-711970345036.us-central1.run.app/39/configuration/configuration/#topicconfigs_compression.gzip.level
is the same content as
https://kafka.apache.org/documentation/#topicconfigs_compression.zstd.level
So maybe it shouldn't be too hard?
Personally, I think the only links we should consider to be "permalinks"
are the front matter and the documentation pages with the version in the
URL. The unversioned documentation pages (like
https://kafka.apache.org/documentation/#topicconfigs_compression.zstd.level)
can change from release to release, so they are not expected to be
permanent.
Do we have any data on inbound links to the site? Maybe from Google
Analytics or similar?
-David A
On Mon, Feb 24, 2025 at 10:17 AM Mickael Maison <mickael.mai...@gmail.com>
wrote:
Hi Harish,
This is definitively something we want to do!
MM1. One aspect to keep in mind in terms of compatibility is whether
existing links will still work. Thousands of online resources link to
the Apache Kafka website. Is it possible to keep all links working?
MM2. A bunch of sections are generated. To do so we have a number of
classes in the public API (for example 0, 1) with toHtml() or main()
methods that output HTML. Should we update these to output Markdown
directly instead?
Converting the website has been discussed several times, I'd suggest
checking at least https://issues.apache.org/jira/browse/KAFKA-2967 to
see what was previously said.
0:
https://kafka.apache.org/39/javadoc/org/apache/kafka/common/config/ConfigDef.html#toHtml()
1:
https://kafka.apache.org/39/javadoc/org/apache/kafka/clients/consumer/ConsumerConfig.html#main(java.lang.String%5B%5D)
Thanks,
Mickael
On Mon, Feb 24, 2025 at 3:33 PM Bruno Cadonna <cado...@apache.org> wrote:
Hi Harish,
Thanks for the KIP! Great initiative!
Do you plan to have a development version of the website online in
parallel to the old HTML version?
I understand that locally with the markdown files and an markdown
editor, we see the rough structure of the docs (which is already an
improvement), but we will not see the end product, because for that we
again need a web server.
I assume that after you migrated all docs to markdown, we want to see
the complete website, review the content, and make some refinements.
Best,
Bruno
On 23.02.25 07:52, Harish Vishwanath wrote:
Thank you David. Appreciate your feedback. My comments below:
DA1) I think this is a very good suggestion. While building the
prototype,
I did spend additional effort for older versions of the doc since
there are
fairly significant differences in the structure and format of the
documents
as compared to 3.x versions. I have written the automation which can do
this, but I do think hosting the older versions (<3.x) in its current
format and moving the rest to markdown will reduce the overall
testing/validation effort. That said, if we foresee making relatively
frequent updates to older versions, it might be worth converting
everything
to markdown.
DA2) My hunch is that this can be quite involved. Rendering a usable
HTML
version requires all the supporting artifacts (css, js, images, hugo
shortcodes, partials etc.,) which will only be present in the website
documentation. Further, the hyper links themselves will not work
without
a webserver. To serve a directory structure based HTML, it will
require us
to have different templates for these docs where dependencies on the
artifacts I mentioned above will need to be removed, links will need
to be
converted to use relative directory locations etc.,
DA3) This is definitely possible. Using tools such as `pandoc` we
should be
able to render plain text versions of markdown files. For ex, the below
would create a plain text version of all docs for 39/ branch., which
can be
packaged with the binary distribution.
find docs/ -name "*.md" -type f | while read -r file; do
dir=$(dirname "$file")
filename=$(basename "$file" .md)
mkdir -p "39-ascii/$dir"
pandoc -f markdown -t plain "$file" -o
"39-ascii/$dir/$filename.txt"
done
DA4) Makes sense. I updated the wiki to reflect this.
Regards,
Harish
On Fri, Feb 21, 2025 at 8:56 AM David Arthur <mum...@gmail.com> wrote:
Thanks for the KIP, Harish!
I am +100 just on the title alone :)
The motivations in the KIP are sound. All of these are very real
annoyances
when it comes to maintaining our docs. I suspect we have not done
anything
to improve them since the system we have worked well enough in the
past. A
better way to say that is: it hasn't been bad enough to motivate us to
change it.
With efforts like this, there is always some question about which
tool or
method we should use. I like that your proposal includes these
specifics as
well as a complete and functional demo.
Few questions:
DA1) Do we really need to convert all of our docs to the new thing? It
might be reasonable to draw a line somewhere (maybe 3.0?) and have
docs
prior to that be hosted elsewhere (kafka.apache.org/old or
something).
This
may or may not be more work than just converting everything. WDYT?
DA2) For our packaged docs (those that go in the binaries we
distribute),
do you think it's possible to render HTML that can be viewed without a
webserver? Currently our packaged docs are kind of useless IMO. It
would be
better if they could be viewed without any extra effort by an
operator.
DA3) As a possible alternative to DA2, could we render the markdown
docs as
ascii text files? This is arguably better for operators since they
don't
always have a GUI or web browser handy.
DA4) Just a nitpick, could you reduce the "Sample directory layout"
down to
one or two examples? It's difficult to see the versioned vs
non-versioned
docs in the current block
Thanks!
David A
On Sun, Feb 16, 2025 at 2:10 PM Harish Vishwanath <
harish.shas...@gmail.com>
wrote:
Hello,
Following up from my previous email regarding moving AK
documentation to
markdown, I wrote KIP-1133
<
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1133%3A+AK+Documentation+and+Website+in+Markdown
detailing
the proposal and would like to start discussions. Please take a look.
Regards,
Harish
--
David Arthur
