Yes, it's relevant to that PR. I wonder, if we want to expose version switcher, it should be in versionless doc (spark-website) rather than the doc being pinned to a specific version.
On Tue, Mar 5, 2024 at 11:18 AM Hyukjin Kwon <gurwls...@apache.org> wrote: > Is this related to https://github.com/apache/spark/pull/42428? > > cc @Yang,Jie(INF) <yangji...@baidu.com> > > On Mon, 4 Mar 2024 at 22:21, Jungtaek Lim <kabhwan.opensou...@gmail.com> > wrote: > >> Shall we revisit this functionality? The API doc is built with individual >> versions, and for each individual version we depend on other released >> versions. This does not seem to be right to me. Also, the functionality is >> only in PySpark API doc which does not seem to be consistent as well. >> >> I don't think this is manageable with the current approach (listing >> versions in version-dependent doc). Let's say we release 3.4.3 after 3.5.1. >> Should we update the versions in 3.5.1 to add 3.4.3 in version switcher? >> How about the time we are going to release the new version after releasing >> 10 versions? What's the criteria of pruning the version? >> >> Unless we have a good answer to these questions, I think it's better to >> revert the functionality - it missed various considerations. >> >> On Fri, Mar 1, 2024 at 2:44 PM Jungtaek Lim <kabhwan.opensou...@gmail.com> >> wrote: >> >>> Thanks for reporting - this is odd - the dropdown did not exist in other >>> recent releases. >>> >>> https://spark.apache.org/docs/3.5.0/api/python/index.html >>> https://spark.apache.org/docs/3.4.2/api/python/index.html >>> https://spark.apache.org/docs/3.3.4/api/python/index.html >>> >>> Looks like the dropdown feature was recently introduced but partially >>> done. The addition of a dropdown was done, but the way how to bump the >>> version was missed to be documented. >>> The contributor proposed the way to update the version "automatically", >>> but the PR wasn't merged. As a result, we are neither having the >>> instruction how to bump the version manually, nor having the automatic bump. >>> >>> * PR for addition of dropdown: >>> https://github.com/apache/spark/pull/42428 >>> * PR for automatically bumping version: >>> https://github.com/apache/spark/pull/42881 >>> >>> We will probably need to add an instruction in the release process to >>> update the version. (For automatic bumping I don't have a good idea.) >>> I'll look into it. Please expect some delay during the holiday weekend >>> in S. Korea. >>> >>> Thanks again. >>> Jungtaek Lim (HeartSaVioR) >>> >>> >>> On Fri, Mar 1, 2024 at 2:14 PM Dongjoon Hyun <dongjoon.h...@gmail.com> >>> wrote: >>> >>>> BTW, Jungtaek. >>>> >>>> PySpark document seems to show a wrong branch. At this time, `master`. >>>> >>>> https://spark.apache.org/docs/3.5.1/api/python/index.html >>>> >>>> PySpark Overview >>>> <https://spark.apache.org/docs/3.5.1/api/python/index.html#pyspark-overview> >>>> >>>> Date: Feb 24, 2024 Version: master >>>> >>>> [image: Screenshot 2024-02-29 at 21.12.24.png] >>>> >>>> >>>> Could you do the follow-up, please? >>>> >>>> Thank you in advance. >>>> >>>> Dongjoon. >>>> >>>> >>>> On Thu, Feb 29, 2024 at 2:48 PM John Zhuge <jzh...@apache.org> wrote: >>>> >>>>> Excellent work, congratulations! >>>>> >>>>> On Wed, Feb 28, 2024 at 10:12 PM Dongjoon Hyun < >>>>> dongjoon.h...@gmail.com> wrote: >>>>> >>>>>> Congratulations! >>>>>> >>>>>> Bests, >>>>>> Dongjoon. >>>>>> >>>>>> On Wed, Feb 28, 2024 at 11:43 AM beliefer <belie...@163.com> wrote: >>>>>> >>>>>>> Congratulations! >>>>>>> >>>>>>> >>>>>>> >>>>>>> At 2024-02-28 17:43:25, "Jungtaek Lim" <kabhwan.opensou...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>> Hi everyone, >>>>>>> >>>>>>> We are happy to announce the availability of Spark 3.5.1! >>>>>>> >>>>>>> Spark 3.5.1 is a maintenance release containing stability fixes. This >>>>>>> release is based on the branch-3.5 maintenance branch of Spark. We >>>>>>> strongly >>>>>>> recommend all 3.5 users to upgrade to this stable release. >>>>>>> >>>>>>> To download Spark 3.5.1, head over to the download page: >>>>>>> https://spark.apache.org/downloads.html >>>>>>> >>>>>>> To view the release notes: >>>>>>> https://spark.apache.org/releases/spark-release-3-5-1.html >>>>>>> >>>>>>> We would like to acknowledge all community members for contributing >>>>>>> to this >>>>>>> release. This release would not have been possible without you. >>>>>>> >>>>>>> Jungtaek Lim >>>>>>> >>>>>>> ps. Yikun is helping us through releasing the official docker image >>>>>>> for Spark 3.5.1 (Thanks Yikun!) It may take some time to be generally >>>>>>> available. >>>>>>> >>>>>>> >>>>> >>>>> -- >>>>> John Zhuge >>>>> >>>>
