For a quick correction: > - For Apache Spark 3.1, we are testing R 4.0 on `master` branch, > but we don't have test coverage on `branch-3.0`. > So, I'm wondering if Spark 3.0.1 supports R 4.0 without any issue.
I believe we now test SparkR at branch-3.0 with R 4.0 after https://github.com/apache/spark/commit/56ec5ddcac8233011c17fc7d120a284707f0f712 2020년 9월 2일 (수) 오후 12:47, Dongjoon Hyun <dongjoon.h...@gmail.com>님이 작성: > +1. > > Thank you all. > > I tested the following additionally with OpenJDK 11.0.8. > > - PySpark UT on Python 3.7.7 with Pandas 0.23.2 / PyArrow 0.15.1. > - JDBC integration suite > - K8s integration suite (except SparkR test) > (Minikube: K8s Client v1.18.8, K8s Server v1.17.11) > > For SparkR, it would be great if someone can check the current status > because SparkR was removed from CRAN. > > - I used `R 3.5.2 (2018-12-20)` like 3.0.0 vote, but > `dev/make-distribution.sh` > failed for me when I used the `--r` option. However, since the > release artifact has SparkR, > I believe it's a problem on my env. > > - Jenkins is using `R 3.6.3 (2020-02-29)` and it seems to succeed, > but K8s IT seems to fail consistently due to some unknown reasons. > (https://github.com/apache/spark/pull/29533) > "Run SparkR on simple dataframe.R example *** FAILED ***" > > - For Apache Spark 3.1, we are testing R 4.0 on `master` branch, > but we don't have test coverage on `branch-3.0`. > So, I'm wondering if Spark 3.0.1 supports R 4.0 without any issue. > > In addition to the one which Xiao mentioned, we also had a late arrival on > the correctness area, > but it wasn't a registered blocker before the 3.0.1 RC3 vote. Given that, > it can be a part of 3.0.2. > > - [SPARK-31511][SQL] Make BytesToBytesMap iterators thread-safe > https://github.com/apache/spark/pull/28286#issuecomment-682019408 > > Bests, > Dongjoon. > > On Tue, Sep 1, 2020 at 7:54 AM Xiao Li <lix...@databricks.com> wrote: > >> Want to change my vote to 0, because we are unable to produce an end-user >> query to hit this bug. >> >> Xiao >> >> On Mon, Aug 31, 2020 at 12:41 PM Xiao Li <lix...@databricks.com> wrote: >> >>> -1 due to a regression introduced by a fix in 3.0.1. >>> >>> See https://github.com/apache/spark/pull/29602 >>> >>> Xiao >>> >>> On Mon, Aug 31, 2020 at 9:26 AM Tom Graves <tgraves...@yahoo.com.invalid> >>> wrote: >>> >>>> +1 >>>> >>>> Tom >>>> >>>> On Friday, August 28, 2020, 09:02:31 AM CDT, 郑瑞峰 <ruife...@foxmail.com> >>>> wrote: >>>> >>>> >>>> Please vote on releasing the following candidate as Apache Spark >>>> version 3.0.1. >>>> >>>> The vote is open until Sep 2nd at 9AM PST and passes if a majority +1 >>>> PMC votes are cast, with a minimum of 3 +1 votes. >>>> >>>> [ ] +1 Release this package as Apache Spark 3.0.1 >>>> [ ] -1 Do not release this package because ... >>>> >>>> To learn more about Apache Spark, please see http://spark.apache.org/ >>>> >>>> There are currently no issues targeting 3.0.1 (try project = SPARK AND >>>> "Target Version/s" = "3.0.1" AND status in (Open, Reopened, "In Progress")) >>>> >>>> The tag to be voted on is v3.0.1-rc3 (commit >>>> dc04bf53fe821b7a07f817966c6c173f3b3788c6): >>>> https://github.com/apache/spark/tree/v3.0.1-rc3 >>>> >>>> The release files, including signatures, digests, etc. can be found at: >>>> https://dist.apache.org/repos/dist/dev/spark/v3.0.1-rc3-bin/ >>>> >>>> Signatures used for Spark RCs can be found in this file: >>>> https://dist.apache.org/repos/dist/dev/spark/KEYS >>>> >>>> The staging repository for this release can be found at: >>>> https://repository.apache.org/content/repositories/orgapachespark-1357/ >>>> >>>> The documentation corresponding to this release can be found at: >>>> https://dist.apache.org/repos/dist/dev/spark/v3.0.1-rc3-docs/ >>>> >>>> The list of bug fixes going into 3.0.1 can be found at the following >>>> URL: >>>> https://s.apache.org/q9g2d >>>> >>>> This release is using the release script of the tag v3.0.1-rc3. >>>> >>>> FAQ >>>> >>>> >>>> ========================= >>>> How can I help test this release? >>>> ========================= >>>> >>>> If you are a Spark user, you can help us test this release by taking >>>> an existing Spark workload and running on this release candidate, then >>>> reporting any regressions. >>>> >>>> If you're working in PySpark you can set up a virtual env and install >>>> the current RC and see if anything important breaks, in the Java/Scala >>>> you can add the staging repository to your projects resolvers and test >>>> with the RC (make sure to clean up the artifact cache before/after so >>>> you don't end up building with an out of date RC going forward). >>>> >>>> =========================================== >>>> What should happen to JIRA tickets still targeting 3.0.1? >>>> =========================================== >>>> >>>> The current list of open tickets targeted at 3.0.1 can be found at: >>>> https://issues.apache.org/jira/projects/SPARK and search for "Target >>>> Version/s" = 3.0.1 >>>> >>>> Committers should look at those and triage. Extremely important bug >>>> fixes, documentation, and API tweaks that impact compatibility should >>>> be worked on immediately. Everything else please retarget to an >>>> appropriate release. >>>> >>>> ================== >>>> But my bug isn't fixed? >>>> ================== >>>> >>>> In order to make timely releases, we will typically not hold the >>>> release unless the bug in question is a regression from the previous >>>> release. That being said, if there is something which is a regression >>>> that has not been correctly targeted please ping me or a committer to >>>> help target the issue. >>>> >>>> >>>> >>> >>> -- >>> <https://databricks.com/sparkaisummit/north-america> >>> >> >> >> -- >> <https://databricks.com/sparkaisummit/north-america> >> >
