Nice. Thank you for confirming, Hyukjin. Bests, Dongjoon.
On Wed, Sep 2, 2020 at 2:14 AM Hyukjin Kwon <gurwls...@gmail.com> wrote: > For a quick correction: > > > - For Apache Spark 3.1, we are testing R 4.0 on `master` branch, > > but we don't have test coverage on `branch-3.0`. > > So, I'm wondering if Spark 3.0.1 supports R 4.0 without any issue. > > I believe we now test SparkR at branch-3.0 with R 4.0 after > https://github.com/apache/spark/commit/56ec5ddcac8233011c17fc7d120a284707f0f712 > > > 2020년 9월 2일 (수) 오후 12:47, Dongjoon Hyun <dongjoon.h...@gmail.com>님이 작성: > >> +1. >> >> Thank you all. >> >> I tested the following additionally with OpenJDK 11.0.8. >> >> - PySpark UT on Python 3.7.7 with Pandas 0.23.2 / PyArrow 0.15.1. >> - JDBC integration suite >> - K8s integration suite (except SparkR test) >> (Minikube: K8s Client v1.18.8, K8s Server v1.17.11) >> >> For SparkR, it would be great if someone can check the current status >> because SparkR was removed from CRAN. >> >> - I used `R 3.5.2 (2018-12-20)` like 3.0.0 vote, but >> `dev/make-distribution.sh` >> failed for me when I used the `--r` option. However, since the >> release artifact has SparkR, >> I believe it's a problem on my env. >> >> - Jenkins is using `R 3.6.3 (2020-02-29)` and it seems to succeed, >> but K8s IT seems to fail consistently due to some unknown reasons. >> (https://github.com/apache/spark/pull/29533) >> "Run SparkR on simple dataframe.R example *** FAILED ***" >> >> - For Apache Spark 3.1, we are testing R 4.0 on `master` branch, >> but we don't have test coverage on `branch-3.0`. >> So, I'm wondering if Spark 3.0.1 supports R 4.0 without any issue. >> >> In addition to the one which Xiao mentioned, we also had a late arrival >> on the correctness area, >> but it wasn't a registered blocker before the 3.0.1 RC3 vote. Given that, >> it can be a part of 3.0.2. >> >> - [SPARK-31511][SQL] Make BytesToBytesMap iterators thread-safe >> https://github.com/apache/spark/pull/28286#issuecomment-682019408 >> >> Bests, >> Dongjoon. >> >> On Tue, Sep 1, 2020 at 7:54 AM Xiao Li <lix...@databricks.com> wrote: >> >>> Want to change my vote to 0, because we are unable to produce an >>> end-user query to hit this bug. >>> >>> Xiao >>> >>> On Mon, Aug 31, 2020 at 12:41 PM Xiao Li <lix...@databricks.com> wrote: >>> >>>> -1 due to a regression introduced by a fix in 3.0.1. >>>> >>>> See https://github.com/apache/spark/pull/29602 >>>> >>>> Xiao >>>> >>>> On Mon, Aug 31, 2020 at 9:26 AM Tom Graves <tgraves...@yahoo.com.invalid> >>>> wrote: >>>> >>>>> +1 >>>>> >>>>> Tom >>>>> >>>>> On Friday, August 28, 2020, 09:02:31 AM CDT, 郑瑞峰 <ruife...@foxmail.com> >>>>> wrote: >>>>> >>>>> >>>>> Please vote on releasing the following candidate as Apache Spark >>>>> version 3.0.1. >>>>> >>>>> The vote is open until Sep 2nd at 9AM PST and passes if a majority +1 >>>>> PMC votes are cast, with a minimum of 3 +1 votes. >>>>> >>>>> [ ] +1 Release this package as Apache Spark 3.0.1 >>>>> [ ] -1 Do not release this package because ... >>>>> >>>>> To learn more about Apache Spark, please see http://spark.apache.org/ >>>>> >>>>> There are currently no issues targeting 3.0.1 (try project = SPARK AND >>>>> "Target Version/s" = "3.0.1" AND status in (Open, Reopened, "In >>>>> Progress")) >>>>> >>>>> The tag to be voted on is v3.0.1-rc3 (commit >>>>> dc04bf53fe821b7a07f817966c6c173f3b3788c6): >>>>> https://github.com/apache/spark/tree/v3.0.1-rc3 >>>>> >>>>> The release files, including signatures, digests, etc. can be found at: >>>>> https://dist.apache.org/repos/dist/dev/spark/v3.0.1-rc3-bin/ >>>>> >>>>> Signatures used for Spark RCs can be found in this file: >>>>> https://dist.apache.org/repos/dist/dev/spark/KEYS >>>>> >>>>> The staging repository for this release can be found at: >>>>> https://repository.apache.org/content/repositories/orgapachespark-1357/ >>>>> >>>>> The documentation corresponding to this release can be found at: >>>>> https://dist.apache.org/repos/dist/dev/spark/v3.0.1-rc3-docs/ >>>>> >>>>> The list of bug fixes going into 3.0.1 can be found at the following >>>>> URL: >>>>> https://s.apache.org/q9g2d >>>>> >>>>> This release is using the release script of the tag v3.0.1-rc3. >>>>> >>>>> FAQ >>>>> >>>>> >>>>> ========================= >>>>> How can I help test this release? >>>>> ========================= >>>>> >>>>> If you are a Spark user, you can help us test this release by taking >>>>> an existing Spark workload and running on this release candidate, then >>>>> reporting any regressions. >>>>> >>>>> If you're working in PySpark you can set up a virtual env and install >>>>> the current RC and see if anything important breaks, in the Java/Scala >>>>> you can add the staging repository to your projects resolvers and test >>>>> with the RC (make sure to clean up the artifact cache before/after so >>>>> you don't end up building with an out of date RC going forward). >>>>> >>>>> =========================================== >>>>> What should happen to JIRA tickets still targeting 3.0.1? >>>>> =========================================== >>>>> >>>>> The current list of open tickets targeted at 3.0.1 can be found at: >>>>> https://issues.apache.org/jira/projects/SPARK and search for "Target >>>>> Version/s" = 3.0.1 >>>>> >>>>> Committers should look at those and triage. Extremely important bug >>>>> fixes, documentation, and API tweaks that impact compatibility should >>>>> be worked on immediately. Everything else please retarget to an >>>>> appropriate release. >>>>> >>>>> ================== >>>>> But my bug isn't fixed? >>>>> ================== >>>>> >>>>> In order to make timely releases, we will typically not hold the >>>>> release unless the bug in question is a regression from the previous >>>>> release. That being said, if there is something which is a regression >>>>> that has not been correctly targeted please ping me or a committer to >>>>> help target the issue. >>>>> >>>>> >>>>> >>>> >>>> -- >>>> <https://databricks.com/sparkaisummit/north-america> >>>> >>> >>> >>> -- >>> <https://databricks.com/sparkaisummit/north-america> >>> >>
