[GitHub] spark pull request #23012: [SPARK-26014][R] Deprecate R prior to version 3.4...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23012 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23012: [SPARK-26014][R] Deprecate R prior to version 3.4...
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23012#discussion_r233290797
--- Diff: docs/index.md ---
@@ -31,7 +31,8 @@ Spark runs on both Windows and UNIX-like systems (e.g.
Linux, Mac OS). It's easy
locally on one machine --- all you need is to have `java` installed on
your system `PATH`,
or the `JAVA_HOME` environment variable pointing to a Java installation.
-Spark runs on Java 8+, Python 2.7+/3.4+ and R 3.1+. For the Scala API,
Spark {{site.SPARK_VERSION}}
+Spark runs on Java 8+, Python 2.7+/3.4+ and R 3.1+. R prior to version 3.4
is deprecated as of Spark 3.0.
--- End diff --
At least Python version was noted here before.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23012: [SPARK-26014][R] Deprecate R prior to version 3.4...
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23012#discussion_r233147644 --- Diff: R/WINDOWS.md --- @@ -3,7 +3,7 @@ To build SparkR on Windows, the following steps are required 1. Install R (>= 3.1) and [Rtools](http://cran.r-project.org/bin/windows/Rtools/). Make sure to -include Rtools and R in `PATH`. +include Rtools and R in `PATH`. Note that R prior to version 3.4 is deprecated as of Spark 3.0. --- End diff -- I see. Thanks, @felixcheung . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23012: [SPARK-26014][R] Deprecate R prior to version 3.4...
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/23012#discussion_r232881732
--- Diff: R/pkg/R/sparkR.R ---
@@ -283,6 +283,10 @@ sparkR.session <- function(
enableHiveSupport = TRUE,
...) {
+ if (utils::compareVersion(paste0(R.version$major, ".", R.version$minor),
"3.4.0") == -1) {
+warning("R prior to version 3.4 is deprecated as of Spark 3.0.")
+ }
--- End diff --
ditto
`Support for R prior to version 3.4 is deprecated since Spark 3.0.0`
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23012: [SPARK-26014][R] Deprecate R prior to version 3.4...
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/23012#discussion_r232882419
--- Diff: docs/index.md ---
@@ -31,7 +31,8 @@ Spark runs on both Windows and UNIX-like systems (e.g.
Linux, Mac OS). It's easy
locally on one machine --- all you need is to have `java` installed on
your system `PATH`,
or the `JAVA_HOME` environment variable pointing to a Java installation.
-Spark runs on Java 8+, Python 2.7+/3.4+ and R 3.1+. For the Scala API,
Spark {{site.SPARK_VERSION}}
+Spark runs on Java 8+, Python 2.7+/3.4+ and R 3.1+. R prior to version 3.4
is deprecated as of Spark 3.0.
--- End diff --
`R prior to version 3.4 support is deprecated as of Spark 3.0.0.`
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23012: [SPARK-26014][R] Deprecate R prior to version 3.4...
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/23012#discussion_r232882178
--- Diff: docs/index.md ---
@@ -31,7 +31,8 @@ Spark runs on both Windows and UNIX-like systems (e.g.
Linux, Mac OS). It's easy
locally on one machine --- all you need is to have `java` installed on
your system `PATH`,
or the `JAVA_HOME` environment variable pointing to a Java installation.
-Spark runs on Java 8+, Python 2.7+/3.4+ and R 3.1+. For the Scala API,
Spark {{site.SPARK_VERSION}}
+Spark runs on Java 8+, Python 2.7+/3.4+ and R 3.1+. R prior to version 3.4
is deprecated as of Spark 3.0.
--- End diff --
with all the other changes, we haven't listed all deprecation here, or have
we?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23012: [SPARK-26014][R] Deprecate R prior to version 3.4...
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/23012#discussion_r232881594 --- Diff: R/WINDOWS.md --- @@ -3,7 +3,7 @@ To build SparkR on Windows, the following steps are required 1. Install R (>= 3.1) and [Rtools](http://cran.r-project.org/bin/windows/Rtools/). Make sure to -include Rtools and R in `PATH`. +include Rtools and R in `PATH`. Note that R prior to version 3.4 is deprecated as of Spark 3.0. --- End diff -- I really would prefer "unsupported" but if we go with this it should say `Note that support for R prior to version 3.4 is deprecated as of Spark 3.0.0.` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23012: [SPARK-26014][R] Deprecate R prior to version 3.4...
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23012#discussion_r232810462 --- Diff: R/WINDOWS.md --- @@ -3,7 +3,7 @@ To build SparkR on Windows, the following steps are required 1. Install R (>= 3.1) and [Rtools](http://cran.r-project.org/bin/windows/Rtools/). Make sure to -include Rtools and R in `PATH`. +include Rtools and R in `PATH`. Note that R prior to version 3.4 is deprecated as of Spark 3.0. --- End diff -- I'm wondering if we can deprecate this in Spark 2.4.1? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23012: [SPARK-26014][R] Deprecate R prior to version 3.4...
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23012#discussion_r232742917
--- Diff: R/pkg/R/sparkR.R ---
@@ -283,6 +283,10 @@ sparkR.session <- function(
enableHiveSupport = TRUE,
...) {
+ if (utils::compareVersion(paste0(R.version$major, ".", R.version$minor),
"3.4.0") == -1) {
+warning("R prior to version 3.4 is deprecated as of Spark 3.0.")
+ }
--- End diff --
@felixcheung, I updated the condition to check major version as well.
BTW, if we add it in .First in general.R, looks it's not going to print out
the warnings in the SparkR shell. When we drop PySpark, we did in a similar
place
(https://github.com/apache/spark/pull/15733/files#diff-01e513a19a2a6f31ecaabd3dd6ac12d9R192).
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23012: [SPARK-26014][R] Deprecate R prior to version 3.4...
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23012#discussion_r232549234
--- Diff: docs/index.md ---
@@ -31,7 +31,8 @@ Spark runs on both Windows and UNIX-like systems (e.g.
Linux, Mac OS). It's easy
locally on one machine --- all you need is to have `java` installed on
your system `PATH`,
or the `JAVA_HOME` environment variable pointing to a Java installation.
-Spark runs on Java 8+, Python 2.7+/3.4+ and R 3.1+. For the Scala API,
Spark {{site.SPARK_VERSION}}
+Spark runs on Java 8+, Python 2.7+/3.4+ and R 3.1+. R prior to version 3.4
is deprecated as of Spark 3.0.
--- End diff --
Ah, yea, I switched this to deprecate it for now. I was a bit curious
about that.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23012: [SPARK-26014][R] Deprecate R prior to version 3.4...
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/23012#discussion_r232549062
--- Diff: docs/index.md ---
@@ -31,7 +31,8 @@ Spark runs on both Windows and UNIX-like systems (e.g.
Linux, Mac OS). It's easy
locally on one machine --- all you need is to have `java` installed on
your system `PATH`,
or the `JAVA_HOME` environment variable pointing to a Java installation.
-Spark runs on Java 8+, Python 2.7+/3.4+ and R 3.1+. For the Scala API,
Spark {{site.SPARK_VERSION}}
+Spark runs on Java 8+, Python 2.7+/3.4+ and R 3.1+. R prior to version 3.4
is deprecated as of Spark 3.0.
--- End diff --
hmm, so R prior to version 3.4 is just deprecated, not dropped in in Spark
3.0?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23012: [SPARK-26014][R] Deprecate R prior to version 3.4...
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23012#discussion_r232549053
--- Diff: docs/index.md ---
@@ -31,7 +31,8 @@ Spark runs on both Windows and UNIX-like systems (e.g.
Linux, Mac OS). It's easy
locally on one machine --- all you need is to have `java` installed on
your system `PATH`,
or the `JAVA_HOME` environment variable pointing to a Java installation.
-Spark runs on Java 8+, Python 2.7+/3.4+ and R 3.1+. For the Scala API,
Spark {{site.SPARK_VERSION}}
+Spark runs on Java 8+, Python 2.7+/3.4+ and R 3.1+. R prior to version 3.4
is deprecated as of Spark 3.0.
--- End diff --
Hm .. I was thinking we could change them when we actually drop the
support. Technically it does support 3.1+ yet although 3.1, 3.2, and 3.3 are
deprecated.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23012: [SPARK-26014][R] Deprecate R prior to version 3.4...
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/23012#discussion_r232548211
--- Diff: docs/index.md ---
@@ -31,7 +31,8 @@ Spark runs on both Windows and UNIX-like systems (e.g.
Linux, Mac OS). It's easy
locally on one machine --- all you need is to have `java` installed on
your system `PATH`,
or the `JAVA_HOME` environment variable pointing to a Java installation.
-Spark runs on Java 8+, Python 2.7+/3.4+ and R 3.1+. For the Scala API,
Spark {{site.SPARK_VERSION}}
+Spark runs on Java 8+, Python 2.7+/3.4+ and R 3.1+. R prior to version 3.4
is deprecated as of Spark 3.0.
--- End diff --
3.1+ -> 3.4+?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23012: [SPARK-26014][R] Deprecate R prior to version 3.4...
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/23012 [SPARK-26014][R] Deprecate R prior to version 3.4 in SparkR ## What changes were proposed in this pull request? This PR proposes to bump up the minimum versions of R from 3.1 to 3.4. R version. 3.1.x is too old. It's released 4.5 years ago. R 3.4.0 is released 1.5 years ago. Considering the timing for Spark 3.0, deprecating lower versions, bumping up R to 3.4 might be reasonable option. It should be good to deprecate and drop < R 3.4 support. If we think about the practice, nothing particular is required within R codes as far as I can tell, except: 1. https://github.com/apache/spark/blob/master/R/pkg/src-native/string_hash_code.c 2. `env` becomes immutable but in some low versions they are mutable ... if I remember correctly .. shouldn't be a big deal in SparkR side. 3. We will need to upgrade Jenkins's R version to 3.4, which mean we're not going to test 3.1 R version - this should be okay because we're already not testing R 3.2, 3.3 and 3.4. We test 3.5 in Appveyor, and 3.1 in Jenkins. ## How was this patch tested? Jenkins tests. You can merge this pull request into a Git repository by running: $ git pull https://github.com/HyukjinKwon/spark SPARK-26014 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/23012.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #23012 commit dc2dbd923a1396ca5a7a950df35da57cc70c2ab8 Author: hyukjinkwon Date: 2018-11-12T05:39:14Z Deprecate R prior to version 3.4 in SparkR --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
