[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR on Windows
[
https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535105#comment-16535105
]
Shivaram Venkataraman commented on SPARK-24535:
---
Thanks [~felixcheung] -- I just got back to work today. Will take a look at the
fix now.
> Fix java version parsing in SparkR on Windows
> -
>
> Key: SPARK-24535
> URL: https://issues.apache.org/jira/browse/SPARK-24535
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 2.3.1, 2.4.0
>Reporter: Shivaram Venkataraman
>Assignee: Felix Cheung
>Priority: Blocker
> Fix For: 2.3.2, 2.4.0
>
>
> We see errors on CRAN of the form
> {code:java}
> java version "1.8.0_144"
> Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
> Picked up _JAVA_OPTIONS: -XX:-UsePerfData
> -- 1. Error: create DataFrame from list or data.frame (@test_basic.R#21)
> --
> subscript out of bounds
> 1: sparkR.session(master = sparkRTestMaster, enableHiveSupport = FALSE,
> sparkConfig = sparkRTestConfig) at
> D:/temp/RtmpIJ8Cc3/RLIBS_3242c713c3181/SparkR/tests/testthat/test_basic.R:21
> 2: sparkR.sparkContext(master, appName, sparkHome, sparkConfigMap,
> sparkExecutorEnvMap,
> sparkJars, sparkPackages)
> 3: checkJavaVersion()
> 4: strsplit(javaVersionFilter[[1]], "[\"]")
> {code}
> The complete log file is at
> http://home.apache.org/~shivaram/SparkR_2.3.1_check_results/Windows/00check.log
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR on Windows
[
https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534502#comment-16534502
]
Felix Cheung commented on SPARK-24535:
--
[~shivaram] I've merged it, but it could be great if you could review the fix
> Fix java version parsing in SparkR on Windows
> -
>
> Key: SPARK-24535
> URL: https://issues.apache.org/jira/browse/SPARK-24535
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 2.3.1, 2.4.0
>Reporter: Shivaram Venkataraman
>Assignee: Felix Cheung
>Priority: Blocker
>
> We see errors on CRAN of the form
> {code:java}
> java version "1.8.0_144"
> Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
> Picked up _JAVA_OPTIONS: -XX:-UsePerfData
> -- 1. Error: create DataFrame from list or data.frame (@test_basic.R#21)
> --
> subscript out of bounds
> 1: sparkR.session(master = sparkRTestMaster, enableHiveSupport = FALSE,
> sparkConfig = sparkRTestConfig) at
> D:/temp/RtmpIJ8Cc3/RLIBS_3242c713c3181/SparkR/tests/testthat/test_basic.R:21
> 2: sparkR.sparkContext(master, appName, sparkHome, sparkConfigMap,
> sparkExecutorEnvMap,
> sparkJars, sparkPackages)
> 3: checkJavaVersion()
> 4: strsplit(javaVersionFilter[[1]], "[\"]")
> {code}
> The complete log file is at
> http://home.apache.org/~shivaram/SparkR_2.3.1_check_results/Windows/00check.log
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR on Windows
[
https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16532145#comment-16532145
]
Saisai Shao commented on SPARK-24535:
-
Hi [~felixcheung] what's the current status of this JIRA, do you have an ETA
about it?
> Fix java version parsing in SparkR on Windows
> -
>
> Key: SPARK-24535
> URL: https://issues.apache.org/jira/browse/SPARK-24535
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 2.3.1, 2.4.0
>Reporter: Shivaram Venkataraman
>Assignee: Felix Cheung
>Priority: Blocker
>
> We see errors on CRAN of the form
> {code:java}
> java version "1.8.0_144"
> Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
> Picked up _JAVA_OPTIONS: -XX:-UsePerfData
> -- 1. Error: create DataFrame from list or data.frame (@test_basic.R#21)
> --
> subscript out of bounds
> 1: sparkR.session(master = sparkRTestMaster, enableHiveSupport = FALSE,
> sparkConfig = sparkRTestConfig) at
> D:/temp/RtmpIJ8Cc3/RLIBS_3242c713c3181/SparkR/tests/testthat/test_basic.R:21
> 2: sparkR.sparkContext(master, appName, sparkHome, sparkConfigMap,
> sparkExecutorEnvMap,
> sparkJars, sparkPackages)
> 3: checkJavaVersion()
> 4: strsplit(javaVersionFilter[[1]], "[\"]")
> {code}
> The complete log file is at
> http://home.apache.org/~shivaram/SparkR_2.3.1_check_results/Windows/00check.log
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR
[
https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16530963#comment-16530963
]
Felix Cheung commented on SPARK-24535:
--
ok, I submitted a modified package to win-builder, and was able to confirm the
problem with launchScript. updating with the fix, result here
https://win-builder.r-project.org/zD6OfPID9JtR/00check.log
> Fix java version parsing in SparkR
> --
>
> Key: SPARK-24535
> URL: https://issues.apache.org/jira/browse/SPARK-24535
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 2.3.1, 2.4.0
>Reporter: Shivaram Venkataraman
>Assignee: Felix Cheung
>Priority: Blocker
>
> We see errors on CRAN of the form
> {code:java}
> java version "1.8.0_144"
> Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
> Picked up _JAVA_OPTIONS: -XX:-UsePerfData
> -- 1. Error: create DataFrame from list or data.frame (@test_basic.R#21)
> --
> subscript out of bounds
> 1: sparkR.session(master = sparkRTestMaster, enableHiveSupport = FALSE,
> sparkConfig = sparkRTestConfig) at
> D:/temp/RtmpIJ8Cc3/RLIBS_3242c713c3181/SparkR/tests/testthat/test_basic.R:21
> 2: sparkR.sparkContext(master, appName, sparkHome, sparkConfigMap,
> sparkExecutorEnvMap,
> sparkJars, sparkPackages)
> 3: checkJavaVersion()
> 4: strsplit(javaVersionFilter[[1]], "[\"]")
> {code}
> The complete log file is at
> http://home.apache.org/~shivaram/SparkR_2.3.1_check_results/Windows/00check.log
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR
[
https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527234#comment-16527234
]
Apache Spark commented on SPARK-24535:
--
User 'felixcheung' has created a pull request for this issue:
https://github.com/apache/spark/pull/21666
> Fix java version parsing in SparkR
> --
>
> Key: SPARK-24535
> URL: https://issues.apache.org/jira/browse/SPARK-24535
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 2.3.1, 2.4.0
>Reporter: Shivaram Venkataraman
>Assignee: Felix Cheung
>Priority: Blocker
>
> We see errors on CRAN of the form
> {code:java}
> java version "1.8.0_144"
> Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
> Picked up _JAVA_OPTIONS: -XX:-UsePerfData
> -- 1. Error: create DataFrame from list or data.frame (@test_basic.R#21)
> --
> subscript out of bounds
> 1: sparkR.session(master = sparkRTestMaster, enableHiveSupport = FALSE,
> sparkConfig = sparkRTestConfig) at
> D:/temp/RtmpIJ8Cc3/RLIBS_3242c713c3181/SparkR/tests/testthat/test_basic.R:21
> 2: sparkR.sparkContext(master, appName, sparkHome, sparkConfigMap,
> sparkExecutorEnvMap,
> sparkJars, sparkPackages)
> 3: checkJavaVersion()
> 4: strsplit(javaVersionFilter[[1]], "[\"]")
> {code}
> The complete log file is at
> http://home.apache.org/~shivaram/SparkR_2.3.1_check_results/Windows/00check.log
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR
[
https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514031#comment-16514031
]
Hyukjin Kwon commented on SPARK-24535:
--
When I tried it before, I remember I faced some issues .. mind if I ask to
share the steps you did just roughly? It doesn't have to be perfect if it's
messy but I want to try what you did roughly.
> Fix java version parsing in SparkR
> --
>
> Key: SPARK-24535
> URL: https://issues.apache.org/jira/browse/SPARK-24535
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 2.3.1, 2.4.0
>Reporter: Shivaram Venkataraman
>Priority: Major
>
> We see errors on CRAN of the form
> {code:java}
> java version "1.8.0_144"
> Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
> Picked up _JAVA_OPTIONS: -XX:-UsePerfData
> -- 1. Error: create DataFrame from list or data.frame (@test_basic.R#21)
> --
> subscript out of bounds
> 1: sparkR.session(master = sparkRTestMaster, enableHiveSupport = FALSE,
> sparkConfig = sparkRTestConfig) at
> D:/temp/RtmpIJ8Cc3/RLIBS_3242c713c3181/SparkR/tests/testthat/test_basic.R:21
> 2: sparkR.sparkContext(master, appName, sparkHome, sparkConfigMap,
> sparkExecutorEnvMap,
> sparkJars, sparkPackages)
> 3: checkJavaVersion()
> 4: strsplit(javaVersionFilter[[1]], "[\"]")
> {code}
> The complete log file is at
> http://home.apache.org/~shivaram/SparkR_2.3.1_check_results/Windows/00check.log
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR
[
https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514020#comment-16514020
]
Shivaram Venkataraman commented on SPARK-24535:
---
I was going to do it manually. If we do it in the PR builder it will be great !
> Fix java version parsing in SparkR
> --
>
> Key: SPARK-24535
> URL: https://issues.apache.org/jira/browse/SPARK-24535
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 2.3.1, 2.4.0
>Reporter: Shivaram Venkataraman
>Priority: Major
>
> We see errors on CRAN of the form
> {code:java}
> java version "1.8.0_144"
> Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
> Picked up _JAVA_OPTIONS: -XX:-UsePerfData
> -- 1. Error: create DataFrame from list or data.frame (@test_basic.R#21)
> --
> subscript out of bounds
> 1: sparkR.session(master = sparkRTestMaster, enableHiveSupport = FALSE,
> sparkConfig = sparkRTestConfig) at
> D:/temp/RtmpIJ8Cc3/RLIBS_3242c713c3181/SparkR/tests/testthat/test_basic.R:21
> 2: sparkR.sparkContext(master, appName, sparkHome, sparkConfigMap,
> sparkExecutorEnvMap,
> sparkJars, sparkPackages)
> 3: checkJavaVersion()
> 4: strsplit(javaVersionFilter[[1]], "[\"]")
> {code}
> The complete log file is at
> http://home.apache.org/~shivaram/SparkR_2.3.1_check_results/Windows/00check.log
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR
[
https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514016#comment-16514016
]
Hyukjin Kwon commented on SPARK-24535:
--
[~shivaram], how did you run this on Windows? I think we should better run this
in PR builder.
> Fix java version parsing in SparkR
> --
>
> Key: SPARK-24535
> URL: https://issues.apache.org/jira/browse/SPARK-24535
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 2.3.1, 2.4.0
>Reporter: Shivaram Venkataraman
>Priority: Major
>
> We see errors on CRAN of the form
> {code:java}
> java version "1.8.0_144"
> Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
> Picked up _JAVA_OPTIONS: -XX:-UsePerfData
> -- 1. Error: create DataFrame from list or data.frame (@test_basic.R#21)
> --
> subscript out of bounds
> 1: sparkR.session(master = sparkRTestMaster, enableHiveSupport = FALSE,
> sparkConfig = sparkRTestConfig) at
> D:/temp/RtmpIJ8Cc3/RLIBS_3242c713c3181/SparkR/tests/testthat/test_basic.R:21
> 2: sparkR.sparkContext(master, appName, sparkHome, sparkConfigMap,
> sparkExecutorEnvMap,
> sparkJars, sparkPackages)
> 3: checkJavaVersion()
> 4: strsplit(javaVersionFilter[[1]], "[\"]")
> {code}
> The complete log file is at
> http://home.apache.org/~shivaram/SparkR_2.3.1_check_results/Windows/00check.log
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR
[
https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514012#comment-16514012
]
Shivaram Venkataraman commented on SPARK-24535:
---
I'm not sure if its only failing on Windows -- the Debian test on CRAN did not
have the same Java version. I can try to do a test on Windows later today to
see what I find.
> Fix java version parsing in SparkR
> --
>
> Key: SPARK-24535
> URL: https://issues.apache.org/jira/browse/SPARK-24535
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 2.3.1, 2.4.0
>Reporter: Shivaram Venkataraman
>Priority: Major
>
> We see errors on CRAN of the form
> {code:java}
> java version "1.8.0_144"
> Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
> Picked up _JAVA_OPTIONS: -XX:-UsePerfData
> -- 1. Error: create DataFrame from list or data.frame (@test_basic.R#21)
> --
> subscript out of bounds
> 1: sparkR.session(master = sparkRTestMaster, enableHiveSupport = FALSE,
> sparkConfig = sparkRTestConfig) at
> D:/temp/RtmpIJ8Cc3/RLIBS_3242c713c3181/SparkR/tests/testthat/test_basic.R:21
> 2: sparkR.sparkContext(master, appName, sparkHome, sparkConfigMap,
> sparkExecutorEnvMap,
> sparkJars, sparkPackages)
> 3: checkJavaVersion()
> 4: strsplit(javaVersionFilter[[1]], "[\"]")
> {code}
> The complete log file is at
> http://home.apache.org/~shivaram/SparkR_2.3.1_check_results/Windows/00check.log
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR
[
https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16513372#comment-16513372
]
Felix Cheung commented on SPARK-24535:
--
is this only failing on windows?
I wonder if this is related to how stdout is redirected in launchScript
> Fix java version parsing in SparkR
> --
>
> Key: SPARK-24535
> URL: https://issues.apache.org/jira/browse/SPARK-24535
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 2.3.1, 2.4.0
>Reporter: Shivaram Venkataraman
>Priority: Major
>
> We see errors on CRAN of the form
> {code:java}
> java version "1.8.0_144"
> Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
> Picked up _JAVA_OPTIONS: -XX:-UsePerfData
> -- 1. Error: create DataFrame from list or data.frame (@test_basic.R#21)
> --
> subscript out of bounds
> 1: sparkR.session(master = sparkRTestMaster, enableHiveSupport = FALSE,
> sparkConfig = sparkRTestConfig) at
> D:/temp/RtmpIJ8Cc3/RLIBS_3242c713c3181/SparkR/tests/testthat/test_basic.R:21
> 2: sparkR.sparkContext(master, appName, sparkHome, sparkConfigMap,
> sparkExecutorEnvMap,
> sparkJars, sparkPackages)
> 3: checkJavaVersion()
> 4: strsplit(javaVersionFilter[[1]], "[\"]")
> {code}
> The complete log file is at
> http://home.apache.org/~shivaram/SparkR_2.3.1_check_results/Windows/00check.log
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
