[jira] [Commented] (SPARK-16510) Move SparkR test JAR into Spark, include its source code
[ https://issues.apache.org/jira/browse/SPARK-16510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381585#comment-15381585 ] Apache Spark commented on SPARK-16510: -- User 'shivaram' has created a pull request for this issue: https://github.com/apache/spark/pull/14243 > Move SparkR test JAR into Spark, include its source code > > > Key: SPARK-16510 > URL: https://issues.apache.org/jira/browse/SPARK-16510 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Shivaram Venkataraman >Assignee: Shivaram Venkataraman > > One of the `NOTE`s in the R CMD check is that we currently include a test JAR > file in SparkR which is a binary only artifact. I think we can take two steps > to address this > (a) I think we should include the source code for this in say core/src/test/ > or something like that. As far as I know the JAR file just needs to have a > single method. > (b) We should move the JAR file out of the SparkR test support and into some > other location in Spark. The trouble is that its tricky to run the test with > CRAN mode then. We could either disable the test for CRAN or download the JAR > from an external URL -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-16510) Move SparkR test JAR into Spark, include its source code
[ https://issues.apache.org/jira/browse/SPARK-16510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381580#comment-15381580 ] Shivaram Venkataraman commented on SPARK-16510: --- I actually found a better way to do this by moving the test into our Scala test suite. Will send a PR soon > Move SparkR test JAR into Spark, include its source code > > > Key: SPARK-16510 > URL: https://issues.apache.org/jira/browse/SPARK-16510 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Shivaram Venkataraman >Assignee: Shivaram Venkataraman > > One of the `NOTE`s in the R CMD check is that we currently include a test JAR > file in SparkR which is a binary only artifact. I think we can take two steps > to address this > (a) I think we should include the source code for this in say core/src/test/ > or something like that. As far as I know the JAR file just needs to have a > single method. > (b) We should move the JAR file out of the SparkR test support and into some > other location in Spark. The trouble is that its tricky to run the test with > CRAN mode then. We could either disable the test for CRAN or download the JAR > from an external URL -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-16510) Move SparkR test JAR into Spark, include its source code
[ https://issues.apache.org/jira/browse/SPARK-16510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15376273#comment-15376273 ] Felix Cheung commented on SPARK-16510: -- I see. I think Jenkins is optimizing out what to build so it might not build the jar? I agree upload jar might be ideal. > Move SparkR test JAR into Spark, include its source code > > > Key: SPARK-16510 > URL: https://issues.apache.org/jira/browse/SPARK-16510 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Shivaram Venkataraman > > One of the `NOTE`s in the R CMD check is that we currently include a test JAR > file in SparkR which is a binary only artifact. I think we can take two steps > to address this > (a) I think we should include the source code for this in say core/src/test/ > or something like that. As far as I know the JAR file just needs to have a > single method. > (b) We should move the JAR file out of the SparkR test support and into some > other location in Spark. The trouble is that its tricky to run the test with > CRAN mode then. We could either disable the test for CRAN or download the JAR > from an external URL -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-16510) Move SparkR test JAR into Spark, include its source code
[ https://issues.apache.org/jira/browse/SPARK-16510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15376010#comment-15376010 ] Shivaram Venkataraman commented on SPARK-16510: --- For (a) the assumption is that we will be building the JAR during the Spark build process using Maven. It need not be run during the R package installation process and can be just used as a reference mechanism for generating a JAR. Also since we dont expect the file to change, we can upload the JAR built as a part of a release to some known location ? > Move SparkR test JAR into Spark, include its source code > > > Key: SPARK-16510 > URL: https://issues.apache.org/jira/browse/SPARK-16510 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Shivaram Venkataraman > > One of the `NOTE`s in the R CMD check is that we currently include a test JAR > file in SparkR which is a binary only artifact. I think we can take two steps > to address this > (a) I think we should include the source code for this in say core/src/test/ > or something like that. As far as I know the JAR file just needs to have a > single method. > (b) We should move the JAR file out of the SparkR test support and into some > other location in Spark. The trouble is that its tricky to run the test with > CRAN mode then. We could either disable the test for CRAN or download the JAR > from an external URL -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-16510) Move SparkR test JAR into Spark, include its source code
[ https://issues.apache.org/jira/browse/SPARK-16510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15376000#comment-15376000 ] Felix Cheung commented on SPARK-16510: -- I suspect (a) is problematic though? Could we assume javac is there to build the jar from source? maybe skip test or download jar is easier? > Move SparkR test JAR into Spark, include its source code > > > Key: SPARK-16510 > URL: https://issues.apache.org/jira/browse/SPARK-16510 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Shivaram Venkataraman > > One of the `NOTE`s in the R CMD check is that we currently include a test JAR > file in SparkR which is a binary only artifact. I think we can take two steps > to address this > (a) I think we should include the source code for this in say core/src/test/ > or something like that. As far as I know the JAR file just needs to have a > single method. > (b) We should move the JAR file out of the SparkR test support and into some > other location in Spark. The trouble is that its tricky to run the test with > CRAN mode then. We could either disable the test for CRAN or download the JAR > from an external URL -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
