[jira] [Created] (SPARK-16299) Capture errors from R workers in daemon.R to avoid deletion of R session temporary directory

2016-06-29 Thread Sun Rui (JIRA)
Sun Rui created SPARK-16299: --- Summary: Capture errors from R workers in daemon.R to avoid deletion of R session temporary directory Key: SPARK-16299 URL: https://issues.apache.org/jira/browse/SPARK-16299 Pr

[jira] [Created] (SPARK-16300) Capture errors from R workers in daemon.R to avoid deletion of R session temporary directory

2016-06-29 Thread Sun Rui (JIRA)
Sun Rui created SPARK-16300: --- Summary: Capture errors from R workers in daemon.R to avoid deletion of R session temporary directory Key: SPARK-16300 URL: https://issues.apache.org/jira/browse/SPARK-16300 Pr

[jira] [Created] (SPARK-16326) Evaluate sparklyr package from RStudio

2016-06-30 Thread Sun Rui (JIRA)
Sun Rui created SPARK-16326: --- Summary: Evaluate sparklyr package from RStudio Key: SPARK-16326 URL: https://issues.apache.org/jira/browse/SPARK-16326 Project: Spark Issue Type: Brainstorming

[jira] [Commented] (SPARK-16326) Evaluate sparklyr package from RStudio

2016-06-30 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356828#comment-15356828 ] Sun Rui commented on SPARK-16326: - [~rxin] [~shivaram] [~davies] [~felixcheung] [~mengxr]

[jira] [Commented] (SPARK-15799) Release SparkR on CRAN

2016-06-30 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356829#comment-15356829 ] Sun Rui commented on SPARK-15799: - we can refer to spark_install in sparklyr package. {c

[jira] [Created] (SPARK-16366) Time comparison failures in SparkR unit tests

2016-07-04 Thread Sun Rui (JIRA)
Sun Rui created SPARK-16366: --- Summary: Time comparison failures in SparkR unit tests Key: SPARK-16366 URL: https://issues.apache.org/jira/browse/SPARK-16366 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-16509) Rename window.partitionBy and window.orderBy

2016-07-12 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15374259#comment-15374259 ] Sun Rui commented on SPARK-16509: - yes, let's rename them to pass R package check. maybe

[jira] [Created] (SPARK-16522) [MESOS] Spark application throws exception on exit

2016-07-13 Thread Sun Rui (JIRA)
Sun Rui created SPARK-16522: --- Summary: [MESOS] Spark application throws exception on exit Key: SPARK-16522 URL: https://issues.apache.org/jira/browse/SPARK-16522 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-16522) [MESOS] Spark application throws exception on exit

2016-07-13 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15374757#comment-15374757 ] Sun Rui commented on SPARK-16522: - I checked SPARK-12967, this issue is different. SPARK-

[jira] [Commented] (SPARK-16522) [MESOS] Spark application throws exception on exit

2016-07-13 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376169#comment-15376169 ] Sun Rui commented on SPARK-16522: - [~mgummelt] The commit is e50efd53f073890d789a8448f850

[jira] [Commented] (SPARK-16509) Rename window.partitionBy and window.orderBy

2016-07-13 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376204#comment-15376204 ] Sun Rui commented on SPARK-16509: - I will send the PR > Rename window.partitionBy and wi

[jira] [Commented] (SPARK-15799) Release SparkR on CRAN

2016-07-14 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378919#comment-15378919 ] Sun Rui commented on SPARK-15799: - 1. Making some methods for the JVM backend public will

[jira] [Commented] (SPARK-16581) Making JVM backend calling functions public

2016-07-15 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15380307#comment-15380307 ] Sun Rui commented on SPARK-16581: - I can work on this if this is not so urgent:) > Makin

[jira] [Commented] (SPARK-16581) Making JVM backend calling functions public

2016-07-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15384187#comment-15384187 ] Sun Rui commented on SPARK-16581: - I am thinking we are going to make public the followin

[jira] [Commented] (SPARK-16464) withColumn() allows illegal creation of duplicate column names on DataFrame

2016-07-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15385418#comment-15385418 ] Sun Rui commented on SPARK-16464: - This bug in SparkR has been fixed in spark 2.0, but ex

[jira] [Closed] (SPARK-16464) withColumn() allows illegal creation of duplicate column names on DataFrame

2016-07-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui closed SPARK-16464. --- Resolution: Duplicate > withColumn() allows illegal creation of duplicate column names on DataFrame > ---

[jira] [Updated] (SPARK-11977) Support accessing a DataFrame column using its name without backticks if the name contains '.'

2016-07-21 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-11977: Summary: Support accessing a DataFrame column using its name without backticks if the name contains '.' (w

[jira] [Commented] (SPARK-12922) Implement gapply() on DataFrame in SparkR

2016-04-27 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15261339#comment-15261339 ] Sun Rui commented on SPARK-12922: - [~Narine] does AppendColumns logical operator (https:

[jira] [Created] (SPARK-14995) Add "since" tag in Roxygen documentation for SparkR API methods

2016-04-28 Thread Sun Rui (JIRA)
Sun Rui created SPARK-14995: --- Summary: Add "since" tag in Roxygen documentation for SparkR API methods Key: SPARK-14995 URL: https://issues.apache.org/jira/browse/SPARK-14995 Project: Spark Issue

[jira] [Commented] (SPARK-14995) Add "since" tag in Roxygen documentation for SparkR API methods

2016-04-28 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15263352#comment-15263352 ] Sun Rui commented on SPARK-14995: - cc [~felixcheung], [~Narine], [~shivaram]. [~felixche

[jira] [Comment Edited] (SPARK-14995) Add "since" tag in Roxygen documentation for SparkR API methods

2016-05-01 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15266031#comment-15266031 ] Sun Rui edited comment on SPARK-14995 at 5/2/16 1:07 AM: - [~felix

[jira] [Commented] (SPARK-14995) Add "since" tag in Roxygen documentation for SparkR API methods

2016-05-01 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15266031#comment-15266031 ] Sun Rui commented on SPARK-14995: - [~felixcheung] I think no need to add "spark". Just s

[jira] [Created] (SPARK-15091) Fix warnings and a failure in SparkR test cases with testthat version 1.0.1

2016-05-03 Thread Sun Rui (JIRA)
Sun Rui created SPARK-15091: --- Summary: Fix warnings and a failure in SparkR test cases with testthat version 1.0.1 Key: SPARK-15091 URL: https://issues.apache.org/jira/browse/SPARK-15091 Project: Spark

[jira] [Created] (SPARK-15159) Remove usage of HiveContext in SparkR unit test cases.

2016-05-05 Thread Sun Rui (JIRA)
Sun Rui created SPARK-15159: --- Summary: Remove usage of HiveContext in SparkR unit test cases. Key: SPARK-15159 URL: https://issues.apache.org/jira/browse/SPARK-15159 Project: Spark Issue Type: Sub-

[jira] [Updated] (SPARK-15159) Remove usage of HiveContext in SparkR.

2016-05-05 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-15159: Summary: Remove usage of HiveContext in SparkR. (was: Remove usage of HiveContext in SparkR unit test case

[jira] [Updated] (SPARK-15159) Remove usage of HiveContext in SparkR.

2016-05-05 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-15159: Description: HiveContext is to be deprecated in 2.0. Replace them with SparkSession.withHiveSupport in Spa

[jira] [Commented] (SPARK-15159) Remove usage of HiveContext in SparkR.

2016-05-05 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273655#comment-15273655 ] Sun Rui commented on SPARK-15159: - [~felixcheung], I guess you are talking about SQLConte

[jira] [Closed] (SPARK-14365) Repartition by column

2016-05-05 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui closed SPARK-14365. --- Resolution: Duplicate > Repartition by column > - > > Key: SPARK-14365 >

[jira] [Commented] (SPARK-14365) Repartition by column

2016-05-05 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273664#comment-15273664 ] Sun Rui commented on SPARK-14365: - [~dselivanov] Could you verify if SPARK-15110 can solv

[jira] [Created] (SPARK-15201) Handle integer overflow correctly in hash code computation

2016-05-07 Thread Sun Rui (JIRA)
Sun Rui created SPARK-15201: --- Summary: Handle integer overflow correctly in hash code computation Key: SPARK-15201 URL: https://issues.apache.org/jira/browse/SPARK-15201 Project: Spark Issue Type:

[jira] [Created] (SPARK-15202) add dapplyCollect() method for DataFrame in SparkR

2016-05-07 Thread Sun Rui (JIRA)
Sun Rui created SPARK-15202: --- Summary: add dapplyCollect() method for DataFrame in SparkR Key: SPARK-15202 URL: https://issues.apache.org/jira/browse/SPARK-15202 Project: Spark Issue Type: New Feat

[jira] [Updated] (SPARK-15202) add dapplyCollect() method for DataFrame in SparkR

2016-05-07 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-15202: Issue Type: Sub-task (was: New Feature) Parent: SPARK-6817 > add dapplyCollect() method for DataFr

[jira] [Commented] (SPARK-15237) SparkR corr function documentation

2016-05-09 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15277454#comment-15277454 ] Sun Rui commented on SPARK-15237: - SparkR supports two types of corr(), something like co

[jira] [Commented] (SPARK-14162) java.lang.IllegalStateException: Did not find registered driver with class oracle.jdbc.OracleDriver

2016-05-09 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15277682#comment-15277682 ] Sun Rui commented on SPARK-14162: - We met the same error. The cause is that in one worker

[jira] [Commented] (SPARK-15159) Remove usage of HiveContext in SparkR.

2016-05-11 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281100#comment-15281100 ] Sun Rui commented on SPARK-15159: - ok, let's do more to match pyspark:) > Remove usage o

[jira] [Updated] (SPARK-15159) SparkSession R API

2016-05-11 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-15159: Summary: SparkSession R API (was: Remove usage of HiveContext in SparkR.) > SparkSession R API > -

[jira] [Commented] (SPARK-15159) SparkSession R API

2016-05-11 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281104#comment-15281104 ] Sun Rui commented on SPARK-15159: - @Vijary Parmar, I changed the scope of this PR. You ca

[jira] [Commented] (SPARK-15159) SparkSession R API

2016-05-11 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281105#comment-15281105 ] Sun Rui commented on SPARK-15159: - maybe this can be split into several subtasks. > Spar

[jira] [Comment Edited] (SPARK-15159) SparkSession R API

2016-05-11 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281105#comment-15281105 ] Sun Rui edited comment on SPARK-15159 at 5/12/16 2:22 AM: -- maybe

[jira] [Commented] (SPARK-14751) SparkR fails on Cassandra map with numeric key

2016-05-12 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281516#comment-15281516 ] Sun Rui commented on SPARK-14751: - if the key type is integer or long, toString works. [~

[jira] [Commented] (SPARK-15196) Add a wrapper for dapply(repartition(col,...), ... )

2016-05-24 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299473#comment-15299473 ] Sun Rui commented on SPARK-15196: - As discussed in https://github.com/apache/spark/pull/1

[jira] [Closed] (SPARK-15196) Add a wrapper for dapply(repartition(col,...), ... )

2016-05-24 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui closed SPARK-15196. --- Resolution: Not A Problem > Add a wrapper for dapply(repartition(col,...), ... ) > --

[jira] [Commented] (SPARK-10053) SparkR isn't exporting lapply

2016-05-24 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299475#comment-15299475 ] Sun Rui commented on SPARK-10053: - as SparkDataFrame has supported dapply() similar to la

[jira] [Closed] (SPARK-10053) SparkR isn't exporting lapply

2016-05-24 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui closed SPARK-10053. --- Resolution: Won't Fix > SparkR isn't exporting lapply > - > > Key

[jira] [Created] (SPARK-15521) Add high level APIs based on dapply and gapply for easier usage

2016-05-24 Thread Sun Rui (JIRA)
Sun Rui created SPARK-15521: --- Summary: Add high level APIs based on dapply and gapply for easier usage Key: SPARK-15521 URL: https://issues.apache.org/jira/browse/SPARK-15521 Project: Spark Issue

[jira] [Commented] (SPARK-15521) Add high level APIs based on dapply and gapply for easier usage

2016-05-24 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299483#comment-15299483 ] Sun Rui commented on SPARK-15521: - cc [~felixcheung] > Add high level APIs based on dapp

[jira] [Comment Edited] (SPARK-15521) Add high level APIs based on dapply and gapply for easier usage

2016-05-24 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299483#comment-15299483 ] Sun Rui edited comment on SPARK-15521 at 5/25/16 5:30 AM: -- cc [~

[jira] [Resolved] (SPARK-10043) Add window functions into SparkR

2016-05-25 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui resolved SPARK-10043. - Resolution: Fixed Fix Version/s: 1.6.0 > Add window functions into SparkR > --

[jira] [Created] (SPARK-17519) [MESOS] Enhance robustness when ExternalShuffleService is broken

2016-09-13 Thread Sun Rui (JIRA)
Sun Rui created SPARK-17519: --- Summary: [MESOS] Enhance robustness when ExternalShuffleService is broken Key: SPARK-17519 URL: https://issues.apache.org/jira/browse/SPARK-17519 Project: Spark Issue

[jira] [Created] (SPARK-17522) [MESOS] More even distribution of executors on Mesos cluster

2016-09-13 Thread Sun Rui (JIRA)
Sun Rui created SPARK-17522: --- Summary: [MESOS] More even distribution of executors on Mesos cluster Key: SPARK-17522 URL: https://issues.apache.org/jira/browse/SPARK-17522 Project: Spark Issue Typ

[jira] [Updated] (SPARK-17522) [MESOS] More even distribution of executors on Mesos cluster

2016-09-13 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-17522: Description: The MesosCoarseGrainedSchedulerBackend launch executors in a round-robin way among accepted o

[jira] [Commented] (SPARK-17522) [MESOS] More even distribution of executors on Mesos cluster

2016-09-17 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15500078#comment-15500078 ] Sun Rui commented on SPARK-17522: - yes. It can be tuned by a config option according to t

[jira] [Commented] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571785#comment-15571785 ] Sun Rui commented on SPARK-17904: - this is a little bit tricky. You don't know the exact

[jira] [Created] (SPARK-17964) Enable SparkR with Mesos client mode

2016-10-16 Thread Sun Rui (JIRA)
Sun Rui created SPARK-17964: --- Summary: Enable SparkR with Mesos client mode Key: SPARK-17964 URL: https://issues.apache.org/jira/browse/SPARK-17964 Project: Spark Issue Type: Sub-task Com

[jira] [Created] (SPARK-17965) Enable SparkR with Mesos cluster mode

2016-10-16 Thread Sun Rui (JIRA)
Sun Rui created SPARK-17965: --- Summary: Enable SparkR with Mesos cluster mode Key: SPARK-17965 URL: https://issues.apache.org/jira/browse/SPARK-17965 Project: Spark Issue Type: Sub-task Co

[jira] [Created] (SPARK-17966) Support Spark packages with R code on Mesos

2016-10-16 Thread Sun Rui (JIRA)
Sun Rui created SPARK-17966: --- Summary: Support Spark packages with R code on Mesos Key: SPARK-17966 URL: https://issues.apache.org/jira/browse/SPARK-17966 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-11524) Support SparkR with Mesos cluster

2016-10-16 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15580960#comment-15580960 ] Sun Rui commented on SPARK-11524: - great, go ahead. Please look at the linking JIRA issue

[jira] [Created] (SPARK-17968) Support using 3rd-party R packages on Mesos

2016-10-16 Thread Sun Rui (JIRA)
Sun Rui created SPARK-17968: --- Summary: Support using 3rd-party R packages on Mesos Key: SPARK-17968 URL: https://issues.apache.org/jira/browse/SPARK-17968 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-11524) Support SparkR with Mesos cluster

2016-10-16 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15581268#comment-15581268 ] Sun Rui commented on SPARK-11524: - for cluster mode, the R script needs to transferred to

[jira] [Comment Edited] (SPARK-11524) Support SparkR with Mesos cluster

2016-10-16 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15581268#comment-15581268 ] Sun Rui edited comment on SPARK-11524 at 10/17/16 5:31 AM: --- for

[jira] [Commented] (SPARK-12922) Implement gapply() on DataFrame in SparkR

2016-02-22 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15158109#comment-15158109 ] Sun Rui commented on SPARK-12922: - [~Narine], yes this depends on https://issues.apache.

[jira] [Comment Edited] (SPARK-12922) Implement gapply() on DataFrame in SparkR

2016-02-22 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15158109#comment-15158109 ] Sun Rui edited comment on SPARK-12922 at 2/23/16 1:44 AM: -- [~Nar

[jira] [Commented] (SPARK-12919) Implement dapply() on DataFrame in SparkR

2016-02-22 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15158112#comment-15158112 ] Sun Rui commented on SPARK-12919: - working on this. > Implement dapply() on DataFrame in

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-02-24 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15162877#comment-15162877 ] Sun Rui commented on SPARK-13178: - Remember to clean the code at https://github.com/apac

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

2016-02-28 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15171294#comment-15171294 ] Sun Rui commented on SPARK-13525: - Is the path of your Rscript in $PATH? If not, you can

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

2016-03-01 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15174720#comment-15174720 ] Sun Rui commented on SPARK-13525: - the interactive R session is for your driver, Rscript

[jira] [Commented] (SPARK-13573) Open SparkR APIs (R package) to allow better 3rd party usage

2016-03-01 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15174726#comment-15174726 ] Sun Rui commented on SPARK-13573: - [~chipsenkbeil] glad to know Toree is to support Spark

[jira] [Commented] (SPARK-13573) Open SparkR APIs (R package) to allow better 3rd party usage

2016-03-08 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15184806#comment-15184806 ] Sun Rui commented on SPARK-13573: - [~chipsenkbeil] It seems that Toree runs SparkR kernel

[jira] [Created] (SPARK-13812) Fix SparkR lint-r test errors

2016-03-10 Thread Sun Rui (JIRA)
Sun Rui created SPARK-13812: --- Summary: Fix SparkR lint-r test errors Key: SPARK-13812 URL: https://issues.apache.org/jira/browse/SPARK-13812 Project: Spark Issue Type: Bug Components: Spa

[jira] [Created] (SPARK-13813) Fix SparkR lint-r test errors

2016-03-10 Thread Sun Rui (JIRA)
Sun Rui created SPARK-13813: --- Summary: Fix SparkR lint-r test errors Key: SPARK-13813 URL: https://issues.apache.org/jira/browse/SPARK-13813 Project: Spark Issue Type: Bug Components: Spa

[jira] [Closed] (SPARK-13813) Fix SparkR lint-r test errors

2016-03-10 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui closed SPARK-13813. --- Resolution: Duplicate > Fix SparkR lint-r test errors > - > > Key

[jira] [Created] (SPARK-13905) Change implementation of as.data.frame() to avoid conflict with the ones in the R base package

2016-03-15 Thread Sun Rui (JIRA)
Sun Rui created SPARK-13905: --- Summary: Change implementation of as.data.frame() to avoid conflict with the ones in the R base package Key: SPARK-13905 URL: https://issues.apache.org/jira/browse/SPARK-13905

[jira] [Updated] (SPARK-13905) Change signature of as.data.frame() to be consistent with the R base package

2016-03-18 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-13905: Description: change the signature of as.data.frame() to be consistent with that in the R base package to me

[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame

2016-03-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200940#comment-15200940 ] Sun Rui commented on SPARK-12148: - An issue reported in the Spark user list may be relate

[jira] [Commented] (SPARK-13905) Change implementation of as.data.frame() to avoid conflict with the ones in the R base package

2016-03-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200938#comment-15200938 ] Sun Rui commented on SPARK-13905: - This issue is not related to as.data.frame() in SparkR

[jira] [Comment Edited] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame

2016-03-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200940#comment-15200940 ] Sun Rui edited comment on SPARK-12148 at 3/18/16 3:33 AM: -- An is

[jira] [Updated] (SPARK-13905) Change signature of as.data.frame() to be consistent with the R base package

2016-03-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-13905: Summary: Change signature of as.data.frame() to be consistent with the R base package (was: Change impleme

[jira] [Updated] (SPARK-13905) Change signature of as.data.frame() to be consistent with the R base package

2016-03-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-13905: Description: (was: SparkR provides a method as.data.frame() to collect a SparkR DataFrame into a local

[jira] [Commented] (SPARK-14037) count(df) is very slow for dataframe constrcuted using SparkR::createDataFrame

2016-03-21 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205790#comment-15205790 ] Sun Rui commented on SPARK-14037: - If possible, just use read.df() to load a DataFrame fr

[jira] [Comment Edited] (SPARK-14037) count(df) is very slow for dataframe constrcuted using SparkR::createDataFrame

2016-03-21 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205790#comment-15205790 ] Sun Rui edited comment on SPARK-14037 at 3/22/16 4:49 AM: -- If po

[jira] [Commented] (SPARK-14006) Builds of 1.6 branch fail R style check

2016-03-21 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205832#comment-15205832 ] Sun Rui commented on SPARK-14006: - [~yhuai] Do you mean a backport PR to branch 1.6? > B

[jira] [Commented] (SPARK-14037) count(df) is very slow for dataframe constrcuted using SparkR::createDataFrame

2016-03-21 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205906#comment-15205906 ] Sun Rui commented on SPARK-14037: - Currently, createDataFrame does not allow specifying p

[jira] [Commented] (SPARK-14074) Do not use install_github in SparkR build

2016-03-22 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207780#comment-15207780 ] Sun Rui commented on SPARK-14074: - yes, unstable source may cause un-expected test failur

[jira] [Commented] (SPARK-14037) count(df) is very slow for dataframe constrcuted using SparkR::createDataFrame

2016-03-22 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207916#comment-15207916 ] Sun Rui commented on SPARK-14037: - spark 1.6.1 release, standalone mode. bin/sparkR --mas

[jira] [Commented] (SPARK-8041) Consistently pass SparkR library directory to SparkR application

2015-06-30 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14607877#comment-14607877 ] Sun Rui commented on SPARK-8041: sorry, this JIRA is obsolete as we are addressing SPARK-6

[jira] [Closed] (SPARK-8041) Consistently pass SparkR library directory to SparkR application

2015-06-30 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui closed SPARK-8041. -- Resolution: Duplicate This issue is covered by SPARK-6797 > Consistently pass SparkR library directory to Spark

[jira] [Commented] (SPARK-6833) Extend `addPackage` so that any given R file can be sourced in the worker before functions are run.

2015-07-01 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609698#comment-14609698 ] Sun Rui commented on SPARK-6833: I tested with --files, that works. So it seems we can clo

[jira] [Created] (SPARK-8894) Example code errors in SparkR documentation

2015-07-08 Thread Sun Rui (JIRA)
Sun Rui created SPARK-8894: -- Summary: Example code errors in SparkR documentation Key: SPARK-8894 URL: https://issues.apache.org/jira/browse/SPARK-8894 Project: Spark Issue Type: Bug Compo

[jira] [Created] (SPARK-8897) SparkR DataFrame fail to return data of float type

2015-07-08 Thread Sun Rui (JIRA)
Sun Rui created SPARK-8897: -- Summary: SparkR DataFrame fail to return data of float type Key: SPARK-8897 URL: https://issues.apache.org/jira/browse/SPARK-8897 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-8952) JsonFile() of SQLContext display improper warning message for a S3 path

2015-07-09 Thread Sun Rui (JIRA)
Sun Rui created SPARK-8952: -- Summary: JsonFile() of SQLContext display improper warning message for a S3 path Key: SPARK-8952 URL: https://issues.apache.org/jira/browse/SPARK-8952 Project: Spark Is

[jira] [Updated] (SPARK-8952) JsonFile() of SQLContext display improper warning message for a S3 path

2015-07-09 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-8952: --- Description: This is an issue reported by Ben Spark . {quote} Spark 1.4 deployed on AWS EMR "jsonFile" is w

[jira] [Commented] (SPARK-8952) JsonFile() of SQLContext display improper warning message for a S3 path

2015-07-09 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620232#comment-14620232 ] Sun Rui commented on SPARK-8952: jsonFile() and parquetFile() will call R normalizePath fu

[jira] [Closed] (SPARK-8897) SparkR DataFrame fail to return data of float type

2015-07-09 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui closed SPARK-8897. -- duplicated > SparkR DataFrame fail to return data of float type > -

[jira] [Commented] (SPARK-8952) JsonFile() of SQLContext display improper warning message for a S3 path

2015-07-13 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14624300#comment-14624300 ] Sun Rui commented on SPARK-8952: Currently normalizePath() is used in several places allo

[jira] [Commented] (SPARK-8844) head/collect is broken in SparkR

2015-07-14 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14626273#comment-14626273 ] Sun Rui commented on SPARK-8844: This is a bug about reading empty DataFrame. will submit

[jira] [Created] (SPARK-11414) Forgot to update usage of "spark.sparkr.r.command" in RRDD in the PR for SPARK-10971

2015-10-29 Thread Sun Rui (JIRA)
Sun Rui created SPARK-11414: --- Summary: Forgot to update usage of "spark.sparkr.r.command" in RRDD in the PR for SPARK-10971 Key: SPARK-11414 URL: https://issues.apache.org/jira/browse/SPARK-11414 Project: S

[jira] [Commented] (SPARK-11414) Forgot to update usage of "spark.sparkr.r.command" in RRDD in the PR for SPARK-10971

2015-10-29 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14981830#comment-14981830 ] Sun Rui commented on SPARK-11414: - Thanks to Michael Sannella (https://github.com/msannel

[jira] [Commented] (SPARK-11340) Support setting driver properties when starting Spark from R programmatically or from RStudio

2015-11-01 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14984778#comment-14984778 ] Sun Rui commented on SPARK-11340: - I have posted the message about this enhancement in th

[jira] [Created] (SPARK-11524) Support SparkR with Mesos cluster

2015-11-04 Thread Sun Rui (JIRA)
Sun Rui created SPARK-11524: --- Summary: Support SparkR with Mesos cluster Key: SPARK-11524 URL: https://issues.apache.org/jira/browse/SPARK-11524 Project: Spark Issue Type: New Feature Com

[jira] [Created] (SPARK-11525) Support spark packages containing R source code in Standalone mode

2015-11-05 Thread Sun Rui (JIRA)
Sun Rui created SPARK-11525: --- Summary: Support spark packages containing R source code in Standalone mode Key: SPARK-11525 URL: https://issues.apache.org/jira/browse/SPARK-11525 Project: Spark Iss

  1   2   3   4   >