[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-17 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15421 @falaki @wangmiao1981 Can we summarize the discussion around the `NegativeArraySizeException` ? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #15471: [WIP][SPARK-17919] Make timeout to RBackend configurable...

2016-10-14 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15471 @falaki Thats an interesting approach to try -- the other thing to try might be to use a separate connection to send back results. i.e. as soon as JVM gets the request it returns `OK` or some

[GitHub] spark issue #15446: [SPARK-17882][SparkR] Fix swallowed exception in RBacken...

2016-10-13 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15446 I updated the JIRA. @jrshust could you close this PR ? We don't have permission to close PRs and only PR authors can do it --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #15446: [SPARK-17882][SparkR] Fix swallowed exception in RBacken...

2016-10-12 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15446 @felixcheung Resolving the JIRA and assigning it to @jrshust sounds good to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark issue #15446: [SPARK-17882][SparkR] Fix swallowed exception in RBacken...

2016-10-12 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15446 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #15433: [SPARK-17822] Use weak reference in JVMObjectTrac...

2016-10-12 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/15433#discussion_r83043023 --- Diff: core/src/main/scala/org/apache/spark/api/r/RBackendHandler.scala --- @@ -263,18 +264,19 @@ private[r] object JVMObjectTracker

[GitHub] spark issue #15446: [SPARK-17882][SparkR] Fix swallowed exception in RBacken...

2016-10-11 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15446 cc @falaki Is this also a part of #15375 ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15446: [SPARK-17882][SparkR] Fix swallowed exception in RBacken...

2016-10-11 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15446 Thanks @jrshust for the PR. Jenkins, ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #15375: [SPARK-17790][SPARKR] Support for parallelizing R data.f...

2016-10-11 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15375 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-11 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15421 Yeah it should be fine to merge this into branch-2.0 when it is ready --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15433: [SPARK-17822] Use weak reference in JVMObjectTracker.obj...

2016-10-11 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15433 @techaddict Thanks for the PR. The test failures seem related to this patch -- Could you check why they are failing ? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #15328: [SPARKR][SPARK-17762] invokeJava fails when serialized a...

2016-10-04 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15328 I'm not sure there is an easy way to handle this on the Java side - The max array size in the JVM is fixed as INT_MAX and AFAIK there isn't any way around it. Taking a step back

[GitHub] spark issue #15340: [SPARKR][DOC] minor formatting and output cleanup for R ...

2016-10-04 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15340 Merging into master and branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15340: [SPARKR][DOC] minor formatting and output cleanup for R ...

2016-10-04 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15340 LGTM. Thanks @felixcheung --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15328: [SPARKR][SPARK-17762] invokeJava fails when serialized a...

2016-10-03 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15328 Hmm - wont each object need to be deserialized on the Java side ? Will that object deserialize succeed ? Or to put it another way, can we add an end-to-end test that will exercise this code path

[GitHub] spark issue #15328: [WIP][SPARKR][SPARK-17762] invokeJava fails when seriali...

2016-10-03 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15328 Can we add a unit test for this ? Also I'm not sure if my earlier comment on Java array sizes was addressed ? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #15328: [WIP][SPARKR][SPARK-17762] invokeJava fails when seriali...

2016-10-03 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15328 AFAIK java arrays also have the same limitation - So if we convert lists to arrays, this will also fail on the JVM side ? --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #15232: [SPARK-17499][SPARKR][FOLLOWUP] Suppress warnings from i...

2016-09-24 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15232 Thanks @HyukjinKwon - Fix looks good. But isn't it better to do the check before we do the na.omit / as.integer ? Or will that miss some cases ? --- If your project is set up for it, you can

[GitHub] spark issue #15200: Skip building R vignettes if Spark is not built

2016-09-23 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15200 Yeah - so I'm thinking we should just auto-generate this and check in the html file in git. Its not that big. When somebody updates the vignette we need to remind them to regenerate it though

[GitHub] spark pull request #15223: Set R package version number along with mvn

2016-09-23 Thread shivaram
GitHub user shivaram opened a pull request: https://github.com/apache/spark/pull/15223 Set R package version number along with mvn ## What changes were proposed in this pull request? This PR sets the R package version while tagging releases. Note that since R doesn't

[GitHub] spark issue #15223: Set R package version number along with mvn

2016-09-23 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15223 cc @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #15200: Skip building R vignettes if Spark is not built

2016-09-22 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15200 The only downside of this is that we won't be uploading our vignette to the website. I think this is fine -- Any thoughts @junyangq @felixcheung cc @rxin --- If your project is set up

[GitHub] spark pull request #15200: Skip building R vignettes if Spark is not built

2016-09-22 Thread shivaram
GitHub user shivaram opened a pull request: https://github.com/apache/spark/pull/15200 Skip building R vignettes if Spark is not built ## What changes were proposed in this pull request? When we build the docs separately we don't have the JAR files from the Spark build

[GitHub] spark pull request #15051: [SPARK-17499][SparkR][ML][MLLib] make the default...

2016-09-17 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/15051#discussion_r79297229 --- Diff: R/pkg/R/mllib.R --- @@ -694,8 +694,14 @@ setMethod("predict", signature(object = "KMeansModel"), #' } #' @note s

[GitHub] spark issue #15131: [SPARK-17577][SparkR] SparkR support add files to Spark ...

2016-09-17 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15131 It looks like `addFile` isn't working on Windows because we try to convert the windows file path into a URI and that fails. Not sure what the fix is in this case. cc @HyukjinKwon who

[GitHub] spark issue #15100: [SPARK-17317][SparkR] Add SparkR vignette to branch 2.0

2016-09-15 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15100 @junyangq We need to manually close this PR as github doesn't pick up commits not merged to master. Can you close this when you get a chance ? --- If your project is set up for it, you can reply

[GitHub] spark issue #15100: [SPARK-17317][SparkR] Add SparkR vignette to branch 2.0

2016-09-15 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15100 Merging this into branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #15100: [SPARK-17317][SparkR] Add SparkR vignette to bran...

2016-09-14 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/15100#discussion_r78799049 --- Diff: R/pkg/vignettes/sparkr-vignettes.Rmd --- @@ -0,0 +1,653 @@ +--- +title: "SparkR - Practical Guide" +output: + htm

[GitHub] spark issue #15097: [SPARK-17540][SparkR][Spark Core] fix SparkR array serde...

2016-09-14 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15097 Thanks @WeichenXu123 Can we add some test cases for this ? Also was this fix related to any problem you saw ? If so we can also include that in the test cases etc. --- If your project is set up

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-09-14 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/1 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-09-14 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/1 Yeah the R change looks fine. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-09-14 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/1 @eyalfa Yes - we get the schema from Scala and assign the column names from that [1]. I think @hvanhovell knows more about the compatibility requirements Also @HyukjinKwon

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-13 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14980 @junyangq As we discussed before, lets open a new PR for 2.0 ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-13 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14980 Thanks @junyangq and @felixcheung - Merging this into master once the AppVeyor check passes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-09-13 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/1 The easiest way to debug the R test case is to see what has changed for this particular `select` query that is failing and then update the test case. If you have compiled with `-Psparkr

[GitHub] spark pull request #15051: [SPARK-17499][ML][MLLib] make the default params ...

2016-09-12 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/15051#discussion_r78406032 --- Diff: R/pkg/R/mllib.R --- @@ -694,8 +694,8 @@ setMethod("predict", signature(object = "KMeansModel"), #' } #' @note s

[GitHub] spark pull request #15051: [SPARK-17499][ML][MLLib] make the default params ...

2016-09-11 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/15051#discussion_r78301160 --- Diff: R/pkg/R/mllib.R --- @@ -694,8 +694,8 @@ setMethod("predict", signature(object = "KMeansModel"), #' } #' @note s

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-11 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14980 Yeah I think it'll be good to do a separate PR and make sure it can build corresponding to the Scala code in branch-2.0 etc. But lets do it after all the comments here are addressed

[GitHub] spark issue #15015: [SPARK-16445][MLlib][SparkR] Fix @return description for...

2016-09-10 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15015 Thanks @keypointt - Merging into master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15029: [DUMMY][SPARKR] Test AppVeyor with a change in R/

2016-09-09 Thread shivaram
Github user shivaram closed the pull request at: https://github.com/apache/spark/pull/15029 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #15031: [DUMMY][SPARKR] Test AppVeyor with a change in R/

2016-09-09 Thread shivaram
Github user shivaram closed the pull request at: https://github.com/apache/spark/pull/15031 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #15031: [DUMMY][SPARKR] Test AppVeyor with a change in R/

2016-09-09 Thread shivaram
GitHub user shivaram opened a pull request: https://github.com/apache/spark/pull/15031 [DUMMY][SPARKR] Test AppVeyor with a change in R/ Dummy pull request to `branch-1.6` test AppVeyor integration. Do not merge You can merge this pull request into a Git repository by running

[GitHub] spark pull request #15029: [DUMMY][SPARKR] Test AppVeyor with a change in R/

2016-09-09 Thread shivaram
GitHub user shivaram opened a pull request: https://github.com/apache/spark/pull/15029 [DUMMY][SPARKR] Test AppVeyor with a change in R/ Dummy pull request to test AppVeyor integration. Do not merge You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-09 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14980 @junyangq @felixcheung Is this good to be merged ? Also some of ML algorithms are not present in 2.0.0, so we might need to remove them from the vignette when backporting ? --- If your project

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-08 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14980 Hmm the Jenkins error was ``` find: cannot delete `pkg/vignettes': Directory not empty ``` --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #15015: [SPARK-16445][MLlib][SparkR] Fix @return description for...

2016-09-08 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15015 Created https://issues.apache.org/jira/browse/INFRA-12590 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #15015: [SPARK-16445][MLlib][SparkR] Fix @return description for...

2016-09-08 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15015 I think we need to file an INFRA ticket like https://issues.apache.org/jira/browse/INFRA-11294 -- I'll file one now --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #15015: [SPARK-16445][MLlib][SparkR] Fix @return description for...

2016-09-08 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15015 @HyukjinKwon I thought it should have run the tests ? Any ideas why its not getting picked up ? Is there some other set of steps that we need to do ? (I'm now investigating how it works for Apache

[GitHub] spark issue #14784: [SPARK-17210][SPARKR] sparkr.zip is not distributed to e...

2016-09-08 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14784 @zjffdu Sorry for the delay. I think the change looks pretty good. We could just add a `logWarning` in case we find a collision ? Also could you bring this up to date with the master branch

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-08 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14980 It looks like the tests are failing because of a new CRAN check `NOTE` about the file `derby.log` not being cleaned up from `vignettes/`. I also noticed that `people.parquet` is created in my

[GitHub] spark issue #14859: [SPARK-17200][PROJECT INFRA][BUILD][SPARKR] Automate bui...

2016-09-08 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14859 Alright, I'm merging this into master. Lets see how this goes in the next few days and we can fine tune this with some follow up PRs if necessary. --- If your project is set up for it, you can

[GitHub] spark issue #15010: [SPARK-17442][SPARKR] Additional arguments in write.df a...

2016-09-08 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15010 Merging into master, branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15010: [SPARK-17442][SPARKR] Additional arguments in write.df a...

2016-09-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15010 LGTM - @falaki any more comments ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15010: [SPARK-17442][SPARKR] Additional arguments in write.df a...

2016-09-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15010 Thanks - the new test looks good --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15007: [SPARKR][SPARK-17442] Additional arguments in write.df a...

2016-09-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15007 @falaki We can use the PR at https://github.com/apache/spark/pull/15010 and close this one ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #14859: [SPARK-17200][PROJECT INFRA][BUILD][SPARKR] Autom...

2016-09-07 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14859#discussion_r77945551 --- Diff: appveyor.yml --- @@ -0,0 +1,51 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements

[GitHub] spark pull request #15010: [SPARK-17442][SPARKR] Additional arguments in wri...

2016-09-07 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/15010#discussion_r77944989 --- Diff: R/pkg/inst/tests/testthat/test_sparkSQL.R --- @@ -243,7 +243,15 @@ test_that("read csv as DataFrame", { expect_equal(count(with

[GitHub] spark pull request #14859: [SPARK-17200][PROJECT INFRA][BUILD][SPARKR] Autom...

2016-09-07 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14859#discussion_r77944576 --- Diff: appveyor.yml --- @@ -0,0 +1,51 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements

[GitHub] spark pull request #15010: [SPARK-17442][SPARKR] Additional arguments in wri...

2016-09-07 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/15010#discussion_r77944380 --- Diff: R/pkg/inst/tests/testthat/test_sparkSQL.R --- @@ -243,7 +243,15 @@ test_that("read csv as DataFrame", { expect_equal(count(with

[GitHub] spark issue #15008: [SPARK-17339][CORE][BRANCH-2.0] Do not use path to get a...

2016-09-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15008 @HyukjinKwon I think you might need to manually close this PR --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #14859: [SPARK-17200][PROJECT INFRA][BUILD][SPARKR] Automate bui...

2016-09-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14859 Thanks. I'll merge this once Jenkins passes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15008: [SPARK-17339][CORE][BRANCH-2.0] Do not use path to get a...

2016-09-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15008 Thanks - Merging this to branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15007: [SPARKR][SPARK-17442] Additional arguments in wri...

2016-09-07 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/15007#discussion_r77934359 --- Diff: R/pkg/R/DataFrame.R --- @@ -2635,7 +2635,7 @@ setMethod("write.df", write <- callJMethod(df@sdf, "write")

[GitHub] spark issue #14859: [SPARK-17200][PROJECT INFRA][BUILD][SPARKR] Automate bui...

2016-09-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14859 LGTM pending the inline comment about hadoop-2.6 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #14859: [SPARK-17200][PROJECT INFRA][BUILD][SPARKR] Autom...

2016-09-07 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14859#discussion_r77920251 --- Diff: appveyor.yml --- @@ -0,0 +1,51 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements

[GitHub] spark pull request #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-07 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14980#discussion_r77873869 --- Diff: R/create-docs.sh --- @@ -43,4 +45,7 @@ Rscript -e 'libDir <- "../../lib"; library(SparkR, lib.loc=libDir); library(knit

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14980 @junyangq - Looks like we should also load SparkR in `create-docs.sh` -- The error in Jenkins is ``` Quitting from lines 22-23 (sparkr-vignettes.Rmd) Error in library(SparkR

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14980 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14993: [SPARK-15509][Follow-up][ML][SparkR] R MLlib algorithms ...

2016-09-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14993 cc @keypointt --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14980 FYI - @shaneknapp is helping us get the requisite packages installed to build this on Jenkins. https://issues.apache.org/jira/browse/SPARK-17420 is the relevant JIRA --- If your project is set

[GitHub] spark issue #14859: [SPARK-17200][PROJECT INFRA][BUILD][SPARKR] Automate bui...

2016-09-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14859 @HyukjinKwon Given that the build is green after #14960 is this good to go ? Any other comments @dongjoon-hyun @srowen --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14980 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14783: SPARK-16785 R dapply doesn't return array or raw columns

2016-09-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14783 Thanks for the update. LGTM. Merging this to master and branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #14960: [SPARK-17339][SPARKR][CORE] Fix some R tests and use Pat...

2016-09-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14960 Looks like it passed ! LGTM pending our Jenkins tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #14783: SPARK-16785 R dapply doesn't return array or raw ...

2016-09-06 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14783#discussion_r77763061 --- Diff: R/pkg/R/utils.R --- @@ -697,3 +697,18 @@ is_master_local <- function(master) { is_sparkR_shell <- function() { grepl("

[GitHub] spark pull request #14783: SPARK-16785 R dapply doesn't return array or raw ...

2016-09-06 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14783#discussion_r77762250 --- Diff: R/pkg/R/utils.R --- @@ -697,3 +697,18 @@ is_master_local <- function(master) { is_sparkR_shell <- function() { grepl("

[GitHub] spark issue #14783: SPARK-16785 R dapply doesn't return array or raw columns

2016-09-06 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14783 Sorry for the delay @clarkfitzg - The code change looks pretty good to me. I just had one question about mixed type columns. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request #14783: SPARK-16785 R dapply doesn't return array or raw ...

2016-09-06 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14783#discussion_r77757807 --- Diff: R/pkg/R/utils.R --- @@ -697,3 +697,18 @@ is_master_local <- function(master) { is_sparkR_shell <- function() { grepl("

[GitHub] spark pull request #14960: [SPARK-17339][SPARKR][CORE] Fix some R tests and ...

2016-09-06 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14960#discussion_r77751910 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -992,7 +992,7 @@ class SparkContext(config: SparkConf) extends Logging

[GitHub] spark issue #14960: [SPARK-17339][SPARKR][CORE] Fix some R tests and use Pat...

2016-09-06 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14960 Thanks @HyukjinKwon - LGTM but for a minor comment I had inline. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #14960: [SPARK-17339][SPARKR][CORE] Fix some R tests and ...

2016-09-06 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14960#discussion_r77665699 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -992,7 +992,7 @@ class SparkContext(config: SparkConf) extends Logging

[GitHub] spark pull request #14960: [WIP][SPARK-17339][SPARKR][CORE] Fix Windows path...

2016-09-05 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14960#discussion_r77573002 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1900,7 +1900,20 @@ private[spark] object Utils extends Logging

[GitHub] spark pull request #14960: [WIP][SPARK-17339][SPARKR][CORE] Fix Windows path...

2016-09-05 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14960#discussion_r77572574 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1900,7 +1900,20 @@ private[spark] object Utils extends Logging

[GitHub] spark pull request #14960: [WIP][SPARK-17339][SPARKR][CORE] Fix Windows path...

2016-09-05 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14960#discussion_r77544947 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1900,7 +1900,20 @@ private[spark] object Utils extends Logging

[GitHub] spark pull request #14960: [WIP][SPARK-17339][SPARKR][CORE] Fix Windows path...

2016-09-05 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14960#discussion_r77544303 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1900,7 +1900,20 @@ private[spark] object Utils extends Logging

[GitHub] spark pull request #14960: [WIP][SPARK-17339][SPARKR][CORE] Fix Windows path...

2016-09-05 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14960#discussion_r77543535 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1900,7 +1900,20 @@ private[spark] object Utils extends Logging

[GitHub] spark issue #13690: [SPARK-15767][R][ML] Decision Tree Regression wrapper in...

2016-09-03 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13690 @vectorijk Is this ready for another round of review ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn...

2016-09-03 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14433#discussion_r77443065 --- Diff: core/src/main/scala/org/apache/spark/internal/Logging.scala --- @@ -135,7 +135,8 @@ private[spark] trait Logging { val replLevel

[GitHub] spark issue #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn't work

2016-09-03 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14433 Merged into master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #14942: [SparkR][Minor] Fix docs for sparkR.session and count

2016-09-02 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14942 Thanks - Merging this into master, branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #14942: [SparkR][Minor] Fix docs for sparkR.session and count

2016-09-02 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14942 cc @felixcheung --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #14939: [SPARK-17376][SPARKR] followup - change since version

2016-09-02 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14939 Ah thanks - I didn't notice this while merging the earlier PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #14934: [SPARKR][DOC] regexp_extract should doc that it returns ...

2016-09-02 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14934 LGTM. Merging into master, branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14934: [SPARKR][DOC] regexp_extract should doc that it r...

2016-09-02 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14934#discussion_r77381083 --- Diff: R/pkg/R/functions.R --- @@ -2876,7 +2876,8 @@ setMethod("randn", signature(seed = "numeric"), #' regexp_extract

[GitHub] spark issue #14935: [SPARK-17376][SPARKR] Spark version should be available ...

2016-09-02 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14935 Merged into master, branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #14934: [SPARKR][DOC] regexp_extract should doc that it r...

2016-09-02 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14934#discussion_r77375452 --- Diff: R/pkg/R/functions.R --- @@ -2876,7 +2876,8 @@ setMethod("randn", signature(seed = "numeric"), #' regexp_extract

[GitHub] spark issue #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn't work

2016-09-02 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14433 LGTM. Thanks @wangmiao1981 -- I'll keep this open for a bit to see if there are any more comments. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #14935: [SPARK-17376][SPARKR] Spark version should be available ...

2016-09-02 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14935 LGTM. I actually think this might also be useful in 2.0.1 ? cc @junyangq --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark issue #13584: [SPARK-15509][ML][SparkR] R MLlib algorithms should supp...

2016-09-01 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13584 LGTM - @felixcheung Feel free to merge when its ready --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn...

2016-09-01 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14433#discussion_r77261262 --- Diff: core/src/main/scala/org/apache/spark/internal/Logging.scala --- @@ -135,7 +136,12 @@ private[spark] trait Logging { val replLevel

<    1   2   3   4   5   6   7   8   9   10   >