[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-05 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17159 I think it's a good idea to get SparkR `rbind` to match behavior of R `data.frame` `rbind`. We should clearly indicate the difference between SparkR `union` and `rbind` then --- If your

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-05 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/17159 @felixcheung OK, did not know it was by design. It does seem that the `union` behavior is similar to R's SQL (in `sqldf`), but as you pointed out, the overload method `rbind` is different

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-04 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17159 hmm... this is somewhat by design - `union` could take in 2 DataFrames that might not match in column names or type. In that case values in one of the DataFrame will be coerced to make things

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17159 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17159 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73897/ Test PASSed. ---

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17159 **[Test build #73897 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73897/testReport)** for PR 17159 at commit

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17159 **[Test build #73897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73897/testReport)** for PR 17159 at commit

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17159 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17159 **[Test build #73895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73895/testReport)** for PR 17159 at commit

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17159 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73895/ Test FAILed. ---

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17159 **[Test build #73895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73895/testReport)** for PR 17159 at commit

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17159 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73888/ Test FAILed. ---

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17159 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17159 **[Test build #73888 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73888/testReport)** for PR 17159 at commit

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17159 **[Test build #73888 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73888/testReport)** for PR 17159 at commit

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-03 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/17159 The current implementation accepts data frames with different schemas. See issues below: ``` df <- createDataFrame(data.frame(name = c("Michael", "Andy", "Justin"), age = c(1, 30,