Github user davies commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134686363
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134671657
Thanks @sun-rui -- Change LGTM.
@davies Any other comments ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134672520
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134671515
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134672484
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134673844
[Test build #41537 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41537/consoleFull)
for PR 8276 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134515605
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134515607
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134515500
[Test build #41518 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41518/console)
for PR 8276 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134523229
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134518705
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134518738
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134523227
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134723995
Alright I'm merging this to master. Note that I'm not porting this
branch-1.5 as I think this is a relatively big change and it doesn't have any
immediate feature
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/8276
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134722643
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134722522
[Test build #41537 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41537/console)
for PR 8276 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134722641
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user sun-rui commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134217045
rebased to master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134217584
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134217519
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134219790
[Test build #41453 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41453/consoleFull)
for PR 8276 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134304021
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134306551
[Test build #41462 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41462/consoleFull)
for PR 8276 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134304025
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134305683
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134305778
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134303924
**[Test build #41453 timed
out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41453/console)**
for PR 8276 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134305750
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user sun-rui commented on a diff in the pull request:
https://github.com/apache/spark/pull/8276#discussion_r37723615
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala
---
@@ -98,27 +98,17 @@ private[r] object SQLUtils {
val bos = new
Github user sun-rui commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134066063
@shivaram , test cases for SerDe added. Now the SerDe does not support
transferring a list of different element types from R side to JVM side. Let's
leave it for
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134066574
Build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134066650
Build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134065726
[Test build #41445 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41445/consoleFull)
for PR 8276 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134064728
Build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134064672
Build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134068634
[Test build #41446 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41446/consoleFull)
for PR 8276 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134121092
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134121090
Build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134120923
[Test build #41446 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41446/console)
for PR 8276 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134105700
[Test build #41445 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41445/console)
for PR 8276 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134105748
Build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134105749
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134359798
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134359836
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134359309
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134360410
[Test build #41469 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41469/consoleFull)
for PR 8276 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134353241
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134353239
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134353089
[Test build #41462 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41462/console)
for PR 8276 at commit
Github user sun-rui commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134443211
The test passed on my machine, I don't know the reason. Anyway, add spark
context initialization into test_Serde to see if it can pass on Jenkins.
---
If your project
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134413057
@sun-rui looks like the test failures are related to this PR
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134443896
[Test build #41504 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41504/consoleFull)
for PR 8276 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134443143
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134443161
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134404199
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134404198
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134404148
[Test build #41469 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41469/console)
for PR 8276 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134477053
[Test build #41518 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41518/consoleFull)
for PR 8276 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134472240
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134472241
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134472211
**[Test build #41504 timed
out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41504/console)**
for PR 8276 at commit
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/8276#discussion_r37829318
--- Diff: R/pkg/inst/tests/test_Serde.R ---
@@ -0,0 +1,67 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+#
Github user sun-rui commented on a diff in the pull request:
https://github.com/apache/spark/pull/8276#discussion_r37832961
--- Diff: R/pkg/inst/tests/test_Serde.R ---
@@ -0,0 +1,67 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+#
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134476227
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-134476212
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/8276#discussion_r37501357
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala
---
@@ -98,27 +98,17 @@ private[r] object SQLUtils {
val bos = new
Github user sun-rui commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132571243
@shivaram, I tried to support ArrayType. By adding code like:
// Convert Seq[Any] to Array[Any]
val value =
if
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132723164
Its fine. Lets not complicate this PR with the `listToSeq` thing as that
has its own issues like you mention.
I think just adding unit tests to SerDe is a good idea
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/8276#discussion_r37465795
--- Diff: R/pkg/R/DataFrame.R ---
@@ -628,18 +628,49 @@ setMethod(dim,
setMethod(collect,
signature(x = DataFrame),
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/8276#discussion_r37465341
--- Diff: R/pkg/R/DataFrame.R ---
@@ -628,18 +628,49 @@ setMethod(dim,
setMethod(collect,
signature(x = DataFrame),
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132782565
I took a more detailed look at the code and I only had some minor comments
inline. So I think it looks pretty good but I think taking this opportunity to
add some
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/8276#discussion_r37466121
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala
---
@@ -98,27 +98,20 @@ private[r] object SQLUtils {
val bos = new
Github user sun-rui commented on a diff in the pull request:
https://github.com/apache/spark/pull/8276#discussion_r37490241
--- Diff: R/pkg/R/DataFrame.R ---
@@ -628,18 +628,49 @@ setMethod(dim,
setMethod(collect,
signature(x = DataFrame),
Github user sun-rui commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132852075
will add test cases.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user sun-rui commented on a diff in the pull request:
https://github.com/apache/spark/pull/8276#discussion_r37490835
--- Diff: R/pkg/R/DataFrame.R ---
@@ -628,18 +628,49 @@ setMethod(dim,
setMethod(collect,
signature(x = DataFrame),
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132900347
[Test build #41301 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41301/console)
for PR 8276 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132900528
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132900531
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132459112
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132459146
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132461114
[Test build #41220 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41220/consoleFull)
for PR 8276 at commit
Github user sun-rui commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132478924
@shivaram, now ArrayType in a DataFrame is still not supported, as
ArrayType's class is something like
scala.collection.mutable.WrappedArray$ofRef, it will be passed as
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132505478
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132505413
**[Test build #41220 timed
out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41220/console)**
for PR 8276 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132505479
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132493158
I see. I think a separate PR for ArrayType is fine, its just harder to
review this change if we can't test it. Its ok, I will take one more closer
look tomorrow
---
Github user sun-rui commented on a diff in the pull request:
https://github.com/apache/spark/pull/8276#discussion_r37490860
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala
---
@@ -98,27 +98,20 @@ private[r] object SQLUtils {
val bos = new
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132868272
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132868301
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132869908
[Test build #41301 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41301/consoleFull)
for PR 8276 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132198637
[Test build #41127 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41127/consoleFull)
for PR 8276 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132198235
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132198166
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
GitHub user sun-rui opened a pull request:
https://github.com/apache/spark/pull/8276
[SPARK-10048][SPARKR] Support arbitrary nested Java array in serde.
This PR:
1. supports transferring arbitrary nested array from JVM to R side in SerDe;
2. based on 1, collect()
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132311929
@sun-rui Generally, the changes looks good to me, could you add unit tests
for ArrayType? Do we want to support create create DataFrame from ArrayType
(could be another
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8276#discussion_r37332791
--- Diff: core/src/main/scala/org/apache/spark/api/r/SerDe.scala ---
@@ -210,22 +213,31 @@ private[spark] object SerDe {
writeType(dos, void)
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132297061
Thanks @sun-rui I'll take a look at this today
cc @davies
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132418166
@sun-rui We can do the `createDataFrame` in another PR, but for this PR can
we add a test case using a JSON file which has an `ArrayType` in it ?
---
If your project
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/8276#issuecomment-132440650
BTW does this fix SPARK-9302 as well or does that require `struct` support ?
---
If your project is set up for it, you can reply to this email and have your
reply
1 - 100 of 111 matches
Mail list logo
