Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17848
**[Test build #79942 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79942/testReport)**
for PR 17848 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17848
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17848
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79942/
Test PASSed.
---
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/18699#discussion_r129447337
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala
---
@@ -177,9 +177,11 @@ private[shared] object
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/18699
cc @jkbradley
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18655#discussion_r129486171
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala
---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18655#discussion_r129486273
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/arrow/ArrowWriterSuite.scala
---
@@ -0,0 +1,247 @@
+/*
+ * Licensed to the Apache
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18655#discussion_r129484715
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowConverters.scala
---
@@ -77,95 +59,55 @@ private[sql] object
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/18655
yes let leave decimal support for folllow-ups
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18708
**[Test build #79954 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79954/testReport)**
for PR 18708 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18708
**[Test build #79954 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79954/testReport)**
for PR 18708 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18708
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79954/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18708
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18655#discussion_r129477058
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowConverters.scala
---
@@ -77,95 +59,55 @@ private[sql] object ArrowConverters
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18735
**[Test build #79949 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79949/testReport)**
for PR 18735 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18735
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79949/
Test PASSed.
---
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18655#discussion_r129477483
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowConverters.scala
---
@@ -77,95 +59,55 @@ private[sql] object ArrowConverters {
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18735
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/18735
cc @tgravescs @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18640
**[Test build #79951 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79951/testReport)**
for PR 18640 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18640
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79951/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18640
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18655
**[Test build #79955 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79955/testReport)**
for PR 18655 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/18655
+1 on holding off for `DecimalType` support
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18655#discussion_r129480076
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowConverters.scala
---
@@ -77,95 +59,55 @@ private[sql] object ArrowConverters
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18711
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18711
**[Test build #79952 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79952/testReport)**
for PR 18711 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18711
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79952/
Test PASSed.
---
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18655#discussion_r129480291
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala
---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18655#discussion_r129480466
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/arrow/ArrowWriterSuite.scala
---
@@ -0,0 +1,247 @@
+/*
+ * Licensed to the
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/18503
LGTM, pending tests
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18503
**[Test build #79956 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79956/testReport)**
for PR 18503 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/18735
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/18725
Are you sure it is caused by rand?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/18655#discussion_r129482346
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/arrow/ArrowConvertersSuite.scala
---
@@ -857,6 +857,449 @@ class ArrowConvertersSuite
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/18655#discussion_r129481638
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala
---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18702
**[Test build #79953 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79953/testReport)**
for PR 18702 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18702
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18702
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79953/
Test PASSed.
---
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18726
I'm not sure what you mean here. The current test code captures references
to contexts that are not cleaned up by the superclass. If you remove it, what
cleans up the contexts that are instantiated
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/18702
@rxin, do you have any thought on this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/18503#discussion_r129228541
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala ---
@@ -479,6 +479,61 @@ class StreamSuite extends StreamTest {
Github user caneGuy commented on the issue:
https://github.com/apache/spark/pull/18723
@kiszk Thanks for your time.
Yes,you are right.
1.OOM happens at io.netty.buffer.CompositeByteBuf.allocBuffer when one
executor(client) fetch blocks from an other(server) , oom happens at
Github user caneGuy commented on a diff in the pull request:
https://github.com/apache/spark/pull/18723#discussion_r129229389
--- Diff:
core/src/main/scala/org/apache/spark/util/io/ChunkedByteBuffer.scala ---
@@ -66,7 +66,7 @@ private[spark] class ChunkedByteBuffer(var chunks:
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18071
Merged to master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18722
**[Test build #3849 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3849/testReport)**
for PR 18722 at commit
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/18722#discussion_r129232934
--- Diff: docs/quick-start.md ---
@@ -421,16 +421,15 @@ $ YOUR_SPARK_HOME/bin/spark-submit \
Lines with a: 46, Lines with b: 23
{%
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/18722#discussion_r129232815
--- Diff: docs/quick-start.md ---
@@ -421,16 +421,15 @@ $ YOUR_SPARK_HOME/bin/spark-submit \
Lines with a: 46, Lines with b: 23
{%
Github user lizhaoch commented on a diff in the pull request:
https://github.com/apache/spark/pull/18722#discussion_r129221078
--- Diff: docs/quick-start.md ---
@@ -421,16 +421,15 @@ $ YOUR_SPARK_HOME/bin/spark-submit \
Lines with a: 46, Lines with b: 23
{% endhighlight
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18071
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18722
**[Test build #3849 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3849/testReport)**
for PR 18722 at commit
Github user zhengruifeng commented on the issue:
https://github.com/apache/spark/pull/18610
LGTM, this is really a great feature
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
GitHub user holdenk opened a pull request:
https://github.com/apache/spark/pull/18734
[WIP][SPARK-21070][PYSPARK] Attempt to update cloudpickle again
## What changes were proposed in this pull request?
Based on https://github.com/apache/spark/pull/18282 by @rgbkrk this PR
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18528
**[Test build #79946 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79946/testReport)**
for PR 18528 at commit
Github user skonto commented on the issue:
https://github.com/apache/spark/pull/18528
@ArtRand I tested the PR on 1.4 have a look pls.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18664
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18664
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79945/
Test FAILed.
---
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/18664
I'm not sure why this is failing, ping @holdenk @wesm . This is after
introducing a timezone in the Arrow data. The error is
```
Traceback (most recent call last):
File
GitHub user jinxing64 opened a pull request:
https://github.com/apache/spark/pull/18735
[SPARK-21530] Update description of spark.shuffle.maxChunksBeingTransferred.
## What changes were proposed in this pull request?
Update the description of
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/18706
Merging to master / 2.2.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/18725
@gatorsmile
thanks you for review it.
What you mean we can split it to two projects?.
Similar to
```
+- Project [FLOOR((rand(8828525941469309371) * 1.0)) AS k#403L]
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18706
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user wesm commented on the issue:
https://github.com/apache/spark/pull/18664
cc @jreback, he may know more quickly. Possible it's a bug in 0.19.x, but
it should be fixable
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17373
**[Test build #79950 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79950/testReport)**
for PR 17373 at commit
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18725#discussion_r129465387
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/randomExpressions.scala
---
@@ -49,6 +49,8 @@ abstract class RDG extends
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/18725
Seems this is a corner case that column pruning doesn't work well, we
should investigate more about why this happens, e.g. check the query plan and
how the `ColumnPruning` rule change the plan.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18711
**[Test build #79952 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79952/testReport)**
for PR 18711 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18528
**[Test build #79946 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79946/testReport)**
for PR 18528 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18734
**[Test build #79947 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79947/testReport)**
for PR 18734 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18528
**[Test build #79948 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79948/testReport)**
for PR 18528 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18664
**[Test build #79945 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79945/testReport)**
for PR 18664 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18528
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/17848
Thanks! Merging to master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/17848
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18528
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79946/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18528
**[Test build #79948 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79948/testReport)**
for PR 18528 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18528
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18528
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79948/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18734
**[Test build #79947 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79947/testReport)**
for PR 18734 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18734
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79947/
Test PASSed.
---
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/18734
cc @rgbkrk since this is based on your original PR. + @HyukjinKwon
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18734
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user rgbkrk commented on the issue:
https://github.com/apache/spark/pull/18734
You are my hero, thank you.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18735
**[Test build #79949 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79949/testReport)**
for PR 18735 at commit
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/18723
Thanks! Merging to master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18723
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/18711#discussion_r129462543
--- Diff: docs/configuration.md ---
@@ -1103,10 +1103,10 @@ Apart from these, the following properties are also
available, and may be useful
The
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/18640
Retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18640
**[Test build #79951 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79951/testReport)**
for PR 18640 at commit
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/18503#discussion_r129465467
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala
---
@@ -363,7 +363,11 @@ private[state]
201 - 290 of 290 matches
Mail list logo
