Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19459#discussion_r144994355
--- Diff: python/pyspark/sql/session.py ---
@@ -414,6 +415,43 @@ def _createFromLocal(self, data, schema):
data = [schema.toInternal(row) for row
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19500
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19488
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.ap
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19500
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19500
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.ap
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19497
Currently, I meant `saveAsNewAPIHadoopFile` comparing to `saveAsHadoopFile`.
```
saveAsNewAPIHadoopFile[...]("") // succeeds
```
```
saveAsHadoopFile[...]("") // fail
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r144993515
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,47 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19508
cc @cloud-fan too
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h..
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/19495
LGTM. Just a bunch of cosmetic nits, but fine to address them separately
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.a
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/19495#discussion_r144991277
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/FlatMapGroupsWithStateSuite.scala
---
@@ -270,6 +270,60 @@ class FlatMapGroupsWithStateSuite
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/19495#discussion_r144992379
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/GroupState.scala ---
@@ -205,92 +205,127 @@ trait GroupState[S] extends LogicalGroupState[S]
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/19495#discussion_r144992332
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/GroupState.scala ---
@@ -205,92 +205,127 @@ trait GroupState[S] extends LogicalGroupState[S]
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/19495#discussion_r144992239
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/GroupStateImpl.scala
---
@@ -119,32 +115,39 @@ private[sql] class GroupStateImpl[
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/19495#discussion_r144992271
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/GroupStateImpl.scala
---
@@ -187,7 +190,7 @@ private[sql] class GroupStateImpl[S]
Github user tdas commented on the issue:
https://github.com/apache/spark/pull/19452
Waiting for build 82819 to pass.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19495
**[Test build #82823 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82823/testReport)**
for PR 19495 at commit
[`2f35889`](https://github.com/apache/spark/commit/2f
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19452
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19452
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82818/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19452
**[Test build #82818 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82818/testReport)**
for PR 19452 at commit
[`8eb1228`](https://github.com/apache/spark/commit/8
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19452
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82817/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19452
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19452
**[Test build #82817 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82817/testReport)**
for PR 19452 at commit
[`d4984c7`](https://github.com/apache/spark/commit/d
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/19495#discussion_r144989072
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/FlatMapGroupsWithStateSuite.scala
---
@@ -270,6 +270,60 @@ class FlatMapGroupsWithStateSuit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19488
**[Test build #82821 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82821/testReport)**
for PR 19488 at commit
[`6159dcc`](https://github.com/apache/spark/commit/61
Github user maryannxue commented on the issue:
https://github.com/apache/spark/pull/19488
Thank you for the comment, @gatorsmile! Code comment updated.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.o
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19439
**[Test build #82822 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82822/testReport)**
for PR 19439 at commit
[`697585f`](https://github.com/apache/spark/commit/69
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19488
@maryannxue Could you update the comments?
LGTM except the code comments.
---
-
To unsubscribe, e-mail: reviews-uns
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19488#discussion_r144983538
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala
---
@@ -205,14 +205,17 @@ object PhysicalAggregation {
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19505
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19505
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82813/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19505
**[Test build #82813 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82813/testReport)**
for PR 19505 at commit
[`7332969`](https://github.com/apache/spark/commit/7
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19451
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19451
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82815/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19451
**[Test build #82815 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82815/testReport)**
for PR 19451 at commit
[`1690d7d`](https://github.com/apache/spark/commit/1
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19495
**[Test build #82820 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82820/testReport)**
for PR 19495 at commit
[`52deee7`](https://github.com/apache/spark/commit/52
Github user tdas commented on the issue:
https://github.com/apache/spark/pull/19452
LGTM. Will merge after tests pass.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mai
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19269
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82814/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19269
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19269
**[Test build #82814 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82814/testReport)**
for PR 19269 at commit
[`f555949`](https://github.com/apache/spark/commit/f
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/19495#discussion_r144972393
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/FlatMapGroupsWithStateSuite.scala
---
@@ -270,6 +270,60 @@ class FlatMapGroupsWithStateSuite
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/19495#discussion_r144972248
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/GroupState.scala ---
@@ -205,92 +205,127 @@ trait GroupState[S] extends LogicalGroupState[S] {
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/19497
`saveAsNewAPIHadoopFile ` simply delegates to `saveAsNewAPIHadoopDataset`
(with some options set), right ? The behavior would be similar ?
Do you mean `saveAsHadoopDataset` instead ?
I d
Github user szhem commented on the issue:
https://github.com/apache/spark/pull/19410
I would happy if anyone can take a look at this PR.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For addition
Github user szhem commented on the issue:
https://github.com/apache/spark/pull/19373
I would happy if anyone can take a look at this PR.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For addition
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19452
**[Test build #82819 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82819/testReport)**
for PR 19452 at commit
[`23db42c`](https://github.com/apache/spark/commit/23
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/19495#discussion_r144963002
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/GroupStateImpl.scala
---
@@ -119,32 +115,39 @@ private[sql] class GroupStateImpl[S]
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19452
**[Test build #82818 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82818/testReport)**
for PR 19452 at commit
[`8eb1228`](https://github.com/apache/spark/commit/8e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19452
**[Test build #82817 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82817/testReport)**
for PR 19452 at commit
[`d4984c7`](https://github.com/apache/spark/commit/d4
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19505
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19505
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82811/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19505
**[Test build #82811 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82811/testReport)**
for PR 19505 at commit
[`fdafb35`](https://github.com/apache/spark/commit/f
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19505
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82810/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19505
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19505
**[Test build #82810 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82810/testReport)**
for PR 19505 at commit
[`122a7bc`](https://github.com/apache/spark/commit/1
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/19269#discussion_r144948292
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/sources/v2/SimpleWritableDataSource.scala
---
@@ -0,0 +1,254 @@
+/*
+ * Licensed to th
Github user skonto commented on a diff in the pull request:
https://github.com/apache/spark/pull/19374#discussion_r144947372
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala
---
@@ -896,8 +913,8 @@ private[spark]
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19390
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19390
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82816/
Test PASSed.
---
Github user skonto commented on a diff in the pull request:
https://github.com/apache/spark/pull/19374#discussion_r144947305
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala
---
@@ -374,6 +375,15 @@ private[spark
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19390
**[Test build #82816 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82816/testReport)**
for PR 19390 at commit
[`658d8c1`](https://github.com/apache/spark/commit/6
Github user skonto commented on a diff in the pull request:
https://github.com/apache/spark/pull/19374#discussion_r144947255
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -373,10 +374,16 @@ class SparkContext(config: SparkConf) extends Logging
{
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19459
Thanks for reviewing @viirya !
I just had some followup questions at
https://github.com/apache/spark/pull/19459#discussion_r144930424 and
https://github.com/apache/spark/pull/19459#disc
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/19459#discussion_r144945183
--- Diff: python/pyspark/sql/session.py ---
@@ -414,6 +415,43 @@ def _createFromLocal(self, data, schema):
data = [schema.toInternal(row) fo
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19390
**[Test build #82816 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82816/testReport)**
for PR 19390 at commit
[`658d8c1`](https://github.com/apache/spark/commit/65
Github user skonto commented on the issue:
https://github.com/apache/spark/pull/19390
@ArtRand I updated the PR check my comments pls.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19451
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19451
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82812/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19451
**[Test build #82812 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82812/testReport)**
for PR 19451 at commit
[`95a7d46`](https://github.com/apache/spark/commit/9
Github user bdrillard commented on the issue:
https://github.com/apache/spark/pull/19480
@mgaido91 It is possible to increase the heap allocated during testing if
that seems like a desirable thing to do (I did so for #18075, but the current
default is already 4GB), see [1] and [2] dep
Github user krishna-pandey commented on the issue:
https://github.com/apache/spark/pull/19419
@rxin , @srowen I think we can enable X-XSS-Protection and
X-Content-Type-Options response header by default. STS Header can be left
configurable or enabled by default when Spark UI is runnin
Github user sathiyapk commented on the issue:
https://github.com/apache/spark/pull/19451
@gatorsmile you are right! actually i forgot to push the updates in the
test suite. I pushed it now and the test cases are passing..
---
-
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19451
**[Test build #82815 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82815/testReport)**
for PR 19451 at commit
[`1690d7d`](https://github.com/apache/spark/commit/16
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/19452#discussion_r144937226
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingSymmetricHashJoinExec.scala
---
@@ -221,43 +228,29 @@ case class Streaming
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/19495#discussion_r144936080
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/GroupStateImpl.scala
---
@@ -119,32 +115,39 @@ private[sql] class GroupStateImpl[
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/19495#discussion_r144936653
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/FlatMapGroupsWithStateSuite.scala
---
@@ -270,6 +270,60 @@ class FlatMapGroupsWithStateSuit
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19505#discussion_r144936208
--- Diff: python/pyspark/sql/functions.py ---
@@ -2192,67 +2205,82 @@ def pandas_udf(f=None, returnType=StringType()):
:param f: user-defined fun
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/19482
This would be fixed by SPARK-20656 which cannot be implemented until
SPARK-18085 is in. Instead of adding patch on top of patch to fix things on the
SHS, I'd rather spend resources into getting the c
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/19419
Yea in general for security features it seems like it's good to turn on
them by default.
---
-
To unsubscribe, e-mail: reviews-uns
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/19495#discussion_r144935264
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/GroupState.scala ---
@@ -205,92 +205,127 @@ trait GroupState[S] extends LogicalGroupState[S]
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19419
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82807/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19419
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19419
**[Test build #82807 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82807/testReport)**
for PR 19419 at commit
[`1cc34e9`](https://github.com/apache/spark/commit/1
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18747#discussion_r144934974
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala
---
@@ -23,21 +23,56 @@ import org.apache.spark.sql
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19451#discussion_r144934607
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceOperatorSuite.scala
---
@@ -50,6 +52,42 @@ class ReplaceOperatorSui
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19269
**[Test build #82814 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82814/testReport)**
for PR 19269 at commit
[`f555949`](https://github.com/apache/spark/commit/f5
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18747#discussion_r144934031
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala
---
@@ -19,10 +19,10 @@ package org.apache.spark.sql.execution
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18747#discussion_r144933968
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala
---
@@ -84,6 +82,7 @@ private[sql] trait ColumnarBatchScan exten
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/19505#discussion_r144933717
--- Diff: python/pyspark/sql/functions.py ---
@@ -2192,67 +2208,82 @@ def pandas_udf(f=None, returnType=StringType()):
:param f: user-defined fun
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/19500
Hi, @gatorsmile .
Could you review this, too?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For ad
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/19502
Thank you, @gatorsmile , @HyukjinKwon , and @viirya !
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Fo
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19505
**[Test build #82813 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82813/testReport)**
for PR 19505 at commit
[`7332969`](https://github.com/apache/spark/commit/73
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/19459#discussion_r144931496
--- Diff: python/pyspark/sql/types.py ---
@@ -1624,6 +1624,50 @@ def to_arrow_type(dt):
return arrow_type
+def to_arrow_schema(s
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/19459#discussion_r144931384
--- Diff: python/pyspark/sql/session.py ---
@@ -414,6 +415,43 @@ def _createFromLocal(self, data, schema):
data = [schema.toInternal(row) fo
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/19459#discussion_r144931295
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/api/python/PythonSQLUtils.scala ---
@@ -29,4 +32,19 @@ private[sql] object PythonSQLUtils {
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19500
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/19459#discussion_r144931022
--- Diff: python/pyspark/sql/dataframe.py ---
@@ -70,12 +70,12 @@ class DataFrame(object):
.. versionadded:: 1.3
"""
-def
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19500
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82809/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19500
**[Test build #82809 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82809/testReport)**
for PR 19500 at commit
[`8abac33`](https://github.com/apache/spark/commit/8
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/19459#discussion_r144930424
--- Diff: python/pyspark/sql/session.py ---
@@ -414,6 +415,43 @@ def _createFromLocal(self, data, schema):
data = [schema.toInternal(row) fo
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/18732
@cloud-fan Sounds good. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-
101 - 200 of 470 matches
Mail list logo
