[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208439973 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2ScanExec.scala --- @@ -39,52 +36,43 @@ case class

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21889 Assuming from https://github.com/apache/spark/pull/21889#issuecomment-408330791, we shouldn't have any identified bug here. What kind of bugs left to be fixed? ---

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208440273 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2ScanExec.scala --- @@ -93,21 +81,17 @@ case class

[GitHub] spark pull request #22013: [SPARK-23939][SQL] Add transform_keys function

2018-08-07 Thread codeatri
Github user codeatri commented on a diff in the pull request: https://github.com/apache/spark/pull/22013#discussion_r20844 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HigherOrderFunctionsSuite.scala --- @@ -181,4 +187,46 @@ class

[GitHub] spark pull request #21889: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-08-07 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/21889#discussion_r208446828 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala --- @@ -0,0 +1,205 @@ +/* + *

[GitHub] spark issue #22013: [SPARK-23939][SQL] Add transform_keys function

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22013 **[Test build #94405 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94405/testReport)** for PR 22013 at commit

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21889 **[Test build #94406 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94406/testReport)** for PR 21889 at commit

[GitHub] spark issue #22027: [SPARK-25010][SQL][FOLLOWUP] Shuffle should also produce...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22027 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22027: [SPARK-25010][SQL][FOLLOWUP] Shuffle should also produce...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22027 **[Test build #94395 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94395/testReport)** for PR 22027 at commit

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-08-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21596 @jerryshao This is for 3.0 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21845 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21845 **[Test build #94396 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94396/testReport)** for PR 21845 at commit

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21845 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208423440 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/streaming/StreamingReadSupport.java --- @@ -0,0 +1,49 @@ +/* + * Licensed

[GitHub] spark issue #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22017 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94394/ Test PASSed. ---

[GitHub] spark issue #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22017 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22017 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94393/ Test PASSed. ---

[GitHub] spark issue #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22017 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-08-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21305 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21845 Let me push this in late tonight or early tomorrow. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22032: [SPARK-25047][ML] Can't assign SerializedLambda to scala...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22032 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22032: [SPARK-25047][ML] Can't assign SerializedLambda to scala...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22032 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208437319 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/BatchReadSupportProvider.java --- @@ -19,18 +19,18 @@ import

[GitHub] spark issue #22032: [SPARK-25047][ML] Can't assign SerializedLambda to scala...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22032 **[Test build #94400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94400/testReport)** for PR 22032 at commit

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208439150 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/streaming/StreamingReadSupport.java --- @@ -0,0 +1,49 @@ +/* + *

[GitHub] spark issue #21520: [SPARK-24505][SQL] Forbidding string interpolation in Co...

2018-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21520 @viirya ~ I was just trying to read the PRs. Would you please mind if I ask where is the "Based on previous discussion" ? ---

[GitHub] spark issue #22027: [SPARK-25010][SQL][FOLLOWUP] Shuffle should also produce...

2018-08-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22027 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22027: [SPARK-25010][SQL][FOLLOWUP] Shuffle should also ...

2018-08-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22027 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21520: [SPARK-24505][SQL] Forbidding string interpolation in Co...

2018-08-07 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21520 @HyukjinKwon Thanks for looking into this. It is based on the comment and discussion here https://github.com/apache/spark/pull/21193#discussion_r186627099. ---

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-07 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21977 Yes, this is for YARN only. I've also opened follow-up issues for Mesos and Kubernetes integration. --- - To unsubscribe,

[GitHub] spark pull request #21608: [SPARK-24626] [SQL] Improve location size calcula...

2018-08-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21608#discussion_r208408858 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceUtils.scala --- @@ -49,4 +51,11 @@ object DataSourceUtils {

[GitHub] spark issue #22031: [TODO][SPARK-23932][SQL] Higher order function zip_with

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22031 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22031: [TODO][SPARK-23932][SQL] Higher order function zip_with

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22031 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94389/ Test PASSed. ---

[GitHub] spark pull request #22030: [SPARK-25048][SQL] Pivoting by multiple columns i...

2018-08-07 Thread maryannxue
Github user maryannxue commented on a diff in the pull request: https://github.com/apache/spark/pull/22030#discussion_r208423936 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala --- @@ -403,20 +415,29 @@ class RelationalGroupedDataset

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208425199 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/streaming/StreamingReadSupport.java --- @@ -0,0 +1,49 @@ +/* + *

[GitHub] spark pull request #22030: [SPARK-25048][SQL] Pivoting by multiple columns i...

2018-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22030#discussion_r208431249 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala --- @@ -403,20 +415,29 @@ class RelationalGroupedDataset

[GitHub] spark issue #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22017 **[Test build #94394 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94394/testReport)** for PR 22017 at commit

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208437780 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/StreamingWriteSupportProvider.java --- @@ -29,24 +28,24 @@ * provide data writing

[GitHub] spark issue #22014: [SPARK-25036][SQL] avoid match may not be exhaustive in ...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22014 **[Test build #94402 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94402/testReport)** for PR 22014 at commit

[GitHub] spark issue #22022: [SPARK-24948][SHS][BACKPORT-2.2] Delegate check access p...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22022 **[Test build #94401 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94401/testReport)** for PR 22022 at commit

[GitHub] spark issue #21733: [SPARK-24763][SS] Remove redundant key data from value i...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21733 **[Test build #94403 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94403/testReport)** for PR 21733 at commit

[GitHub] spark issue #22022: [SPARK-24948][SHS][BACKPORT-2.2] Delegate check access p...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22022 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208438912 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/streaming/MicroBatchReadSupport.java --- @@ -0,0 +1,49 @@ +/* + *

[GitHub] spark issue #22014: [SPARK-25036][SQL] avoid match may not be exhaustive in ...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22014 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22022: [SPARK-24948][SHS][BACKPORT-2.2] Delegate check access p...

2018-08-07 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/22022 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22014: [SPARK-25036][SQL] avoid match may not be exhaustive in ...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22014 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22022: [SPARK-24948][SHS][BACKPORT-2.2] Delegate check access p...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22022 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21889 > but @gatorsmile wants to review it in a follow-on PR. I need a confirmation from @gatorsmile. I don't want to ignore his decision here in > Just FYI, we are unable to merge

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-07 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/21889 See https://github.com/apache/spark/pull/21320#issuecomment-406353694 for @gatorsmile's request to move the changes to `ParquetReadSupport.scala` to another PR. There was another,

[GitHub] spark issue #22027: [SPARK-25010][SQL][FOLLOWUP] Shuffle should also produce...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22027 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22030: [SPARK-25048][SQL] Pivoting by multiple columns i...

2018-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22030#discussion_r208420184 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala --- @@ -403,20 +415,29 @@ class RelationalGroupedDataset

[GitHub] spark issue #22014: [SPARK-25036][SQL] avoid match may not be exhaustive in ...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22014 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22014: [SPARK-25036][SQL] avoid match may not be exhaustive in ...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22014 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94390/ Test PASSed. ---

[GitHub] spark issue #22014: [SPARK-25036][SQL] avoid match may not be exhaustive in ...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22014 **[Test build #94390 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94390/testReport)** for PR 22014 at commit

[GitHub] spark pull request #21608: [SPARK-24626] [SQL] Improve location size calcula...

2018-08-07 Thread Achuth17
Github user Achuth17 commented on a diff in the pull request: https://github.com/apache/spark/pull/21608#discussion_r208423190 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceUtils.scala --- @@ -49,4 +51,11 @@ object DataSourceUtils {

[GitHub] spark issue #22032: [SPARK-25047][ML] Can't assign SerializedLambda to scala...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22032 **[Test build #94398 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94398/testReport)** for PR 22032 at commit

[GitHub] spark issue #22032: [SPARK-25047][ML] Can't assign SerializedLambda to scala...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22032 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22033: [SPARK-23935][SQL][followup] mapEntry throw org.codehaus...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22033 **[Test build #94399 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94399/testReport)** for PR 22033 at commit

[GitHub] spark issue #22031: [TODO][SPARK-23932][SQL] Higher order function zip_with

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22031 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22031: [TODO][SPARK-23932][SQL] Higher order function zip_with

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22031 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94392/ Test PASSed. ---

[GitHub] spark pull request #21859: [SPARK-24900][SQL]Speed up sort when the dataset ...

2018-08-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21859#discussion_r208436900 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/SmallDataSortBenchmark.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the

[GitHub] spark pull request #21859: [SPARK-24900][SQL]Speed up sort when the dataset ...

2018-08-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21859#discussion_r208437010 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/SmallDataSortBenchmark.scala --- @@ -0,0 +1,85 @@ +/* + * Licensed to the

[GitHub] spark pull request #21859: [SPARK-24900][SQL]Speed up sort when the dataset ...

2018-08-07 Thread sddyljsx
Github user sddyljsx commented on a diff in the pull request: https://github.com/apache/spark/pull/21859#discussion_r208441135 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/SmallDataSortBenchmark.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to

[GitHub] spark pull request #21859: [SPARK-24900][SQL]Speed up sort when the dataset ...

2018-08-07 Thread sddyljsx
Github user sddyljsx commented on a diff in the pull request: https://github.com/apache/spark/pull/21859#discussion_r208441067 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/SmallDataSortBenchmark.scala --- @@ -0,0 +1,85 @@ +/* + * Licensed to

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21977 Does this work by setting some container configs? Maybe we can apply this to k8s later, cc @liyinan926 --- - To unsubscribe,

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21889 > but @gatorsmile wants to review it in a follow-on PR. Where did he say it after the comment above? Also why don't you address my comments if you're going to push more changes

[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21608 **[Test build #94397 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94397/testReport)** for PR 21608 at commit

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21305 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21305 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94384/ Test PASSed. ---

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208371927 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/streaming/ContinuousReadSupport.java --- @@ -0,0 +1,72 @@ +/* + * Licensed

[GitHub] spark issue #22031: [TODO][SPARK-23932][SQL] Higher order function zip_with

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22031 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208390359 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/streaming/StreamingReadSupport.java --- @@ -0,0 +1,49 @@ +/* + * Licensed

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208390264 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/streaming/StreamingReadSupport.java --- @@ -0,0 +1,49 @@ +/* + * Licensed

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22009 **[Test build #94377 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94377/testReport)** for PR 22009 at commit

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208333129 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/BatchWriteSupportProvider.java --- @@ -21,32 +21,32 @@ import

[GitHub] spark pull request #21977: SPARK-25004: Add spark.executor.pyspark.memory li...

2018-08-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21977#discussion_r208339172 --- Diff: python/pyspark/worker.py --- @@ -259,6 +260,26 @@ def main(infile, outfile): "PYSPARK_DRIVER_PYTHON are correctly

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208345467 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/ScanConfig.java --- @@ -18,22 +18,16 @@ package

[GitHub] spark issue #22027: [SPARK-25010][SQL][FOLLOWUP] Shuffle should also produce...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22027 **[Test build #94381 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94381/testReport)** for PR 22027 at commit

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208348226 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/SupportsReportStatistics.java --- @@ -20,18 +20,18 @@ import

[GitHub] spark issue #22014: [SPARK-25036][SQL] avoid match may not be exhaustive in ...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22014 **[Test build #94388 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94388/testReport)** for PR 22014 at commit

[GitHub] spark issue #22006: [SPARK-25031][SQL] Fix MapType schema print

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22006 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94383/ Test PASSed. ---

[GitHub] spark issue #22006: [SPARK-25031][SQL] Fix MapType schema print

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22006 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21305 **[Test build #94384 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94384/testReport)** for PR 21305 at commit

[GitHub] spark issue #22014: [SPARK-25036][SQL] avoid match may not be exhaustive in ...

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22014 **[Test build #94390 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94390/testReport)** for PR 22014 at commit

[GitHub] spark issue #22014: [SPARK-25036][SQL] avoid match may not be exhaustive in ...

2018-08-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22014 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22014: [SPARK-25036][SQL] avoid match may not be exhaustive in ...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22014 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208372512 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/streaming/MicroBatchReadSupport.java --- @@ -0,0 +1,49 @@ +/* + * Licensed

[GitHub] spark issue #22014: [SPARK-25036][SQL] avoid match may not be exhaustive in ...

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22014 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208372469 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/streaming/StreamingReadSupport.java --- @@ -0,0 +1,49 @@ +/* + *

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208384141 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala --- @@ -80,17 +80,17 @@ object

[GitHub] spark issue #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22017 **[Test build #94393 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94393/testReport)** for PR 22017 at commit

[GitHub] spark pull request #22031: [TODO][SPARK-23932][SQL] Higher order function zi...

2018-08-07 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/22031#discussion_r208403145 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -442,3 +442,91 @@ case class

[GitHub] spark pull request #22031: [TODO][SPARK-23932][SQL] Higher order function zi...

2018-08-07 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/22031#discussion_r208398313 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -442,3 +442,91 @@ case class

[GitHub] spark pull request #22031: [TODO][SPARK-23932][SQL] Higher order function zi...

2018-08-07 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/22031#discussion_r208399620 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -442,3 +442,91 @@ case class

[GitHub] spark issue #22028: [SPARK-25046][SQL] Fix Alter View can excute sql like "A...

2018-08-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22028 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22009 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22009 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94377/ Test FAILed. ---

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208333413 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/ContinuousReadSupportProvider.java --- @@ -20,17 +20,30 @@ import java.util.Optional;

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r208338263 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/StreamingWriteSupportProvider.java --- @@ -29,24 +28,24 @@ * provide data writing

[GitHub] spark issue #22031: [TODO][SPARK-23932][SQL] Higher order function zip_with

2018-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22031 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

<    1   2   3   4   5   6   7   >