[GitHub] spark issue #20219: [SPARK-23025][SQL] Support Null type in scala reflection

2018-01-12 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20219 Thanks! Merged to master/2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage...

2018-01-12 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20204#discussion_r161187608 --- Diff: python/run-tests-with-coverage --- @@ -0,0 +1,69 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation

[GitHub] spark issue #20184: [SPARK-22987][Core] UnsafeExternalSorter cases OOM when ...

2018-01-12 Thread liutang123
Github user liutang123 commented on the issue: https://github.com/apache/spark/pull/20184 Hi, @jerryshao , we can produce this issue as follows: ``` $ bin/spark-shell --master local --conf spark.sql.windowExec.buffer.spill.threshold=1 --driver-memory 1G scala>sc.range(1,

[GitHub] spark issue #20241: [SPARK-23008][ML][FOLLOW-UP] mark OneHotEncoder python A...

2018-01-12 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/20241 LGTM thanks. Merged to master / branch-2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20241: [SPARK-23008][ML][FOLLOW-UP] mark OneHotEncoder python A...

2018-01-12 Thread 6high
Github user 6high commented on the issue: https://github.com/apache/spark/pull/20241 good --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20241: [SPARK-23008][ML][FOLLOW-UP] mark OneHotEncoder p...

2018-01-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20241 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20219: [SPARK-23025][SQL] Support Null type in scala ref...

2018-01-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20219 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20246: [SPARK-23054][SQL] Fix incorrect results of casting User...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20246 **[Test build #86033 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86033/testReport)** for PR 20246 at commit

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2018-01-12 Thread yolile
Github user yolile commented on the issue: https://github.com/apache/spark/pull/13617 @JeremyNixon @avulanov @MLnick @mengxr @jkbradley any update? Is this PR going to be merged? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20245: [SPARK-21213][SQL][FOLLOWUP] Use compatible types for co...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20245 **[Test build #86029 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86029/testReport)** for PR 20245 at commit

[GitHub] spark issue #20222: [SPARK-23028] Bump master branch version to 2.4.0-SNAPSH...

2018-01-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20222 BTW, does anyone know why it suddently takes logner time? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20222: [SPARK-23028] Bump master branch version to 2.4.0-SNAPSH...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20222 **[Test build #86026 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86026/testReport)** for PR 20222 at commit

[GitHub] spark issue #20246: [SPARK-23054][SQL] Fix incorrect results of casting User...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20246 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86033/ Test FAILed. ---

[GitHub] spark issue #20246: [SPARK-23054][SQL] Fix incorrect results of casting User...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20246 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20246: [SPARK-23054][SQL] Fix incorrect results of casting User...

2018-01-12 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20246 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20246: [SPARK-23054][SQL] Fix incorrect results of casting User...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20246 **[Test build #86034 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86034/testReport)** for PR 20246 at commit

[GitHub] spark issue #20189: [SPARK-22975][SS] MetricsReporter should not throw excep...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20189 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86030/ Test PASSed. ---

[GitHub] spark issue #20189: [SPARK-22975][SS] MetricsReporter should not throw excep...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20189 **[Test build #86030 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86030/testReport)** for PR 20189 at commit

[GitHub] spark issue #20189: [SPARK-22975][SS] MetricsReporter should not throw excep...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20189 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19224: [SPARK-20990][SQL] Read all JSON documents in files when...

2018-01-12 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19224 kindly ping @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-12 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20023 any more comments @gatorsmile @cloud-fan @dongjoon-hyun @viirya @hvanhovell ? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20214: [SPARK-23023][SQL] Cast field data to strings in showStr...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20214 **[Test build #86028 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86028/testReport)** for PR 20214 at commit

[GitHub] spark issue #20214: [SPARK-23023][SQL] Cast field data to strings in showStr...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20214 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86028/ Test FAILed. ---

[GitHub] spark issue #20214: [SPARK-23023][SQL] Cast field data to strings in showStr...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20214 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20222: [SPARK-23028] Bump master branch version to 2.4.0...

2018-01-12 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20222#discussion_r161211752 --- Diff: dev/run-tests-jenkins.py --- @@ -181,8 +181,8 @@ def main(): short_commit_hash = ghprb_actual_commit[0:7] # format:

[GitHub] spark issue #20222: [SPARK-23028] Bump master branch version to 2.4.0-SNAPSH...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20222 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20222: [SPARK-23028] Bump master branch version to 2.4.0-SNAPSH...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20222 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86026/ Test PASSed. ---

[GitHub] spark issue #20245: [SPARK-21213][SQL][FOLLOWUP] Use compatible types for co...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20245 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20245: [SPARK-21213][SQL][FOLLOWUP] Use compatible types for co...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20245 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86029/ Test PASSed. ---

[GitHub] spark issue #20249: [SPARK-23057][SPARK-19235][SQL] SET LOCATION should chan...

2018-01-12 Thread xubo245
Github user xubo245 commented on the issue: https://github.com/apache/spark/pull/20249 @gatorsmile Please review it --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20249: [SPARK-23057][SPARK-19235][SQL] SET LOCATION should chan...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20249 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20228: [SPARK-23036][SQL] Add withGlobalTempView for testing an...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20228 **[Test build #86041 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86041/testReport)** for PR 20228 at commit

[GitHub] spark pull request #20214: [SPARK-23023][SQL] Cast field data to strings in ...

2018-01-12 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20214#discussion_r161255253 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -1255,6 +1255,34 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark issue #20222: [SPARK-23028] Bump master branch version to 2.4.0-SNAPSH...

2018-01-12 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20222 is it good to go? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20240: [SPARK-23049][SQL] `spark.sql.files.ignoreCorruptFiles` ...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20240 **[Test build #86038 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86038/testReport)** for PR 20240 at commit

[GitHub] spark issue #20251: [Spark-23051][core] Fix for broken job description in Sp...

2018-01-12 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20251 @smurakozi thanks for the fix! can you put a screenshot of the UI to show it's fixed? thanks! --- - To unsubscribe, e-mail:

[GitHub] spark issue #20245: [SPARK-21213][SQL][FOLLOWUP] Use compatible types for co...

2018-01-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20245 Do these values actually take on values larger than Long.MaxValue? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20212: Update rdd-programming-guide.md

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20212 **[Test build #4036 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4036/testReport)** for PR 20212 at commit

[GitHub] spark pull request #20247: [SPARK-23021][SQL] AnalysisBarrier should overrid...

2018-01-12 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/20247 [SPARK-23021][SQL] AnalysisBarrier should override innerChildren to print correct explain output ## What changes were proposed in this pull request? `AnalysisBarrier` in the current master cuts

[GitHub] spark pull request #20249: [SPARK-23057][SPARK-19235][SQL] SET LOCATION shou...

2018-01-12 Thread xubo245
GitHub user xubo245 opened a pull request: https://github.com/apache/spark/pull/20249 [SPARK-23057][SPARK-19235][SQL] SET LOCATION should change the path of partition in table ## What changes were proposed in this pull request? Fix error of SE T LOCATION SET

[GitHub] spark pull request #20246: [SPARK-23054][SQL] Fix incorrect results of casti...

2018-01-12 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20246#discussion_r161255621 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala --- @@ -282,6 +282,8 @@ case class Cast(child: Expression,

[GitHub] spark issue #20251: [Spark-23051][core] Fix for broken job description in Sp...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20251 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20179: [SPARK-22982] Remove unsafe asynchronous close() call fr...

2018-01-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20179 I cherry-picked to 2.2 as well, at least. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20251: [Spark-23051][core] Fix for broken job descriptio...

2018-01-12 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20251#discussion_r161273733 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -429,20 +429,40 @@ private[ui] class JobDataSource( val

[GitHub] spark issue #20240: [SPARK-23049][SQL] `spark.sql.files.ignoreCorruptFiles` ...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20240 **[Test build #86047 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86047/testReport)** for PR 20240 at commit

[GitHub] spark pull request #5475: [SPARK-6844][SQL] Clean up accumulators used in In...

2018-01-12 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/5475#discussion_r161253866 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/columnar/PartitionBatchPruningSuite.scala --- @@ -39,6 +39,8 @@ class PartitionBatchPruningSuite

[GitHub] spark pull request #20153: [SPARK-22392][SQL] data source v2 columnar batch ...

2018-01-12 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20153#discussion_r161257419 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2ScanExec.scala --- @@ -37,40 +35,58 @@ import

[GitHub] spark issue #20222: [SPARK-23028] Bump master branch version to 2.4.0-SNAPSH...

2018-01-12 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20222 BTW don't forget https://github.com/apache/spark/pull/20222#discussion_r160740904 :) --- - To unsubscribe, e-mail:

[GitHub] spark issue #20246: [SPARK-23054][SQL] Fix incorrect results of casting User...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20246 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20072: [SPARK-22790][SQL] add a configurable factor to describe...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20072 **[Test build #86045 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86045/testReport)** for PR 20072 at commit

[GitHub] spark issue #20246: [SPARK-23054][SQL] Fix incorrect results of casting User...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20246 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86036/ Test FAILed. ---

[GitHub] spark issue #20225: [SPARK-23033] Don't use task level retry for continuous ...

2018-01-12 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20225 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20247: [SPARK-23021][SQL] AnalysisBarrier should override inner...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20247 **[Test build #86039 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86039/testReport)** for PR 20247 at commit

[GitHub] spark issue #20248: [SPARK-23058][SQL] Fix non printable field delim issue

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20248 **[Test build #86040 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86040/testReport)** for PR 20248 at commit

[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #86044 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86044/testReport)** for PR 20226 at commit

[GitHub] spark issue #20239: [SPARK-23047][PYTHON][SQL] Change MapVector to NullableM...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20239 **[Test build #86043 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86043/testReport)** for PR 20239 at commit

[GitHub] spark issue #20246: [SPARK-23054][SQL] Fix incorrect results of casting User...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20246 **[Test build #86036 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86036/testReport)** for PR 20246 at commit

[GitHub] spark issue #20239: [SPARK-23047][PYTHON][SQL] Change MapVector to NullableM...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20239 **[Test build #86048 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86048/testReport)** for PR 20239 at commit

[GitHub] spark issue #20212: Update rdd-programming-guide.md

2018-01-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20212 OK consider that and http://spark.apache.org/contributing.html for the future. I'll just merge this. --- - To unsubscribe,

[GitHub] spark issue #20248: [SPARK-23058][SQL] Fix non printable field delim issue

2018-01-12 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20248 [Non printable characters](http://www.theasciicode.com.ar/): https://user-images.githubusercontent.com/5399861/34880068-33152b7a-f7ea-11e7-8203-570e61c7a21c.png;> ---

[GitHub] spark pull request #20153: [SPARK-22392][SQL] data source v2 columnar batch ...

2018-01-12 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20153#discussion_r161252949 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -90,14 +92,56 @@ case class

[GitHub] spark pull request #5475: [SPARK-6844][SQL] Clean up accumulators used in In...

2018-01-12 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/5475#discussion_r161254105 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala --- @@ -297,4 +298,21 @@ class CachedTableSuite extends QueryTest {

[GitHub] spark issue #20250: [SPARK-23059][SQL][TEST] Correct some improper with view...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20250 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20211: [SPARK-23011][PYTHON][SQL] Prepend missing groupi...

2018-01-12 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20211#discussion_r161256435 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala --- @@ -457,13 +458,26 @@ class RelationalGroupedDataset

[GitHub] spark issue #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregation func...

2018-01-12 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/19872 @ueshin I think this PR is close to be merged. About you comment https://github.com/apache/spark/pull/19872#pullrequestreview-87762872 on udaf that supports partial aggregation, is there

[GitHub] spark issue #18991: [SPARK-21783][SQL] Turn on ORC filter push-down by defau...

2018-01-12 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18991 I think we still have time for 2.3? I'm not worried about correctness, but we should show people how much it improves. --- -

[GitHub] spark pull request #20163: [SPARK-22966][PYTHON][SQL] Python UDFs with retur...

2018-01-12 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20163#discussion_r161268321 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/EvaluatePython.scala --- @@ -144,6 +145,7 @@ object EvaluatePython { }

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19893 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19893 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86035/ Test FAILed. ---

[GitHub] spark issue #20212: Update rdd-programming-guide.md

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20212 **[Test build #4036 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4036/testReport)** for PR 20212 at commit

[GitHub] spark pull request #20246: [SPARK-23054][SQL] Fix incorrect results of casti...

2018-01-12 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20246#discussion_r161255803 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala --- @@ -282,6 +282,8 @@ case class Cast(child: Expression,

[GitHub] spark issue #20246: [SPARK-23054][SQL] Fix incorrect results of casting User...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20246 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20251: [Spark-23051][core] Fix for broken job descriptio...

2018-01-12 Thread smurakozi
GitHub user smurakozi opened a pull request: https://github.com/apache/spark/pull/20251 [Spark-23051][core] Fix for broken job description in Spark UI ## What changes were proposed in this pull request? In 2.2, Spark UI displayed the stage description if the job description

[GitHub] spark issue #20153: [SPARK-22392][SQL] data source v2 columnar batch reader

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20153 **[Test build #86042 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86042/testReport)** for PR 20153 at commit

[GitHub] spark issue #20153: [SPARK-22392][SQL] data source v2 columnar batch reader

2018-01-12 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20153 also cc @rxin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20251: [Spark-23051][core] Fix for broken job descriptio...

2018-01-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20251#discussion_r161267849 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -429,20 +429,40 @@ private[ui] class JobDataSource( val

[GitHub] spark pull request #20251: [Spark-23051][core] Fix for broken job descriptio...

2018-01-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20251#discussion_r161268598 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -429,20 +429,40 @@ private[ui] class JobDataSource( val

[GitHub] spark issue #20222: [SPARK-23028] Bump master branch version to 2.4.0-SNAPSH...

2018-01-12 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20222 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20251: [Spark-23051][core] Fix for broken job descriptio...

2018-01-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20251#discussion_r161268434 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -429,20 +429,40 @@ private[ui] class JobDataSource( val

[GitHub] spark pull request #20251: [Spark-23051][core] Fix for broken job descriptio...

2018-01-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20251#discussion_r161268478 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -429,20 +429,40 @@ private[ui] class JobDataSource( val

[GitHub] spark issue #20251: [Spark-23051][core] Fix for broken job description in Sp...

2018-01-12 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/20251 Screenshots: [In 2.2](https://issues.apache.org/jira/secure/attachment/12905889/in-2.2.png) [In 2.3 before

[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20240: [SPARK-23049][SQL] `spark.sql.files.ignoreCorruptFiles` ...

2018-01-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20240 Retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...

2018-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86044/ Test FAILed. ---

[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #86044 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86044/testReport)** for PR 20226 at commit

[GitHub] spark issue #20239: [SPARK-23047][PYTHON][SQL] Change MapVector to NullableM...

2018-01-12 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/20239 `MapVector` is still used in Arrow internal code but it should not be returned to user directly.

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19893 **[Test build #86035 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86035/testReport)** for PR 19893 at commit

[GitHub] spark pull request #20212: Update rdd-programming-guide.md

2018-01-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20212 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20248: [SPARK-23058][SQL] Fix non printable field delim ...

2018-01-12 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20248 [SPARK-23058][SQL] Fix non printable field delim issue ## What changes were proposed in this pull request? Create a table with non printable delim like below: ```sql CREATE

[GitHub] spark pull request #20248: [SPARK-23058][SQL] Fix non printable field delim ...

2018-01-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20248#discussion_r161240770 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -1023,7 +1023,12 @@ case class ShowCreateTableCommand(table:

[GitHub] spark pull request #20214: [SPARK-23023][SQL] Cast field data to strings in ...

2018-01-12 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20214#discussion_r161254906 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -237,13 +237,17 @@ class Dataset[T] private[sql]( private[sql] def

[GitHub] spark issue #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpark

2018-01-12 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20217 Both are fine to me, and seems reusing `registerFunction` is more Python style(correct me if I was wrong). My suggestion is to put the UDF registration interface in `UDFRegistration`,

[GitHub] spark pull request #20153: [SPARK-22392][SQL] data source v2 columnar batch ...

2018-01-12 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20153#discussion_r161265463 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala --- @@ -17,21 +17,24 @@ package

[GitHub] spark issue #20228: [SPARK-23036][SQL][TEST] Add withGlobalTempView for test...

2018-01-12 Thread xubo245
Github user xubo245 commented on the issue: https://github.com/apache/spark/pull/20228 @gatorsmile Sure. This is only for TEST. Done , I put '[SQL]' into title too --- - To unsubscribe, e-mail:

[GitHub] spark issue #20072: [SPARK-22790][SQL] add a configurable factor to describe...

2018-01-12 Thread CodingCat
Github user CodingCat commented on the issue: https://github.com/apache/spark/pull/20072 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20251: [Spark-23051][core] Fix for broken job descriptio...

2018-01-12 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20251#discussion_r161273851 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -429,20 +429,40 @@ private[ui] class JobDataSource( val

[GitHub] spark issue #20245: [SPARK-21213][SQL][FOLLOWUP] Use compatible types for co...

2018-01-12 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20245 yea, it's a super corner-case (almost impossible). So, this is just a correctness issue as intellij suggests incompatible comparisons. (Actually, I'm not sure why we use `BigInt` for the statistics

[GitHub] spark issue #20225: [SPARK-23033] Don't use task level retry for continuous ...

2018-01-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20225 **[Test build #86046 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86046/testReport)** for PR 20225 at commit

[GitHub] spark pull request #20250: [SPARK-23059][SQL][TEST] Correct some improper wi...

2018-01-12 Thread xubo245
GitHub user xubo245 opened a pull request: https://github.com/apache/spark/pull/20250 [SPARK-23059][SQL][TEST] Correct some improper with view related method usage ## What changes were proposed in this pull request? Correct some improper with view related method

[GitHub] spark pull request #20228: [SPARK-23036][SQL][TEST] Add withGlobalTempView f...

2018-01-12 Thread xubo245
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/spark/pull/20228#discussion_r161253370 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/GlobalTempViewSuite.scala --- @@ -140,8 +140,8 @@ class GlobalTempViewSuite extends

  1   2   3   4   5   >