[GitHub] spark issue #20928: Fix small typo in configuration doc

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20928 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20928: Fix small typo in configuration doc

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20928 **[Test build #88816 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88816/testReport)** for PR 20928 at commit

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20959 **[Test build #88817 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88817/testReport)** for PR 20959 at commit

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20959 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88817/ Test FAILed. ---

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20959 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20960: [MINOR][CORE] Show block manager id when remove R...

2018-04-02 Thread jiangxb1987
GitHub user jiangxb1987 opened a pull request: https://github.com/apache/spark/pull/20960 [MINOR][CORE] Show block manager id when remove RDD/Broadcast fails. ## What changes were proposed in this pull request? Address

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-04-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20850 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2018-04-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18576 I checked if we could fix the join issue: https://github.com/apache/spark/commit/c4ca4f3dbcc69b7bccb9e1b093dd8488d15eedee --- -

[GitHub] spark pull request #20959: [SPARK-23846][SQL] The samplingRatio option for C...

2018-04-02 Thread MaxGekk
GitHub user MaxGekk opened a pull request: https://github.com/apache/spark/pull/20959 [SPARK-23846][SQL] The samplingRatio option for CSV datasource ## What changes were proposed in this pull request? I propose to support the `samplingRatio` option for schema inferring of

[GitHub] spark pull request #20959: [SPARK-23846][SQL] The samplingRatio option for C...

2018-04-02 Thread sujithjay
Github user sujithjay commented on a diff in the pull request: https://github.com/apache/spark/pull/20959#discussion_r178567646 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2127,4 +2127,39 @@ class JsonSuite extends

[GitHub] spark issue #20931: [SPARK-23815][Core]Spark writer dynamic partition overwr...

2018-04-02 Thread sujithjay
Github user sujithjay commented on the issue: https://github.com/apache/spark/pull/20931 Hi @fangshil , can you try and add test cases to verify this changes introduced in this patch? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20959 **[Test build #88817 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88817/testReport)** for PR 20959 at commit

[GitHub] spark issue #20960: [MINOR][CORE] Show block manager id when remove RDD/Broa...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20960 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20960: [MINOR][CORE] Show block manager id when remove RDD/Broa...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20960 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1904/

[GitHub] spark issue #20960: [MINOR][CORE] Show block manager id when remove RDD/Broa...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20960 **[Test build #88819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88819/testReport)** for PR 20960 at commit

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-04-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r178578044 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala --- @@ -50,7 +50,9 @@ abstract class CSVDataSource

[GitHub] spark issue #20928: Fix small typo in configuration doc

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20928 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88816/ Test PASSed. ---

[GitHub] spark issue #20919: Feature/apply func to rdd

2018-04-02 Thread sujithjay
Github user sujithjay commented on the issue: https://github.com/apache/spark/pull/20919 Hi @gianmarcodonetti , can you edit the description of this PR to include an explanation of the changes made? Also, excluding very minor changes, you should first discuss the changes you are

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-04-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r178577209 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -524,6 +524,8 @@ class DataFrameReader private[sql](sparkSession:

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-04-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r178578327 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala --- @@ -118,6 +120,61 @@ object CSVDataSource {

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-04-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r178578284 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala --- @@ -118,6 +120,61 @@ object CSVDataSource {

[GitHub] spark issue #20928: Fix small typo in configuration doc

2018-04-02 Thread dsakuma
Github user dsakuma commented on the issue: https://github.com/apache/spark/pull/20928 @HyukjinKwon I think I'm gonna fininsh today. And I've added the issue in the description. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20928: Fix small typo in configuration doc

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20928 **[Test build #88816 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88816/testReport)** for PR 20928 at commit

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20959 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20959 **[Test build #88818 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88818/testReport)** for PR 20959 at commit

[GitHub] spark pull request #20957: Branch 2.3

2018-04-02 Thread rameshch16
GitHub user rameshch16 opened a pull request: https://github.com/apache/spark/pull/20957 Branch 2.3 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how this

[GitHub] spark issue #20930: [SPARK-23811][Core] FetchFailed comes before Success of ...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20930 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88806/ Test FAILed. ---

[GitHub] spark issue #20958: [SPARK-23844][SS] Fix socket source honors recovered off...

2018-04-02 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20958 Not sure why the test is not triggered, maybe jenkins is down. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20957: Branch 2.3

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20957 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20956: [SPARK-23841][ML] NodeIdCache should unpersist th...

2018-04-02 Thread sujithjay
Github user sujithjay commented on a diff in the pull request: https://github.com/apache/spark/pull/20956#discussion_r178518257 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/NodeIdCache.scala --- @@ -166,9 +166,13 @@ private[spark] class NodeIdCache( }

[GitHub] spark pull request #20956: [SPARK-23841][ML] NodeIdCache should unpersist th...

2018-04-02 Thread sujithjay
Github user sujithjay commented on a diff in the pull request: https://github.com/apache/spark/pull/20956#discussion_r178517357 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/NodeIdCache.scala --- @@ -95,7 +95,7 @@ private[spark] class NodeIdCache( splits:

[GitHub] spark issue #20957: Branch 2.3

2018-04-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20957 @rameshch16, seems mistakenly open. mind closing this please? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20928: Fix small typo in configuration doc

2018-04-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20928 Hey @dsakuma, have you had a change to look other docs too? if you are busy, let me just go and merge this. BTW, mind adding `Closes #20948` in the PR description too so that that PR can be

[GitHub] spark pull request #20958: []Fix socket source honors recovered offsets issu...

2018-04-02 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/20958 []Fix socket source honors recovered offsets issue ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-04-02 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20717 any more comments @cloud-fan @dongjoon-hyun ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20930: [SPARK-23811][Core] FetchFailed comes before Success of ...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20930 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20930: [SPARK-23811][Core] FetchFailed comes before Success of ...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20930 **[Test build #88806 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88806/testReport)** for PR 20930 at commit

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88801/ Test PASSed. ---

[GitHub] spark pull request #20930: [SPARK-23811][Core] FetchFailed comes before Succ...

2018-04-02 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/20930#discussion_r178499952 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -794,6 +794,19 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request #20313: [SPARK-22974][ML] Attach attributes to output col...

2018-04-02 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/20313#discussion_r178517391 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/CountVectorizer.scala --- @@ -264,7 +265,9 @@ class CountVectorizerModel(

[GitHub] spark pull request #20930: [SPARK-23811][Core] FetchFailed comes before Succ...

2018-04-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20930#discussion_r178537775 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -794,6 +794,19 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-04-02 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178495813 --- Diff: python/pyspark/sql/readwriter.py --- @@ -237,6 +237,8 @@ def json(self, path, schema=None, primitivesAsString=None, prefersDecimal=None,

[GitHub] spark issue #20931: [SPARK-23815][Core]Spark writer dynamic partition overwr...

2018-04-02 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20931 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20954: [BACKPORT][SPARK-23040][CORE] Returns interruptib...

2018-04-02 Thread jiangxb1987
Github user jiangxb1987 closed the pull request at: https://github.com/apache/spark/pull/20954 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20940: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20940 **[Test build #88800 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88800/testReport)** for PR 20940 at commit

[GitHub] spark issue #20940: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20940 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88800/ Test PASSed. ---

[GitHub] spark issue #20940: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20940 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20957: Branch 2.3

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20957 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-04-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r178498692 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java --- @@ -20,49 +20,78 @@ import

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-04-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178496032 --- Diff: python/pyspark/sql/readwriter.py --- @@ -237,6 +237,8 @@ def json(self, path, schema=None, primitivesAsString=None, prefersDecimal=None,

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-04-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r178496095 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java --- @@ -20,49 +20,78 @@ import

[GitHub] spark issue #20930: [SPARK-23811][Core] FetchFailed comes before Success of ...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20930 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20930: [SPARK-23811][Core] FetchFailed comes before Success of ...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20930 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1895/

[GitHub] spark issue #20930: [SPARK-23811][Core] FetchFailed comes before Success of ...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20930 **[Test build #88806 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88806/testReport)** for PR 20930 at commit

[GitHub] spark issue #20886: [SPARK-19724][SQL]create a managed table with an existed...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20886 **[Test build #88807 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88807/testReport)** for PR 20886 at commit

[GitHub] spark issue #20886: [SPARK-19724][SQL]create a managed table with an existed...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20886 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1896/

[GitHub] spark issue #20886: [SPARK-19724][SQL]create a managed table with an existed...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20886 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-04-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r178582307 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -289,27 +295,19 @@ private[csv] object

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20959 **[Test build #88818 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88818/testReport)** for PR 20959 at commit

[GitHub] spark pull request #20942: [SPARK-15009][PYTHON][FOLLOWUP] Add default param...

2018-04-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20942 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20961: [SPARK-23823][SQL] Keep origin in transformExpres...

2018-04-02 Thread JiahuiJiang
GitHub user JiahuiJiang opened a pull request: https://github.com/apache/spark/pull/20961 [SPARK-23823][SQL] Keep origin in transformExpression Fixes https://issues.apache.org/jira/browse/SPARK-23823 Keep origin for all the methods using transformExpression ## What

[GitHub] spark issue #20928: Fix small typo in configuration doc

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20928 **[Test build #88820 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88820/testReport)** for PR 20928 at commit

[GitHub] spark issue #20928: Fix small typo in configuration doc

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20928 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88820/ Test PASSed. ---

[GitHub] spark issue #20928: Fix small typo in configuration doc

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20928 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20962: [SPARK-23847][PYTHON][SQL]Add asc_nulls_first, asc_nulls...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20962 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1909/

[GitHub] spark issue #20962: [SPARK-23847][PYTHON][SQL]Add asc_nulls_first, asc_nulls...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20962 **[Test build #88825 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88825/testReport)** for PR 20962 at commit

[GitHub] spark pull request #20553: [SPARK-23285][K8S] Add a config property for spec...

2018-04-02 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20553#discussion_r178612630 --- Diff: docs/running-on-kubernetes.md --- @@ -549,14 +549,23 @@ specific to Spark on Kubernetes. spark.kubernetes.driver.limit.cores

[GitHub] spark issue #20963: [SPARK-23849][SQL] Tests for the samplingRatio option of...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20963 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-04-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20850 @hvanhovell Here are results of `UnsafeProjectionBenchmark`. I have not seen the regression. With master `529f847105fa8d98a5dc4d20955e4870df6bc1c5` ``` OpenJDK 64-Bit Server VM

[GitHub] spark issue #20753: [SPARK-23582][SQL] StaticInvoke should support interpret...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20753 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-04-02 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/20910 @vanzin can you take a look? Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-04-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r178581875 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1279,4 +1279,62 @@ class CSVSuite extends

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-04-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r178584381 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -236,38 +236,44 @@ private[csv] object

[GitHub] spark issue #20722: [SPARK-23571][K8S] Delete auxiliary Kubernetes resources...

2018-04-02 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/20722 @felixcheung I think we want to move resource management to the driver such that the life time of the resources is bound to the life time of the driver pod. This works regardless of if the user

[GitHub] spark issue #20942: [SPARK-15009][PYTHON][FOLLOWUP] Add default param checks...

2018-04-02 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20942 merged to master, thanks @holdenk and @WeichenXu123 ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20858: [SPARK-23736][SQL] Extending the concat function to supp...

2018-04-02 Thread mn-mikke
Github user mn-mikke commented on the issue: https://github.com/apache/spark/pull/20858 Any other comments? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20908: [WIP][SPARK-23672][PYTHON] Document support for n...

2018-04-02 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/20908#discussion_r178606547 --- Diff: python/pyspark/sql/tests.py --- @@ -3966,6 +3967,23 @@ def random_udf(v): random_udf = random_udf.asNondeterministic()

[GitHub] spark issue #20753: [SPARK-23582][SQL] StaticInvoke should support interpret...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20753 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1906/

[GitHub] spark issue #20907: [WIP][SPARK-11237][ML] Add pmml export for k-means in Sp...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20907 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20907: [WIP][SPARK-11237][ML] Add pmml export for k-means in Sp...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20907 **[Test build #88823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88823/testReport)** for PR 20907 at commit

[GitHub] spark issue #20907: [WIP][SPARK-11237][ML] Add pmml export for k-means in Sp...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20907 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1907/

[GitHub] spark issue #20962: [SPARK-23847][PYTHON][SQL]Add asc_nulls_first, asc_nulls...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20962 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20553: [SPARK-23285][K8S] Add a config property for specifying ...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20553 **[Test build #88826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88826/testReport)** for PR 20553 at commit

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20959 **[Test build #88827 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88827/testReport)** for PR 20959 at commit

[GitHub] spark issue #20553: [SPARK-23285][K8S] Add a config property for specifying ...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20553 **[Test build #88826 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88826/testReport)** for PR 20553 at commit

[GitHub] spark pull request #20963: [SPARK-23849][SQL] Tests for the samplingRatio op...

2018-04-02 Thread MaxGekk
GitHub user MaxGekk opened a pull request: https://github.com/apache/spark/pull/20963 [SPARK-23849][SQL] Tests for the samplingRatio option of JSON datasource ## What changes were proposed in this pull request? Proposed tests checks that only subset of input dataset is

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20959 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88818/ Test FAILed. ---

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20959 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20939: [SPARK-23823][SQL] ResolveReferences should preserve tre...

2018-04-02 Thread JiahuiJiang
Github user JiahuiJiang commented on the issue: https://github.com/apache/spark/pull/20939 @gatorsmile Seems the tests are still flaking :/ And @robertzk proposed this fix https://github.com/JiahuiJiang/spark/pull/2 instead, where we keep the origin directly inside

[GitHub] spark pull request #20962: [SPARK-23847][PYTHON][SQL]Add asc_nulls_first, as...

2018-04-02 Thread huaxingao
GitHub user huaxingao opened a pull request: https://github.com/apache/spark/pull/20962 [SPARK-23847][PYTHON][SQL]Add asc_nulls_first, asc_nulls_last to PySpark ## What changes were proposed in this pull request? Column.scala and Functions.scala have asc_nulls_first,

[GitHub] spark issue #20908: [WIP][SPARK-23672][PYTHON] Document support for nested r...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20908 **[Test build #88824 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88824/testReport)** for PR 20908 at commit

[GitHub] spark issue #20908: [WIP][SPARK-23672][PYTHON] Document support for nested r...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20908 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1908/

[GitHub] spark issue #20908: [WIP][SPARK-23672][PYTHON] Document support for nested r...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20908 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20908: [WIP][SPARK-23672][PYTHON] Document support for nested r...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20908 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20908: [WIP][SPARK-23672][PYTHON] Document support for nested r...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20908 **[Test build #88824 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88824/testReport)** for PR 20908 at commit

[GitHub] spark issue #20908: [WIP][SPARK-23672][PYTHON] Document support for nested r...

2018-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20908 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88824/ Test FAILed. ---

[GitHub] spark pull request #20943: [ SPARK-23825] [K8s] Requesting memory + memory o...

2018-04-02 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/20943#discussion_r178615678 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/steps/BasicDriverConfigurationStepSuite.scala --- @@ -91,7

[GitHub] spark pull request #20553: [SPARK-23285][K8S] Add a config property for spec...

2018-04-02 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20553#discussion_r178618810 --- Diff: docs/running-on-kubernetes.md --- @@ -549,14 +549,22 @@ specific to Spark on Kubernetes. spark.kubernetes.driver.limit.cores

[GitHub] spark issue #20963: [SPARK-23849][SQL] Tests for the samplingRatio option of...

2018-04-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20963 **[Test build #88828 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88828/testReport)** for PR 20963 at commit

  1   2   3   >