[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-04-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178478627 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2127,4 +2127,243 @@ class JsonSuite extends

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-04-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178475983 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/CreateJacksonParser.scala --- @@ -39,11 +40,36 @@ private[sql] object

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-04-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178476459 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala --- @@ -361,6 +361,15 @@ class JacksonParser(

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-04-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178478647 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2127,4 +2127,243 @@ class JsonSuite extends

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-04-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178478996 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2127,4 +2127,243 @@ class JsonSuite extends

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-04-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178478271 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonDataSource.scala --- @@ -175,33 +185,43 @@ object

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-04-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178475654 --- Diff: python/pyspark/sql/readwriter.py --- @@ -237,6 +237,8 @@ def json(self, path, schema=None, primitivesAsString=None, prefersDecimal=None,

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-04-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178477311 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala --- @@ -107,4 +126,9 @@ private[sql] class JSONOptions(

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-04-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178476391 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala --- @@ -86,14 +85,34 @@ private[sql] class JSONOptions(

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-04-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178478616 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2127,4 +2127,243 @@ class JsonSuite extends

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1893/

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-04-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20850 **[Test build #88801 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88801/testReport)** for PR 20850 at commit

[GitHub] spark issue #20897: [MINOR][DOC] Fix a few markdown typos

2018-04-01 Thread Lemonjing
Github user Lemonjing commented on the issue: https://github.com/apache/spark/pull/20897 see http://spark.apache.org/docs/latest/ml-features.html#elementwiseproduct --- - To unsubscribe, e-mail:

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-04-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20697 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1867/ ---

[GitHub] spark issue #20956: [SPARK-23841][ML] NodeIdCache should unpersist the last ...

2018-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20956 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88805/ Test PASSed. ---

[GitHub] spark issue #20956: [SPARK-23841][ML] NodeIdCache should unpersist the last ...

2018-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20956 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20956: [SPARK-23841][ML] NodeIdCache should unpersist the last ...

2018-04-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20956 **[Test build #88805 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88805/testReport)** for PR 20956 at commit

[GitHub] spark pull request #20897: [MINOR][DOC] Fix a few markdown typos

2018-04-01 Thread Lemonjing
Github user Lemonjing commented on a diff in the pull request: https://github.com/apache/spark/pull/20897#discussion_r178481268 --- Diff: docs/mllib-pmml-model-export.md --- @@ -7,15 +7,15 @@ displayTitle: PMML model export - RDD-based API * Table of contents {:toc}

[GitHub] spark issue #20897: [MINOR][DOC] Fix a few markdown typos

2018-04-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20897 **[Test build #88802 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88802/testReport)** for PR 20897 at commit

[GitHub] spark pull request #20897: [MINOR][DOC] Fix a few markdown typos

2018-04-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20897#discussion_r178481415 --- Diff: docs/mllib-pmml-model-export.md --- @@ -7,15 +7,15 @@ displayTitle: PMML model export - RDD-based API * Table of contents {:toc}

[GitHub] spark issue #20937: [SPARK-23723][SPARK-23724][SQL] Support custom encoding ...

2018-04-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20937 @cloud-fan and @gatorsmile, just for doubly sure, you guys are at least positive on this feature itself, right? --- - To

[GitHub] spark issue #20897: [MINOR][DOC] Fix a few markdown typos

2018-04-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20897 **[Test build #88803 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88803/testReport)** for PR 20897 at commit

[GitHub] spark issue #20897: [MINOR][DOC] Fix a few markdown typos

2018-04-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20897 **[Test build #88802 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88802/testReport)** for PR 20897 at commit

[GitHub] spark issue #20897: [MINOR][DOC] Fix a few markdown typos

2018-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20897 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20897: [MINOR][DOC] Fix a few markdown typos

2018-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20897 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88802/ Test PASSed. ---

[GitHub] spark issue #20897: [MINOR][DOC] Fix a few markdown typos

2018-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20897 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88803/ Test PASSed. ---

[GitHub] spark issue #20897: [MINOR][DOC] Fix a few markdown typos

2018-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20897 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20897: [MINOR][DOC] Fix a few markdown typos

2018-04-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20897 **[Test build #88803 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88803/testReport)** for PR 20897 at commit

[GitHub] spark issue #20897: [MINOR][DOC] Fix a few markdown typos

2018-04-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20897 **[Test build #88804 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88804/testReport)** for PR 20897 at commit

[GitHub] spark pull request #20916: [SPARK-19964][core] Avoid reading from remote rep...

2018-04-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20916#discussion_r178483438 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -106,6 +106,9 @@ class SparkSubmitSuite // Necessary to make

[GitHub] spark issue #20897: [MINOR][DOC] Fix a few markdown typos

2018-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20897 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88804/ Test PASSed. ---

[GitHub] spark issue #20897: [MINOR][DOC] Fix a few markdown typos

2018-04-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20897 **[Test build #88804 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88804/testReport)** for PR 20897 at commit

[GitHub] spark issue #20897: [MINOR][DOC] Fix a few markdown typos

2018-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20897 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20897: [MINOR][DOC] Fix a few markdown typos

2018-04-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20897 @felixcheung, does this look good to you too? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20956: [SPARK-23841][ML] NodeIdCache should unpersist th...

2018-04-01 Thread zhengruifeng
GitHub user zhengruifeng opened a pull request: https://github.com/apache/spark/pull/20956 [SPARK-23841][ML] NodeIdCache should unpersist the last cached nodeIdsForInstances ## What changes were proposed in this pull request? unpersist the last cached nodeIdsForInstances in

[GitHub] spark issue #20956: [SPARK-23841][ML] NodeIdCache should unpersist the last ...

2018-04-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20956 **[Test build #88805 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88805/testReport)** for PR 20956 at commit

[GitHub] spark issue #20956: [SPARK-23841][ML] NodeIdCache should unpersist the last ...

2018-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20956 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20956: [SPARK-23841][ML] NodeIdCache should unpersist the last ...

2018-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20956 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1894/

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-04-01 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-04-01 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r178469520 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/GenerateColumnAccessor.scala --- @@ -212,11 +210,11 @@ object

[GitHub] spark pull request #20886: [SPARK-19724][SQL]create a managed table with an ...

2018-04-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20886#discussion_r178470268 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -298,15 +302,28 @@ class SessionCatalog(

[GitHub] spark pull request #20886: [SPARK-19724][SQL]create a managed table with an ...

2018-04-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20886#discussion_r178470264 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -298,15 +302,28 @@ class SessionCatalog(

[GitHub] spark issue #20886: [SPARK-19724][SQL]create a managed table with an existed...

2018-04-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20886 This is a behavior change, we need to make it configurable and also document it in the migration guide. --- - To

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-04-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20697 **[Test build #88798 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88798/testReport)** for PR 20697 at commit

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88798/ Test PASSed. ---

[GitHub] spark pull request #20930: [SPARK-23811][Core] FetchFailed comes before Succ...

2018-04-01 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20930#discussion_r178470893 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -794,6 +794,19 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request #20795: [SPARK-23486]cache the function name from the cat...

2018-04-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20795#discussion_r178471847 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -1072,8 +1072,17 @@ class SessionCatalog(

[GitHub] spark issue #20886: [SPARK-19724][SQL]create a managed table with an existed...

2018-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20886 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20886: [SPARK-19724][SQL]create a managed table with an existed...

2018-04-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20886 **[Test build #88799 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88799/testReport)** for PR 20886 at commit

[GitHub] spark pull request #20886: [SPARK-19724][SQL]create a managed table with an ...

2018-04-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20886#discussion_r178470143 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -103,6 +103,60 @@ class InMemoryCatalogedDDLSuite

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-04-01 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r178470952 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeArrayWriter.java --- @@ -32,141 +30,123 @@ */

[GitHub] spark pull request #20795: [SPARK-23486]cache the function name from the cat...

2018-04-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20795#discussion_r178471806 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -1072,8 +1072,17 @@ class SessionCatalog(

[GitHub] spark pull request #20795: [SPARK-23486]cache the function name from the cat...

2018-04-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20795#discussion_r178471822 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -1072,8 +1072,17 @@ class SessionCatalog(

[GitHub] spark pull request #20886: [SPARK-19724][SQL]create a managed table with an ...

2018-04-01 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/20886#discussion_r178472405 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -298,15 +302,28 @@ class SessionCatalog(

[GitHub] spark issue #20886: [SPARK-19724][SQL]create a managed table with an existed...

2018-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20886 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1892/

<    1   2