[GitHub] spark pull request #21369: [SPARK-22713][CORE] ExternalAppendOnlyMap leaks w...

2018-05-21 Thread eyalfa
Github user eyalfa commented on a diff in the pull request: https://github.com/apache/spark/pull/21369#discussion_r189794046 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala --- @@ -305,8 +310,8 @@ class ExternalAppendOnlyMap[K, V, C](

[GitHub] spark pull request #21369: [SPARK-22713][CORE] ExternalAppendOnlyMap leaks w...

2018-05-21 Thread eyalfa
Github user eyalfa commented on a diff in the pull request: https://github.com/apache/spark/pull/21369#discussion_r189794097 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala --- @@ -585,17 +591,24 @@ class ExternalAppendOnlyMap[K, V, C](

[GitHub] spark issue #21367: [SPARK-24321][SQL] Extract common code from Divide/Remai...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21367 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3444/ Tes

[GitHub] spark issue #21367: [SPARK-24321][SQL] Extract common code from Divide/Remai...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21367 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21106 **[Test build #90939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90939/testReport)** for PR 21106 at commit [`ed525f6`](https://github.com/apache/spark/commit/ed

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21106 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3443/ Tes

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21367: [SPARK-24321][SQL] Extract common code from Divide/Remai...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21367 **[Test build #90938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90938/testReport)** for PR 21367 at commit [`951b5bd`](https://github.com/apache/spark/commit/95

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21389 **[Test build #90937 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90937/testReport)** for PR 21389 at commit [`00208be`](https://github.com/apache/spark/commit/00

[GitHub] spark pull request #21380: [SPARK-24329][SQL] Remove comments filtering befo...

2018-05-21 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21380#discussion_r189791852 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -196,7 +198,7 @@ class UnivocityParser(

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90932/ Test FAILed. ---

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21106 **[Test build #90932 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90932/testReport)** for PR 21106 at commit [`ed525f6`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #21367: [SPARK-24321][SQL] Extract common code from Divide/Remai...

2018-05-21 Thread rednaxelafx
Github user rednaxelafx commented on the issue: https://github.com/apache/spark/pull/21367 Updated PR, addressed @cloud-fan and @viirya 's comments. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21369: [SPARK-22713][CORE] ExternalAppendOnlyMap leaks when spi...

2018-05-21 Thread JerryLead
Github user JerryLead commented on the issue: https://github.com/apache/spark/pull/21369 Thank you all for fixing this issue. I'm sorry that I'm now writing a research paper about Spark GC and going to submit this paper next month. Since there is a lot of paper work to do, the verific

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90930/ Test PASSed. ---

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21366 **[Test build #90930 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90930/testReport)** for PR 21366 at commit [`caffe23`](https://github.com/apache/spark/commit/c

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-21 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r189785968 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeBlockSuite.scala --- @@ -23,12 +23,20 @@ import org.apache.

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-21 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r189786155 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -167,9 +170,40 @@ object Block {

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-21 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r189786071 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -3626,6 +3626,125 @@ object functions { */ def map_values(e: Colu

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21319 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21319 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90933/ Test FAILed. ---

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21319 **[Test build #90933 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90933/testReport)** for PR 21319 at commit [`3c5d175`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90934/ Test FAILed. ---

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21389 **[Test build #90934 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90934/testReport)** for PR 21389 at commit [`0d88bcb`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21368: [SPARK-16451][repl] Fail shell if SparkSession fa...

2018-05-21 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21368#discussion_r189781578 --- Diff: python/pyspark/sql/session.py --- @@ -547,6 +547,40 @@ def _create_from_pandas_with_arrow(self, pdf, schema, timezone): df._schem

[GitHub] spark pull request #21368: [SPARK-16451][repl] Fail shell if SparkSession fa...

2018-05-21 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21368#discussion_r189781240 --- Diff: repl/scala-2.12/src/main/scala/org/apache/spark/repl/SparkILoop.scala --- @@ -37,7 +37,14 @@ class SparkILoop(in0: Option[BufferedReader], out

[GitHub] spark pull request #21343: [SPARK-24292][SQL] Proxy user cannot connect to H...

2018-05-21 Thread LantaoJin
Github user LantaoJin closed the pull request at: https://github.com/apache/spark/pull/21343 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21343: [SPARK-24292][SQL] Proxy user cannot connect to HiveMeta...

2018-05-21 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/21343 Cool, SPARK-23639 also works for me. Close. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional c

[GitHub] spark issue #18894: [SPARK-21673] Use the correct sandbox environment variab...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18894 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90927/ Test PASSed. ---

[GitHub] spark issue #18894: [SPARK-21673] Use the correct sandbox environment variab...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18894 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18894: [SPARK-21673] Use the correct sandbox environment variab...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18894 **[Test build #90927 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90927/testReport)** for PR 18894 at commit [`4ccb4be`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #21380: [SPARK-24329][SQL] Remove comments filtering befo...

2018-05-21 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21380#discussion_r189780841 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -300,14 +302,11 @@ private[csv] object Un

[GitHub] spark pull request #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmar...

2018-05-21 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21288#discussion_r189780637 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/FilterPushdownBenchmark.scala --- @@ -0,0 +1,437 @@ +/* + * Licensed to t

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21193 **[Test build #90931 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90931/testReport)** for PR 21193 at commit [`96c594a`](https://github.com/apache/spark/commit/96

[GitHub] spark pull request #21380: [SPARK-24329][SQL] Remove comments filtering befo...

2018-05-21 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21380#discussion_r189780079 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -300,14 +302,11 @@ private[csv] object Univoci

[GitHub] spark issue #21382: [SPARK-24332][SS][MESOS]Fix places reading 'spark.networ...

2018-05-21 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21382 @susanxhuynh --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-05-21 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19691 You don't address all the comments? e.g., how about the @mgaido91 comment? https://github.com/apache/spark/pull/19691/files#r179944192 Also, you need to add tests for a non-supported left-side lit

[GitHub] spark issue #21388: [SPARK-24336][SQL] Support 'pass through' transformation...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21388 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21388: [SPARK-24336][SQL] Support 'pass through' transformation...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21388 **[Test build #90922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90922/testReport)** for PR 21388 at commit [`139aefa`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-21 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r189767747 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -167,9 +170,40 @@ object Block {

[GitHub] spark issue #21388: [SPARK-24336][SQL] Support 'pass through' transformation...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21388 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90922/ Test PASSed. ---

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-21 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r189768074 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -167,9 +170,40 @@ object Block {

[GitHub] spark issue #21381: [SPARK-24330][SQL]Refactor ExecuteWriteTask in FileForma...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21381 **[Test build #90936 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90936/testReport)** for PR 21381 at commit [`8af4579`](https://github.com/apache/spark/commit/8a

[GitHub] spark issue #21381: [SPARK-24330][SQL]Refactor ExecuteWriteTask in FileForma...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21381 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3442/ Tes

[GitHub] spark issue #21381: [SPARK-24330][SQL]Refactor ExecuteWriteTask in FileForma...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21381 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21381: [SPARK-24330][SQL]Refactor ExecuteWriteTask in FileForma...

2018-05-21 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/21381 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90926/ Test PASSed. ---

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21366 **[Test build #90926 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90926/testReport)** for PR 21366 at commit [`c2b9733`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #21343: [SPARK-24292][SQL] Proxy user cannot connect to HiveMeta...

2018-05-21 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/21343 @vanzin Seems duplicated. Let me check. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

2018-05-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21319 Yea this is still a workaround before having stats in the physical plan. My major concern is, moving stats to physical plan is a huge change and I may not have enough free time to drive it. It wil

[GitHub] spark issue #21381: [SPARK-24330][SQL]Refactor ExecuteWriteTask in FileForma...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21381 **[Test build #90929 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90929/testReport)** for PR 21381 at commit [`8af4579`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21369: [SPARK-22713][CORE] ExternalAppendOnlyMap leaks w...

2018-05-21 Thread advancedxy
Github user advancedxy commented on a diff in the pull request: https://github.com/apache/spark/pull/21369#discussion_r189768075 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala --- @@ -305,8 +310,8 @@ class ExternalAppendOnlyMap[K, V, C](

[GitHub] spark issue #21381: [SPARK-24330][SQL]Refactor ExecuteWriteTask in FileForma...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21381 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21369: [SPARK-22713][CORE] ExternalAppendOnlyMap leaks w...

2018-05-21 Thread advancedxy
Github user advancedxy commented on a diff in the pull request: https://github.com/apache/spark/pull/21369#discussion_r189768116 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala --- @@ -585,17 +591,24 @@ class ExternalAppendOnlyMap[K, V, C

[GitHub] spark pull request #21369: [SPARK-22713][CORE] ExternalAppendOnlyMap leaks w...

2018-05-21 Thread advancedxy
Github user advancedxy commented on a diff in the pull request: https://github.com/apache/spark/pull/21369#discussion_r189768301 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala --- @@ -630,7 +643,7 @@ private[spark] object ExternalAppendO

[GitHub] spark pull request #21369: [SPARK-22713][CORE] ExternalAppendOnlyMap leaks w...

2018-05-21 Thread advancedxy
Github user advancedxy commented on a diff in the pull request: https://github.com/apache/spark/pull/21369#discussion_r189768335 --- Diff: core/src/test/scala/org/apache/spark/util/collection/ExternalAppendOnlyMapSuite.scala --- @@ -23,8 +23,9 @@ import org.apache.spark._ imp

[GitHub] spark pull request #21369: [SPARK-22713][CORE] ExternalAppendOnlyMap leaks w...

2018-05-21 Thread advancedxy
Github user advancedxy commented on a diff in the pull request: https://github.com/apache/spark/pull/21369#discussion_r189768271 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala --- @@ -585,17 +591,24 @@ class ExternalAppendOnlyMap[K, V, C

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3438/ Tes

[GitHub] spark issue #21381: [SPARK-24330][SQL]Refactor ExecuteWriteTask in FileForma...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21381 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90929/ Test FAILed. ---

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21319 **[Test build #90933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90933/testReport)** for PR 21319 at commit [`3c5d175`](https://github.com/apache/spark/commit/3c

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3439/ Tes

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21389 **[Test build #90934 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90934/testReport)** for PR 21389 at commit [`0d88bcb`](https://github.com/apache/spark/commit/0d

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21319 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21106 **[Test build #90932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90932/testReport)** for PR 21106 at commit [`ed525f6`](https://github.com/apache/spark/commit/ed

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21319 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3440/ Tes

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21106 @cloud-fan @rednaxelafx Accessing confs are now simpler. Please review this again. Thanks. --- - To unsubscribe, e-mail: reviews-

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-05-21 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/21389 [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFileFormat ## What changes were proposed in this pull request? This pr added code to verify a schema in Json/Orc/ParquetFileFormat along wit

[GitHub] spark issue #21045: [SPARK-23931][SQL] Adds zip function to sparksql

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21045 **[Test build #90935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90935/testReport)** for PR 21045 at commit [`caca5a9`](https://github.com/apache/spark/commit/ca

[GitHub] spark issue #21045: [SPARK-23931][SQL] Adds zip function to sparksql

2018-05-21 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21045 ok to test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark

[GitHub] spark issue #21385: [SPARK-24234][SS] Support multiple row writers in contin...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21385 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21379: [SPARK-24327][SQL] Add an option to quote a parti...

2018-05-21 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21379#discussion_r189774838 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRelation.scala --- @@ -78,7 +79,12 @@ private[sql] object JDBCRelation

[GitHub] spark issue #21385: [SPARK-24234][SS] Support multiple row writers in contin...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21385 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90923/ Test PASSed. ---

[GitHub] spark issue #21385: [SPARK-24234][SS] Support multiple row writers in contin...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21385 **[Test build #90923 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90923/testReport)** for PR 21385 at commit [`79c3158`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3441/ Tes

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21366 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3326/ ---

[GitHub] spark pull request #21368: [SPARK-16451][repl] Fail shell if SparkSession fa...

2018-05-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21368#discussion_r189762943 --- Diff: repl/src/main/scala/org/apache/spark/repl/Main.scala --- @@ -79,44 +81,50 @@ object Main extends Logging { } def createSparkSess

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21366 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3326/ --- --

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3437/ Tes

[GitHub] spark pull request #21368: [SPARK-16451][repl] Fail shell if SparkSession fa...

2018-05-21 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21368#discussion_r189760439 --- Diff: repl/src/main/scala/org/apache/spark/repl/Main.scala --- @@ -79,44 +81,50 @@ object Main extends Logging { } def createSpar

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21366 **[Test build #90930 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90930/testReport)** for PR 21366 at commit [`caffe23`](https://github.com/apache/spark/commit/ca

[GitHub] spark issue #21368: [SPARK-16451][repl] Fail shell if SparkSession fails to ...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21368 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90928/ Test PASSed. ---

[GitHub] spark issue #21368: [SPARK-16451][repl] Fail shell if SparkSession fails to ...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21368 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21368: [SPARK-16451][repl] Fail shell if SparkSession fails to ...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21368 **[Test build #90928 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90928/testReport)** for PR 21368 at commit [`6d53ca0`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #21380: [SPARK-24329][SQL] Remove comments filtering befo...

2018-05-21 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21380#discussion_r189759406 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -196,7 +198,7 @@ class UnivocityParser(

[GitHub] spark pull request #21380: [SPARK-24329][SQL] Remove comments filtering befo...

2018-05-21 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21380#discussion_r18975 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -300,14 +302,11 @@ private[csv] object Un

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-05-21 Thread DazhuangSu
Github user DazhuangSu commented on the issue: https://github.com/apache/spark/pull/19691 @maropu @mgaido91 path has passed all tests. Any advice please? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.a

[GitHub] spark issue #21381: [SPARK-24330][SQL]Refactor ExecuteWriteTask in FileForma...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21381 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3436/ Tes

[GitHub] spark issue #21381: [SPARK-24330][SQL]Refactor ExecuteWriteTask in FileForma...

2018-05-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21381 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21381: [SPARK-24330][SQL]Refactor ExecuteWriteTask in FileForma...

2018-05-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21381 **[Test build #90929 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90929/testReport)** for PR 21381 at commit [`8af4579`](https://github.com/apache/spark/commit/8a

[GitHub] spark pull request #21388: [SPARK-24336][SQL] Support 'pass through' transfo...

2018-05-21 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21388#discussion_r189756841 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -474,8 +477,80 @@ abstract class SparkStrategies extends

[GitHub] spark pull request #21388: [SPARK-24336][SQL] Support 'pass through' transfo...

2018-05-21 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21388#discussion_r189756939 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -474,8 +477,80 @@ abstract class SparkStrategies extends

  1   2   3   4   5   6   >