[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22394 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95970/ Test PASSed. ---

[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22394 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22388 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22388 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22344: [SPARK-25352][SQL] Perform ordered global limit when lim...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22344 **[Test build #95966 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95966/testReport)** for PR 22344 at commit

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-09-11 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r216874530 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -414,9 +425,48 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #22344: [SPARK-25352][SQL] Perform ordered global limit when lim...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22344 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22344: [SPARK-25352][SQL] Perform ordered global limit when lim...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22344 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95966/ Test FAILed. ---

[GitHub] spark pull request #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warni...

2018-09-11 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/22400 [SPARK-25238][PYTHON] lint-python: Fix W605 warnings for pycodestyle 2.4 (This change is a subset of the changes needed for the JIRA; see https://github.com/apache/spark/pull/22231) ##

[GitHub] spark pull request #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warni...

2018-09-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22400#discussion_r216886428 --- Diff: dev/run-tests-jenkins.py --- @@ -115,7 +115,7 @@ def run_tests(tests_timeout):

[GitHub] spark issue #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warnings for...

2018-09-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22400 CC @cclauss @holdenk --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warnings for...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22400 **[Test build #95973 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95973/testReport)** for PR 22400 at commit

[GitHub] spark issue #13206: [SPARK-15420] [SQL] Add repartition and sort to prepare ...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13206 **[Test build #95964 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95964/consoleFull)** for PR 13206 at commit

[GitHub] spark issue #13206: [SPARK-15420] [SQL] Add repartition and sort to prepare ...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13206 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #13206: [SPARK-15420] [SQL] Add repartition and sort to prepare ...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13206 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95964/ Test FAILed. ---

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22337 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95958/ Test PASSed. ---

[GitHub] spark issue #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22379 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22379 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95963/ Test FAILed. ---

[GitHub] spark pull request #22375: [WIP][SPARK-25388][Test][SQL] Detect incorrect nu...

2018-09-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r216868299 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,8 @@ trait

[GitHub] spark pull request #22375: [WIP][SPARK-25388][Test][SQL] Detect incorrect nu...

2018-09-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r216868343 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,8 @@ trait

[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-11 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/22353 ping @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warni...

2018-09-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22400#discussion_r216886496 --- Diff: python/pyspark/sql/functions.py --- @@ -283,7 +283,8 @@ def approxCountDistinct(col, rsd=None): @since(2.1) def

[GitHub] spark pull request #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warni...

2018-09-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22400#discussion_r216886472 --- Diff: python/pyspark/ml/feature.py --- @@ -303,7 +303,7 @@ def _create_model(self, java_model): class

[GitHub] spark issue #22344: [SPARK-25352][SQL] Perform ordered global limit when lim...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22344 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22344: [SPARK-25352][SQL] Perform ordered global limit when lim...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22344 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22344: [SPARK-25352][SQL] Perform ordered global limit when lim...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22344 **[Test build #95974 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95974/testReport)** for PR 22344 at commit

[GitHub] spark issue #22357: [SPARK-25363][SQL] Fix schema pruning in where clause by...

2018-09-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22357 LGTM from me too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22343: [SPARK-25391][SQL] Make behaviors consistent when conver...

2018-09-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22343 Compatibility is not a gold rule if it sacrifices correctness. Fast and **wrong** result doesn't looks like benefits to me. Do you think the customer want to get a wrong result like Hive?

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95960/ Test PASSed. ---

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95960 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95960/testReport)** for PR 21596 at commit

[GitHub] spark issue #22355: [SPARK-25358][SQL] MutableProjection supports fallback t...

2018-09-11 Thread sadhen
Github user sadhen commented on the issue: https://github.com/apache/spark/pull/22355 @maropu there are tips to view the codegen: > spark.sql("explain codegen select 1 + 1").show(false) You may explain an aggregation, and the NoOp check is obvious. ---

[GitHub] spark issue #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22379 **[Test build #95963 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95963/testReport)** for PR 22379 at commit

[GitHub] spark pull request #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite....

2018-09-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22394#discussion_r216875122 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala --- @@ -245,28 +249,32 @@ class

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-11 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22376 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22386: [SPARK-25399][SS] Continuous processing state should not...

2018-09-11 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/22386 Great thanks for your comment and fix @mukulmurthy! We'll also port this soon. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22376 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/3031/ ---

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22376 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22344: [SPARK-25352][SQL] Perform ordered global limit when lim...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22344 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22376 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/3031/ ---

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22376 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22344: [SPARK-25352][SQL] Perform ordered global limit when lim...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22344 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22358: [SPARK-25366][SQL]Zstd and brotli CompressionCode...

2018-09-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22358#discussion_r216881788 --- Diff: docs/sql-programming-guide.md --- @@ -965,6 +965,8 @@ Configuration of Parquet can be done using the `setConf` method on `SparkSession

[GitHub] spark issue #22393: [MINOR][DOCS] Axe deprecated doc refs

2018-09-11 Thread MichaelChirico
Github user MichaelChirico commented on the issue: https://github.com/apache/spark/pull/22393 @srowen thanks; done --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warnings for...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22400 **[Test build #95973 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95973/testReport)** for PR 22400 at commit

[GitHub] spark issue #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warnings for...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22400 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95973/ Test FAILed. ---

[GitHub] spark issue #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warnings for...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22400 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22399#discussion_r216896517 --- Diff: launcher/src/main/java/org/apache/spark/launcher/AbstractAppHandle.java --- @@ -72,11 +74,7 @@ public void stop() { @Override public

[GitHub] spark issue #22344: [SPARK-25352][SQL] Perform ordered global limit when lim...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22344 **[Test build #95966 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95966/testReport)** for PR 22344 at commit

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22388 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95967/ Test FAILed. ---

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22388 **[Test build #95967 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95967/testReport)** for PR 22388 at commit

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22388 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite....

2018-09-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22394#discussion_r216874181 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala --- @@ -156,20 +156,24 @@ class

[GitHub] spark issue #22054: [SPARK-24703][SQL]: To add support to multiply CalendarI...

2018-09-11 Thread priyankagargnitk
Github user priyankagargnitk commented on the issue: https://github.com/apache/spark/pull/22054 PLease review this PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #22386: [SPARK-25399][SS] Continuous processing state sho...

2018-09-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22386 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22348: [SPARK-25354][SQL] Reduce unneeded operation in n...

2018-09-11 Thread SongYadong
Github user SongYadong closed the pull request at: https://github.com/apache/spark/pull/22348 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22348: [SPARK-25354][SQL] Reduce unneeded operation in nextKeyV...

2018-09-11 Thread SongYadong
Github user SongYadong commented on the issue: https://github.com/apache/spark/pull/22348 Sounds reasonable. I will close this PR. Thank you! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22394 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22379 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22394 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22394 **[Test build #95970 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95970/testReport)** for PR 22394 at commit

[GitHub] spark issue #22393: Axe deprecated doc refs

2018-09-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22393 CC @felixcheung I'd also prefix the title with `[MINOR][DOCS]` to match our conventions --- - To unsubscribe, e-mail:

[GitHub] spark issue #22398: [SPARK-23820][CORE] Enable use of long form of callsite ...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22398 **[Test build #4336 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4336/testReport)** for PR 22398 at commit

[GitHub] spark issue #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warnings for...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22400 **[Test build #95975 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95975/testReport)** for PR 22400 at commit

[GitHub] spark issue #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warnings for...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22400 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

2018-09-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22388 Incorrect import still may exist there. ``` [error]

[GitHub] spark issue #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warnings for...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22400 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...

2018-09-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22394 Hey @mallman, let's just target to fix the problem in the JIRA without other refactorings. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite....

2018-09-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22394#discussion_r216903678 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala --- @@ -245,28 +249,32 @@ class

[GitHub] spark issue #22390: [SPARK-25402][SQL] Null handling in BooleanSimplificatio...

2018-09-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22390 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22390: [SPARK-25402][SQL] Null handling in BooleanSimplificatio...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22390 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22390: [SPARK-25402][SQL] Null handling in BooleanSimplificatio...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22390 **[Test build #95976 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95976/testReport)** for PR 22390 at commit

[GitHub] spark issue #22390: [SPARK-25402][SQL] Null handling in BooleanSimplificatio...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22390 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22337 **[Test build #95958 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95958/testReport)** for PR 22337 at commit

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22376 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/22341 Thanks @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22376 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95962/ Test FAILed. ---

[GitHub] spark issue #22358: [SPARK-25366][SQL]Zstd and brotli CompressionCodec are n...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22358 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22358: [SPARK-25366][SQL]Zstd and brotli CompressionCode...

2018-09-11 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22358#discussion_r216873177 --- Diff: docs/sql-programming-guide.md --- @@ -965,6 +965,8 @@ Configuration of Parquet can be done using the `setConf` method on `SparkSession

[GitHub] spark issue #22376: [SPARK-25021][K8S][BACKPORT] Add spark.executor.pyspark....

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22376 **[Test build #95971 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95971/testReport)** for PR 22376 at commit

[GitHub] spark pull request #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite....

2018-09-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22394#discussion_r216875341 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala --- @@ -245,28 +249,32 @@ class

[GitHub] spark pull request #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite....

2018-09-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22394#discussion_r216875288 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala --- @@ -245,28 +249,32 @@ class

[GitHub] spark issue #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new execut...

2018-09-11 Thread Ngone51
Github user Ngone51 commented on the issue: https://github.com/apache/spark/pull/22288 As I mentioned at https://github.com/apache/spark/pull/22288#discussion_r216874530, I'm quite worry about this killing behaviour. I thik we should kill a executor iff it is idle. By

[GitHub] spark issue #22344: [SPARK-25352][SQL] Perform ordered global limit when lim...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22344 **[Test build #95972 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95972/testReport)** for PR 22344 at commit

[GitHub] spark pull request #20414: [SPARK-23243][SQL] Shuffle+Repartition on an RDD ...

2018-09-11 Thread jiangxb1987
Github user jiangxb1987 closed the pull request at: https://github.com/apache/spark/pull/20414 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20611: [SPARK-23425][SQL]Support wildcard in HDFS path f...

2018-09-11 Thread sujith71955
Github user sujith71955 commented on a diff in the pull request: https://github.com/apache/spark/pull/20611#discussion_r216685077 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -303,94 +303,44 @@ case class LoadDataCommand(

[GitHub] spark pull request #22382: [SPARK-23243] [SPARK-20715][CORE][2.2] Fix RDD.re...

2018-09-11 Thread bersprockets
Github user bersprockets closed the pull request at: https://github.com/apache/spark/pull/22382 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22382: [SPARK-23243] [SPARK-20715][CORE][2.2] Fix RDD.repartiti...

2018-09-11 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/22382 Thanks! Closing. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20611: [SPARK-23425][SQL]Support wildcard in HDFS path f...

2018-09-11 Thread sujith71955
Github user sujith71955 commented on a diff in the pull request: https://github.com/apache/spark/pull/20611#discussion_r216693375 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -303,94 +303,44 @@ case class LoadDataCommand(

[GitHub] spark issue #22357: [SPARK-25363][SQL] Fix schema pruning in where clause by...

2018-09-11 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/22357 @viirya Please amend

[GitHub] spark pull request #22390: [SPARK-25402][SQL] Null handling in BooleanSimpli...

2018-09-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22390#discussion_r216713161 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -263,10 +263,12 @@ object BooleanSimplification

[GitHub] spark issue #22326: [SPARK-25314][SQL] Fix Python UDF accessing attributes f...

2018-09-11 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/22326 @cloud-fan Thanks for your comment. ``` IIUC, you are pulling out the join condition with python UDF and create a filter above join. Then the join become a cross join, which usually

[GitHub] spark issue #22395: [SPARK-16323][SQ] Add IntegralDivide expression

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22395 **[Test build #95954 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95954/testReport)** for PR 22395 at commit

[GitHub] spark issue #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow b...

2018-09-11 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22391 the failure seems persistent, though unrelated to this PR --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22386: [SPARK-25399] Continuous processing state should ...

2018-09-11 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22386#discussion_r216723459 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala --- @@ -1029,6 +1030,35 @@ class StreamSuite extends StreamTest {

[GitHub] spark issue #22348: [SPARK-25354][SQL] Reduce unneeded operation in nextKeyV...

2018-09-11 Thread SongYadong
Github user SongYadong commented on the issue: https://github.com/apache/spark/pull/22348 @dongjoon-hyun . You are right, DataSourceReadBenchmark result show the benefit is too small even in some cases is covered up by fluctuation. Java HotSpot(TM) 64-Bit Server VM

[GitHub] spark issue #22364: [SPARK-25379][SQL] Improve AttributeSet and ColumnPrunin...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22364 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22357: [SPARK-25363][SQL] Fix schema pruning in where clause by...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22357 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22213: [SPARK-25221][DEPLOY] Consistent trailing whitespace tre...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22213 **[Test build #95934 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95934/testReport)** for PR 22213 at commit

[GitHub] spark issue #22385: [SPARK-25400][CORE][TEST] Increase test timeouts

2018-09-11 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/22385 @dongjoon-hyun sure thing, done --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

<    1   2   3   4   5   6   7   >