[GitHub] [spark] dongjoon-hyun commented on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

dongjoon-hyun commented on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494251030 Is there a reason to use that? > How about change hive.version.short to

[GitHub] [spark] cloud-fan commented on a change in pull request #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded

cloud-fan commented on a change in pull request #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded URL: https://github.com/apache/spark/pull/24655#discussion_r285861258 ## File path:

[GitHub] [spark] cloud-fan commented on a change in pull request #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded

cloud-fan commented on a change in pull request #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded URL: https://github.com/apache/spark/pull/24655#discussion_r285861100 ## File path:

[GitHub] [spark] SparkQA commented on issue #24335: [SPARK-27425][SQL] Add count_if functions

SparkQA commented on issue #24335: [SPARK-27425][SQL] Add count_if functions URL: https://github.com/apache/spark/pull/24335#issuecomment-494249556 **[Test build #105595 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105595/testReport)** for PR 24335 at

[GitHub] [spark] francis0407 commented on issue #24344: [SPARK-27440][SQL] Optimize uncorrelated predicate subquery

francis0407 commented on issue #24344: [SPARK-27440][SQL] Optimize uncorrelated predicate subquery URL: https://github.com/apache/spark/pull/24344#issuecomment-494249369 I'm not sure who is familiar with this, could we ping other reviewers? cc @cloud-fan, @viirya, @dilipbiswal

[GitHub] [spark] AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494249335 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA commented on issue #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded

SparkQA commented on issue #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded URL: https://github.com/apache/spark/pull/24655#issuecomment-494249540 **[Test build #105594 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494249339 Test PASSed. Refer to this link for build results (access rights

[GitHub] [spark] AmplabJenkins removed a comment on issue #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded

AmplabJenkins removed a comment on issue #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded URL: https://github.com/apache/spark/pull/24655#issuecomment-494249159 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #24335: [SPARK-27425][SQL] Add count_if functions

AmplabJenkins removed a comment on issue #24335: [SPARK-27425][SQL] Add count_if functions URL: https://github.com/apache/spark/pull/24335#issuecomment-494249204 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] SparkQA removed a comment on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

SparkQA removed a comment on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494198847 **[Test build #105585 has

[GitHub] [spark] AmplabJenkins commented on issue #24335: [SPARK-27425][SQL] Add count_if functions

AmplabJenkins commented on issue #24335: [SPARK-27425][SQL] Add count_if functions URL: https://github.com/apache/spark/pull/24335#issuecomment-494249204 Merged build finished. Test PASSed. This is an automated message from

[GitHub] [spark] AmplabJenkins removed a comment on issue #24335: [SPARK-27425][SQL] Add count_if functions

AmplabJenkins removed a comment on issue #24335: [SPARK-27425][SQL] Add count_if functions URL: https://github.com/apache/spark/pull/24335#issuecomment-494249210 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded

AmplabJenkins removed a comment on issue #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded URL: https://github.com/apache/spark/pull/24655#issuecomment-494249155 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded

AmplabJenkins commented on issue #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded URL: https://github.com/apache/spark/pull/24655#issuecomment-494249159 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494249339 Test PASSed. Refer to this link for build results (access rights to CI

[GitHub] [spark] AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494249335 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24335: [SPARK-27425][SQL] Add count_if functions

AmplabJenkins commented on issue #24335: [SPARK-27425][SQL] Add count_if functions URL: https://github.com/apache/spark/pull/24335#issuecomment-494249210 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded

AmplabJenkins commented on issue #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded URL: https://github.com/apache/spark/pull/24655#issuecomment-494249155 Merged build finished. Test PASSed.

[GitHub] [spark] cloud-fan closed pull request #24533: [SPARK-27637][Shuffle] For nettyBlockTransferService, if IOException occurred while fetching data, check whether relative executor is alive befo

cloud-fan closed pull request #24533: [SPARK-27637][Shuffle] For nettyBlockTransferService, if IOException occurred while fetching data, check whether relative executor is alive before retry URL: https://github.com/apache/spark/pull/24533

[GitHub] [spark] SparkQA commented on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

SparkQA commented on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494248935 **[Test build #105585 has

[GitHub] [spark] gatorsmile commented on issue #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded

gatorsmile commented on issue #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded URL: https://github.com/apache/spark/pull/24655#issuecomment-494248380 cc @rednaxelafx This is an automated

[GitHub] [spark] cloud-fan commented on issue #24533: [SPARK-27637][Shuffle] For nettyBlockTransferService, if IOException occurred while fetching data, check whether relative executor is alive befor

cloud-fan commented on issue #24533: [SPARK-27637][Shuffle] For nettyBlockTransferService, if IOException occurred while fetching data, check whether relative executor is alive before retry URL: https://github.com/apache/spark/pull/24533#issuecomment-494248312 thanks, merging to master!

[GitHub] [spark] HyukjinKwon commented on issue #24335: [SPARK-27425][SQL] Add count_if functions

HyukjinKwon commented on issue #24335: [SPARK-27425][SQL] Add count_if functions URL: https://github.com/apache/spark/pull/24335#issuecomment-494248222 retest this please This is an automated message from the Apache Git

[GitHub] [spark] SparkQA commented on issue #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded

SparkQA commented on issue #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded URL: https://github.com/apache/spark/pull/24655#issuecomment-494248027 **[Test build #105593 has

[GitHub] [spark] cloud-fan commented on a change in pull request #24654: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans

cloud-fan commented on a change in pull request #24654: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans URL: https://github.com/apache/spark/pull/24654#discussion_r285858384 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala

[GitHub] [spark] JoshRosen opened a new pull request #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded

JoshRosen opened a new pull request #24655: [SPARK-27786] Fix Sha1, Md5, and Base64 codegen when commons-codec is shaded URL: https://github.com/apache/spark/pull/24655 ## What changes were proposed in this pull request? When running a custom build of Spark which shades

[GitHub] [spark] HyukjinKwon commented on issue #24650: [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled

HyukjinKwon commented on issue #24650: [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled URL: https://github.com/apache/spark/pull/24650#issuecomment-494247197 Yea, SGTM. This is an

[GitHub] [spark] BryanCutler commented on issue #24650: [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled

BryanCutler commented on issue #24650: [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled URL: https://github.com/apache/spark/pull/24650#issuecomment-494245140 I had another thought about this, the stuff in `doAfterLastPartition` could be removed from

[GitHub] [spark] cryeo commented on issue #24335: [SPARK-27425][SQL] Add count_if functions

cryeo commented on issue #24335: [SPARK-27425][SQL] Add count_if functions URL: https://github.com/apache/spark/pull/24335#issuecomment-494240078 OK. I just did it :) This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series

AmplabJenkins removed a comment on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series URL: https://github.com/apache/spark/pull/24643#issuecomment-494238084 Test PASSed. Refer to this link for build

[GitHub] [spark] AmplabJenkins removed a comment on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative

AmplabJenkins removed a comment on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative URL: https://github.com/apache/spark/pull/24497#issuecomment-494237359 Test FAILed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled

AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled URL: https://github.com/apache/spark/pull/24650#issuecomment-494237335 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] dongjoon-hyun commented on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

dongjoon-hyun commented on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494237607 Ur, @wangyum . During merging, it seems that we missed to upgrade

[GitHub] [spark] AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled

AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled URL: https://github.com/apache/spark/pull/24650#issuecomment-494237332 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series

AmplabJenkins commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series URL: https://github.com/apache/spark/pull/24643#issuecomment-494238084 Test PASSed. Refer to this link for build results

[GitHub] [spark] AmplabJenkins removed a comment on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative

AmplabJenkins removed a comment on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative URL: https://github.com/apache/spark/pull/24497#issuecomment-494237355 Merged build finished. Test FAILed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series

AmplabJenkins removed a comment on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series URL: https://github.com/apache/spark/pull/24643#issuecomment-494238081 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series

AmplabJenkins commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series URL: https://github.com/apache/spark/pull/24643#issuecomment-494238081 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA commented on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative

SparkQA commented on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative URL: https://github.com/apache/spark/pull/24497#issuecomment-494237224 **[Test build #105589 has

[GitHub] [spark] SparkQA removed a comment on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative

SparkQA removed a comment on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative URL: https://github.com/apache/spark/pull/24497#issuecomment-494224638 **[Test build #105589 has

[GitHub] [spark] AmplabJenkins commented on issue #24650: [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled

AmplabJenkins commented on issue #24650: [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled URL: https://github.com/apache/spark/pull/24650#issuecomment-494237335 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #24650: [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled

SparkQA commented on issue #24650: [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled URL: https://github.com/apache/spark/pull/24650#issuecomment-494237043 **[Test build #105588 has

[GitHub] [spark] SparkQA removed a comment on issue #24650: [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled

SparkQA removed a comment on issue #24650: [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled URL: https://github.com/apache/spark/pull/24650#issuecomment-494208967 **[Test build #105588 has

[GitHub] [spark] wangyum commented on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

wangyum commented on issue #24620: [SPARK-27737][SQL][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494239108 How about change `hive.version.short` to 2.3.0?

[GitHub] [spark] SparkQA commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series

SparkQA commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series URL: https://github.com/apache/spark/pull/24643#issuecomment-494238420 **[Test build #105592 has

[GitHub] [spark] AmplabJenkins commented on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative

AmplabJenkins commented on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative URL: https://github.com/apache/spark/pull/24497#issuecomment-494237355 Merged build finished. Test FAILed.

[GitHub] [spark] AmplabJenkins commented on issue #24650: [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled

AmplabJenkins commented on issue #24650: [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled URL: https://github.com/apache/spark/pull/24650#issuecomment-494237332 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative

AmplabJenkins commented on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative URL: https://github.com/apache/spark/pull/24497#issuecomment-494237359 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] wenxuanguan commented on issue #24631: [SPARK-27774][CORE][MLLIB] Avoid hardcoded configs

wenxuanguan commented on issue #24631: [SPARK-27774][CORE][MLLIB] Avoid hardcoded configs URL: https://github.com/apache/spark/pull/24631#issuecomment-494234979 retest this please This is an automated message from the Apache

[GitHub] [spark] HyukjinKwon commented on issue #24335: [SPARK-27425][SQL] Add count_if functions

HyukjinKwon commented on issue #24335: [SPARK-27425][SQL] Add count_if functions URL: https://github.com/apache/spark/pull/24335#issuecomment-494234508 okie. can you rebase? This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins removed a comment on issue #24654: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans

AmplabJenkins removed a comment on issue #24654: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans URL: https://github.com/apache/spark/pull/24654#issuecomment-494230020 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24654: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans

AmplabJenkins removed a comment on issue #24654: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans URL: https://github.com/apache/spark/pull/24654#issuecomment-494230023 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #24654: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans

SparkQA commented on issue #24654: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans URL: https://github.com/apache/spark/pull/24654#issuecomment-494230349 **[Test build #105591 has

[GitHub] [spark] AmplabJenkins commented on issue #24654: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans

AmplabJenkins commented on issue #24654: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans URL: https://github.com/apache/spark/pull/24654#issuecomment-494230020 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24654: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans

AmplabJenkins commented on issue #24654: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans URL: https://github.com/apache/spark/pull/24654#issuecomment-494230023 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] viirya opened a new pull request #24654: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans

viirya opened a new pull request #24654: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans URL: https://github.com/apache/spark/pull/24654 ## What changes were proposed in this pull request? Because a review is resolved during analysis when we create a

[GitHub] [spark] viirya commented on issue #24654: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans

viirya commented on issue #24654: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans URL: https://github.com/apache/spark/pull/24654#issuecomment-494229725 cc @dongjoon-hyun @gatorsmile @hvanhovell

[GitHub] [spark] HyukjinKwon commented on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

HyukjinKwon commented on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#issuecomment-494227943 Looks fine given skimming the codes.

[GitHub] [spark] HyukjinKwon commented on a change in pull request #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

HyukjinKwon commented on a change in pull request #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#discussion_r285839287 ## File path:

[GitHub] [spark] AmplabJenkins removed a comment on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series

AmplabJenkins removed a comment on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series URL: https://github.com/apache/spark/pull/24643#issuecomment-494227322 Test FAILed. Refer to this link for build

[GitHub] [spark] AmplabJenkins removed a comment on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series

AmplabJenkins removed a comment on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series URL: https://github.com/apache/spark/pull/24643#issuecomment-494227317 Merged build finished. Test FAILed.

[GitHub] [spark] AmplabJenkins commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series

AmplabJenkins commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series URL: https://github.com/apache/spark/pull/24643#issuecomment-494227322 Test FAILed. Refer to this link for build results

[GitHub] [spark] SparkQA removed a comment on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series

SparkQA removed a comment on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series URL: https://github.com/apache/spark/pull/24643#issuecomment-494227049 **[Test build #105590 has

[GitHub] [spark] SparkQA commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series

SparkQA commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series URL: https://github.com/apache/spark/pull/24643#issuecomment-494227311 **[Test build #105590 has

[GitHub] [spark] AmplabJenkins commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series

AmplabJenkins commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series URL: https://github.com/apache/spark/pull/24643#issuecomment-494227317 Merged build finished. Test FAILed.

[GitHub] [spark] SparkQA commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series

SparkQA commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series URL: https://github.com/apache/spark/pull/24643#issuecomment-494227049 **[Test build #105590 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series

AmplabJenkins removed a comment on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series URL: https://github.com/apache/spark/pull/24643#issuecomment-494226759 Test PASSed. Refer to this link for build

[GitHub] [spark] AmplabJenkins removed a comment on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series

AmplabJenkins removed a comment on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series URL: https://github.com/apache/spark/pull/24643#issuecomment-494226755 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series

AmplabJenkins commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series URL: https://github.com/apache/spark/pull/24643#issuecomment-494226759 Test PASSed. Refer to this link for build results

[GitHub] [spark] AmplabJenkins commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series

AmplabJenkins commented on issue #24643: [SPARK-26412][PySpark][SQL][WIP] Allow Pandas UDF to take an iterator of pd.Series or an iterator of tuple of pd.Series URL: https://github.com/apache/spark/pull/24643#issuecomment-494226755 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA commented on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative

SparkQA commented on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative URL: https://github.com/apache/spark/pull/24497#issuecomment-494224638 **[Test build #105589 has

[GitHub] [spark] AmplabJenkins commented on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative

AmplabJenkins commented on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative URL: https://github.com/apache/spark/pull/24497#issuecomment-494224334 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative

AmplabJenkins removed a comment on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative URL: https://github.com/apache/spark/pull/24497#issuecomment-494224334 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins commented on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative

AmplabJenkins commented on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative URL: https://github.com/apache/spark/pull/24497#issuecomment-494224326 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative

AmplabJenkins removed a comment on issue #24497: [SPARK-27630][CORE]Stage retry causes totalRunningTasks calculation to be negative URL: https://github.com/apache/spark/pull/24497#issuecomment-494224326 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24631: [SPARK-27774][CORE][MLLIB] Avoid hardcoded configs

AmplabJenkins removed a comment on issue #24631: [SPARK-27774][CORE][MLLIB] Avoid hardcoded configs URL: https://github.com/apache/spark/pull/24631#issuecomment-494215397 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] beliefer commented on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource.

beliefer commented on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource. URL: https://github.com/apache/spark/pull/24647#issuecomment-494215705 > > I'm not sure about the contract here, whether providers are required to be stateless. > > Since

[GitHub] [spark] beliefer edited a comment on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource.

beliefer edited a comment on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource. URL: https://github.com/apache/spark/pull/24647#issuecomment-494214438 > I'm not sure about the contract here, whether providers are required to be stateless. If they're not then

[GitHub] [spark] AmplabJenkins removed a comment on issue #24631: [SPARK-27774][CORE][MLLIB] Avoid hardcoded configs

AmplabJenkins removed a comment on issue #24631: [SPARK-27774][CORE][MLLIB] Avoid hardcoded configs URL: https://github.com/apache/spark/pull/24631#issuecomment-494215389 Merged build finished. Test FAILed. This is an

[GitHub] [spark] AmplabJenkins commented on issue #24631: [SPARK-27774][CORE][MLLIB] Avoid hardcoded configs

AmplabJenkins commented on issue #24631: [SPARK-27774][CORE][MLLIB] Avoid hardcoded configs URL: https://github.com/apache/spark/pull/24631#issuecomment-494215397 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24631: [SPARK-27774][CORE][MLLIB] Avoid hardcoded configs

AmplabJenkins commented on issue #24631: [SPARK-27774][CORE][MLLIB] Avoid hardcoded configs URL: https://github.com/apache/spark/pull/24631#issuecomment-494215389 Merged build finished. Test FAILed. This is an automated

[GitHub] [spark] SparkQA removed a comment on issue #24631: [SPARK-27774][CORE][MLLIB] Avoid hardcoded configs

SparkQA removed a comment on issue #24631: [SPARK-27774][CORE][MLLIB] Avoid hardcoded configs URL: https://github.com/apache/spark/pull/24631#issuecomment-494201262 **[Test build #105586 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105586/testReport)**

[GitHub] [spark] SparkQA commented on issue #24631: [SPARK-27774][CORE][MLLIB] Avoid hardcoded configs

SparkQA commented on issue #24631: [SPARK-27774][CORE][MLLIB] Avoid hardcoded configs URL: https://github.com/apache/spark/pull/24631#issuecomment-494215242 **[Test build #105586 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105586/testReport)** for PR

[GitHub] [spark] beliefer commented on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource.

beliefer commented on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource. URL: https://github.com/apache/spark/pull/24647#issuecomment-494214438 > I'm not sure about the contract here, whether providers are required to be stateless. If they're not then this

[GitHub] [spark] AmplabJenkins removed a comment on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals

AmplabJenkins removed a comment on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals URL: https://github.com/apache/spark/pull/24593#issuecomment-494211702 Test PASSed. Refer to this link for build

[GitHub] [spark] AmplabJenkins removed a comment on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals

AmplabJenkins removed a comment on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals URL: https://github.com/apache/spark/pull/24593#issuecomment-494211699 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals

AmplabJenkins commented on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals URL: https://github.com/apache/spark/pull/24593#issuecomment-494211699 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals

AmplabJenkins commented on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals URL: https://github.com/apache/spark/pull/24593#issuecomment-494211702 Test PASSed. Refer to this link for build results

[GitHub] [spark] SparkQA removed a comment on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals

SparkQA removed a comment on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals URL: https://github.com/apache/spark/pull/24593#issuecomment-494181330 **[Test build #105583 has

[GitHub] [spark] SparkQA commented on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals

SparkQA commented on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals URL: https://github.com/apache/spark/pull/24593#issuecomment-494211385 **[Test build #105583 has

[GitHub] [spark] viirya commented on issue #24464: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans

viirya commented on issue #24464: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans URL: https://github.com/apache/spark/pull/24464#issuecomment-494210180 oh sorry for that and thanks @gatorsmile, @hvanhovell and @dongjoon-hyun I will follow up with

[GitHub] [spark] kiszk commented on issue #24636: [SPARK-27684][SQL] Avoid conversion overhead for primitive types

kiszk commented on issue #24636: [SPARK-27684][SQL] Avoid conversion overhead for primitive types URL: https://github.com/apache/spark/pull/24636#issuecomment-494210096 Good PR. I will review this carefully. One minor comment: I like performance improvements using `Benchmark`

[GitHub] [spark] SparkQA commented on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

SparkQA commented on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#issuecomment-494208967 **[Test build #105588 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#issuecomment-494208636 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#issuecomment-494208642 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins commented on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

AmplabJenkins commented on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#issuecomment-494208636 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

AmplabJenkins commented on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#issuecomment-494208642 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#issuecomment-494208207 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#issuecomment-494208203 Merged build finished. Test PASSed.

  1   2   3   4   5   6   7   8   >