[jira] [Created] (SPARK-31351) Migration Guide Auditing for Spark 3.0 Release

Xiao Li created SPARK-31351: --- Summary: Migration Guide Auditing for Spark 3.0 Release Key: SPARK-31351 URL: https://issues.apache.org/jira/browse/SPARK-31351 Project: Spark Issue Type:

[jira] [Updated] (SPARK-30888) Add version information to the configuration of Network

[ https://issues.apache.org/jira/browse/SPARK-30888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30888: - Fix Version/s: 3.0.0 > Add version information to the configuration of Network >

[jira] [Updated] (SPARK-30891) Add version information to the configuration of History

[ https://issues.apache.org/jira/browse/SPARK-30891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30891: - Fix Version/s: 3.0.0 > Add version information to the configuration of History >

[jira] [Updated] (SPARK-30887) Add version information to the configuration of Deploy

[ https://issues.apache.org/jira/browse/SPARK-30887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30887: - Fix Version/s: 3.0.0 > Add version information to the configuration of Deploy >

[jira] [Updated] (SPARK-30910) Add version information to the configuration of R

[ https://issues.apache.org/jira/browse/SPARK-30910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30910: - Fix Version/s: 3.0.0 > Add version information to the configuration of R >

[jira] [Updated] (SPARK-30912) Add version information to the configuration of Streaming.scala

[ https://issues.apache.org/jira/browse/SPARK-30912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30912: - Fix Version/s: 3.0.0 > Add version information to the configuration of Streaming.scala >

[jira] [Updated] (SPARK-30909) Add version information to the configuration of Python

[ https://issues.apache.org/jira/browse/SPARK-30909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30909: - Fix Version/s: 3.0.0 > Add version information to the configuration of Python >

[jira] [Updated] (SPARK-30908) Add version information to the configuration of Kryo

[ https://issues.apache.org/jira/browse/SPARK-30908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30908: - Fix Version/s: 3.0.0 > Add version information to the configuration of Kryo >

[jira] [Updated] (SPARK-30889) Add version information to the configuration of Worker

[ https://issues.apache.org/jira/browse/SPARK-30889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30889: - Fix Version/s: 3.0.0 > Add version information to the configuration of Worker >

[jira] [Updated] (SPARK-30914) Add version information to the configuration of UI

[ https://issues.apache.org/jira/browse/SPARK-30914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30914: - Fix Version/s: 3.0.0 > Add version information to the configuration of UI >

[jira] [Updated] (SPARK-31002) Add version information to the configuration of Core

[ https://issues.apache.org/jira/browse/SPARK-31002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31002: - Fix Version/s: 3.0.0 > Add version information to the configuration of Core >

[jira] [Updated] (SPARK-30841) Add version information to the configuration of SQL

[ https://issues.apache.org/jira/browse/SPARK-30841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30841: - Fix Version/s: 3.0.0 > Add version information to the configuration of SQL >

[jira] [Updated] (SPARK-30839) Add version information for Spark configuration

[ https://issues.apache.org/jira/browse/SPARK-30839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30839: - Target Version/s: 3.0.0 > Add version information for Spark configuration >

[jira] [Commented] (SPARK-30839) Add version information for Spark configuration

[ https://issues.apache.org/jira/browse/SPARK-30839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17075692#comment-17075692 ] Hyukjin Kwon commented on SPARK-30839: -- Target version is now 3.0. We're tentatively trying to

[jira] [Updated] (SPARK-31346) Add new configuration to make sure temporary directory cleaned

[ https://issues.apache.org/jira/browse/SPARK-31346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackey Lee updated SPARK-31346: --- Description: In InsertIntoHiveTable and InsertIntoHiveDirCommand, we use deleteExternalTmpPath to

[jira] [Updated] (SPARK-31350) Coalesce bucketed tables for join if applicable

[ https://issues.apache.org/jira/browse/SPARK-31350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim updated SPARK-31350: -- Description: The following example of joining two bucketed tables introduces a full shuffle:

[jira] [Created] (SPARK-31350) Coalesce bucketed tables for join if applicable

Terry Kim created SPARK-31350: - Summary: Coalesce bucketed tables for join if applicable Key: SPARK-31350 URL: https://issues.apache.org/jira/browse/SPARK-31350 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-31347) Unable to run Spark Job on Federated Yarn Cluster, AMRMToken invalid

[ https://issues.apache.org/jira/browse/SPARK-31347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Babble Shack updated SPARK-31347: - Attachment: spark.debug.log > Unable to run Spark Job on Federated Yarn Cluster, AMRMToken

[jira] [Comment Edited] (SPARK-31345) Spark fails to write hive parquet table with empty array

[ https://issues.apache.org/jira/browse/SPARK-31345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17075314#comment-17075314 ] Dongjoon Hyun edited comment on SPARK-31345 at 4/4/20, 9:39 PM: For the

[jira] [Commented] (SPARK-31345) Spark fails to write hive parquet table with empty array

[ https://issues.apache.org/jira/browse/SPARK-31345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17075314#comment-17075314 ] Dongjoon Hyun commented on SPARK-31345: --- For the record, the above example doesn't work in all 2.x

[jira] [Updated] (SPARK-31345) Spark fails to write hive parquet table with empty array

[ https://issues.apache.org/jira/browse/SPARK-31345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31345: -- Affects Version/s: 2.0.2 > Spark fails to write hive parquet table with empty array >

[jira] [Updated] (SPARK-31345) Spark fails to write hive parquet table with empty array

[ https://issues.apache.org/jira/browse/SPARK-31345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31345: -- Affects Version/s: 2.1.3 > Spark fails to write hive parquet table with empty array >

[jira] [Updated] (SPARK-31345) Spark fails to write hive parquet table with empty array

[ https://issues.apache.org/jira/browse/SPARK-31345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31345: -- Affects Version/s: 2.2.3 > Spark fails to write hive parquet table with empty array >

[jira] [Comment Edited] (SPARK-31345) Spark fails to write hive parquet table with empty array

[ https://issues.apache.org/jira/browse/SPARK-31345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17075305#comment-17075305 ] Dongjoon Hyun edited comment on SPARK-31345 at 4/4/20, 9:29 PM: Thank

[jira] [Comment Edited] (SPARK-31345) Spark fails to write hive parquet table with empty array

[ https://issues.apache.org/jira/browse/SPARK-31345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17075305#comment-17075305 ] Dongjoon Hyun edited comment on SPARK-31345 at 4/4/20, 9:27 PM: Thank

[jira] [Resolved] (SPARK-31345) Spark fails to write hive parquet table with empty array

[ https://issues.apache.org/jira/browse/SPARK-31345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31345. --- Resolution: Duplicate Thank you for reporting, [~naitree]. This is fixed via SPARK-29462 .

[jira] [Updated] (SPARK-31345) Spark fails to write hive parquet table with empty array

[ https://issues.apache.org/jira/browse/SPARK-31345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31345: -- Affects Version/s: 2.3.4 > Spark fails to write hive parquet table with empty array >

[jira] [Updated] (SPARK-31345) Spark fails to write hive parquet table with empty array

[ https://issues.apache.org/jira/browse/SPARK-31345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31345: -- Description: When writing an existing Hive Parquet table using Spark SQL, I encountered an

[jira] [Updated] (SPARK-31345) Spark fails to write hive parquet table with empty array

[ https://issues.apache.org/jira/browse/SPARK-31345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31345: -- Description: When writing an existing Hive Parquet table using Spark SQL, I encountered an

[jira] [Updated] (SPARK-31346) Add new configuration to verify temporary directory cleaned

[ https://issues.apache.org/jira/browse/SPARK-31346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31346: -- Affects Version/s: (was: 3.0.1) > Add new configuration to verify temporary directory

[jira] [Updated] (SPARK-31349) Document builtin aggregate function

[ https://issues.apache.org/jira/browse/SPARK-31349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kevin yu updated SPARK-31349: - Parent: SPARK-28588 Issue Type: Sub-task (was: Task) > Document builtin aggregate function >

[jira] [Created] (SPARK-31349) Document builtin aggregate function

kevin yu created SPARK-31349: Summary: Document builtin aggregate function Key: SPARK-31349 URL: https://issues.apache.org/jira/browse/SPARK-31349 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-31348) Document Join in SQL Reference

Huaxin Gao created SPARK-31348: -- Summary: Document Join in SQL Reference Key: SPARK-31348 URL: https://issues.apache.org/jira/browse/SPARK-31348 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-31347) Unable to run Spark Job on Federated Yarn Cluster, AMRMToken invalid

[ https://issues.apache.org/jira/browse/SPARK-31347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Babble Shack updated SPARK-31347: - Description: Running Spark on Yarn 3.2.1 in federated cluster ApplicationMaster fails to

[jira] [Updated] (SPARK-31347) Unable to run Spark Job on Federated Yarn Cluster, AMRMToken invalid

[ https://issues.apache.org/jira/browse/SPARK-31347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Babble Shack updated SPARK-31347: - Attachment: router-yarn-site.xml > Unable to run Spark Job on Federated Yarn Cluster, AMRMToken

[jira] [Updated] (SPARK-31347) Unable to run Spark Job on Federated Yarn Cluster, AMRMToken invalid

[ https://issues.apache.org/jira/browse/SPARK-31347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Babble Shack updated SPARK-31347: - Attachment: spark.out mapred.out mapred.log

[jira] [Created] (SPARK-31347) Unable to run Spark Job on Federated Yarn Cluster, AMRMToken invalid

Babble Shack created SPARK-31347: Summary: Unable to run Spark Job on Federated Yarn Cluster, AMRMToken invalid Key: SPARK-31347 URL: https://issues.apache.org/jira/browse/SPARK-31347 Project: Spark

[jira] [Created] (SPARK-31346) Add new configuration to verify temporary directory cleaned

Jackey Lee created SPARK-31346: -- Summary: Add new configuration to verify temporary directory cleaned Key: SPARK-31346 URL: https://issues.apache.org/jira/browse/SPARK-31346 Project: Spark

[jira] [Updated] (SPARK-31345) Spark fails to write hive parquet table with empty array

[ https://issues.apache.org/jira/browse/SPARK-31345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naitree Zhu updated SPARK-31345: Component/s: (was: Spark Core) SQL > Spark fails to write hive parquet table

[jira] [Updated] (SPARK-31345) Spark fails to write hive parquet table with empty array

[ https://issues.apache.org/jira/browse/SPARK-31345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naitree Zhu updated SPARK-31345: Description: When writing an existing Hive Parquet table using Spark SQL, I encountered an error

[jira] [Created] (SPARK-31345) Spark fails to write hive parquet table with empty array

Naitree Zhu created SPARK-31345: --- Summary: Spark fails to write hive parquet table with empty array Key: SPARK-31345 URL: https://issues.apache.org/jira/browse/SPARK-31345 Project: Spark Issue