[jira] [Updated] (SPARK-33668) Fix flaky test "Verify logging configuration is picked from the provided SPARK_CONF_DIR/log4j.properties."

[ https://issues.apache.org/jira/browse/SPARK-33668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-33668: Description: The test is flaking, with multiple flaked instances - the reason for the

[jira] [Created] (SPARK-33668) Fix flaky test "Verify logging configuration is picked from the provided SPARK_CONF_DIR/log4j.properties."

Prashant Sharma created SPARK-33668: --- Summary: Fix flaky test "Verify logging configuration is picked from the provided SPARK_CONF_DIR/log4j.properties." Key: SPARK-33668 URL:

[jira] [Commented] (SPARK-33667) Respect case sensitivity in V1 SHOW PARTITIONS

[ https://issues.apache.org/jira/browse/SPARK-33667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244426#comment-17244426 ] Apache Spark commented on SPARK-33667: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-33667) Respect case sensitivity in V1 SHOW PARTITIONS

[ https://issues.apache.org/jira/browse/SPARK-33667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244425#comment-17244425 ] Apache Spark commented on SPARK-33667: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33667) Respect case sensitivity in V1 SHOW PARTITIONS

[ https://issues.apache.org/jira/browse/SPARK-33667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33667: Assignee: (was: Apache Spark) > Respect case sensitivity in V1 SHOW PARTITIONS >

[jira] [Assigned] (SPARK-33667) Respect case sensitivity in V1 SHOW PARTITIONS

[ https://issues.apache.org/jira/browse/SPARK-33667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33667: Assignee: Apache Spark > Respect case sensitivity in V1 SHOW PARTITIONS >

[jira] [Updated] (SPARK-33667) Respect case sensitivity in V1 SHOW PARTITIONS

[ https://issues.apache.org/jira/browse/SPARK-33667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33667: --- Description: SHOW PARTITIONS is case sensitive, and doesn't respect the SQL config

[jira] [Created] (SPARK-33667) Respect case sensitivity in V1 SHOW PARTITIONS

Maxim Gekk created SPARK-33667: -- Summary: Respect case sensitivity in V1 SHOW PARTITIONS Key: SPARK-33667 URL: https://issues.apache.org/jira/browse/SPARK-33667 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-33614) Fix the constant folding rule to skip it if the expression fails to execute

[ https://issues.apache.org/jira/browse/SPARK-33614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33614: Assignee: (was: Apache Spark) > Fix the constant folding rule to skip it if the

[jira] [Assigned] (SPARK-33614) Fix the constant folding rule to skip it if the expression fails to execute

[ https://issues.apache.org/jira/browse/SPARK-33614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33614: Assignee: Apache Spark > Fix the constant folding rule to skip it if the expression

[jira] [Commented] (SPARK-33614) Fix the constant folding rule to skip it if the expression fails to execute

[ https://issues.apache.org/jira/browse/SPARK-33614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244402#comment-17244402 ] Apache Spark commented on SPARK-33614: -- User 'luluorta' has created a pull request for this issue:

[jira] [Commented] (SPARK-33614) Fix the constant folding rule to skip it if the expression fails to execute

[ https://issues.apache.org/jira/browse/SPARK-33614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244401#comment-17244401 ] Apache Spark commented on SPARK-33614: -- User 'luluorta' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33666) Fix Flaky Test: HiveThriftHttpServerSuite

[ https://issues.apache.org/jira/browse/SPARK-33666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33666: Assignee: (was: Apache Spark) > Fix Flaky Test: HiveThriftHttpServerSuite >

[jira] [Commented] (SPARK-33666) Fix Flaky Test: HiveThriftHttpServerSuite

[ https://issues.apache.org/jira/browse/SPARK-33666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244387#comment-17244387 ] Apache Spark commented on SPARK-33666: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-33666) Fix Flaky Test: HiveThriftHttpServerSuite

[ https://issues.apache.org/jira/browse/SPARK-33666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33666: Assignee: Apache Spark > Fix Flaky Test: HiveThriftHttpServerSuite >

[jira] [Created] (SPARK-33666) Fix Flaky Test: HiveThriftHttpServerSuite

Dongjoon Hyun created SPARK-33666: - Summary: Fix Flaky Test: HiveThriftHttpServerSuite Key: SPARK-33666 URL: https://issues.apache.org/jira/browse/SPARK-33666 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-33651) allow CREATE EXTERNAL TABLE with LOCATION for data source tables

[ https://issues.apache.org/jira/browse/SPARK-33651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33651. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30595

[jira] [Comment Edited] (SPARK-33564) Prometheus metrics for Master and Worker isn't working

[ https://issues.apache.org/jira/browse/SPARK-33564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244356#comment-17244356 ] Dongjoon Hyun edited comment on SPARK-33564 at 12/4/20, 11:43 PM: -- Here

[jira] [Comment Edited] (SPARK-33564) Prometheus metrics for Master and Worker isn't working

[ https://issues.apache.org/jira/browse/SPARK-33564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244356#comment-17244356 ] Dongjoon Hyun edited comment on SPARK-33564 at 12/4/20, 11:42 PM: -- Here

[jira] [Commented] (SPARK-33564) Prometheus metrics for Master and Worker isn't working

[ https://issues.apache.org/jira/browse/SPARK-33564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244356#comment-17244356 ] Dongjoon Hyun commented on SPARK-33564: --- Here are the short answers. Yes, it does. > Just to

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244341#comment-17244341 ] Baohe Zhang commented on SPARK-26399: - Seems the work of this ticket is already done by

[jira] [Commented] (SPARK-33141) capture SQL configs when creating permanent views

[ https://issues.apache.org/jira/browse/SPARK-33141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244322#comment-17244322 ] Apache Spark commented on SPARK-33141: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-33141) capture SQL configs when creating permanent views

[ https://issues.apache.org/jira/browse/SPARK-33141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244320#comment-17244320 ] Apache Spark commented on SPARK-33141: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Resolved] (SPARK-33662) Setting version to 3.2.0-SNAPSHOT

[ https://issues.apache.org/jira/browse/SPARK-33662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33662. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30606

[jira] [Assigned] (SPARK-33662) Setting version to 3.2.0-SNAPSHOT

[ https://issues.apache.org/jira/browse/SPARK-33662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33662: - Assignee: Dongjoon Hyun > Setting version to 3.2.0-SNAPSHOT >

[jira] [Resolved] (SPARK-33660) Update Kafka Headers Documentation in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-33660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-33660. -- Fix Version/s: 3.2.0 3.0.2 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-33660) Update Kafka Headers Documentation in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-33660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-33660: Assignee: German Schiavon Matteo > Update Kafka Headers Documentation in Structured

[jira] [Commented] (SPARK-24837) Add kafka as spark metrics sink

[ https://issues.apache.org/jira/browse/SPARK-24837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244300#comment-17244300 ] Zikun commented on SPARK-24837: --- Why is this marked as resolved? The kafka sink for spark metrics is still

[jira] [Created] (SPARK-33665) Enable ShuffleBlockPusher to stop pushing blocks for a particular shuffle partition

Chandni Singh created SPARK-33665: - Summary: Enable ShuffleBlockPusher to stop pushing blocks for a particular shuffle partition Key: SPARK-33665 URL: https://issues.apache.org/jira/browse/SPARK-33665

[jira] [Commented] (SPARK-33664) Migrate ALTER TABLE ... RENAME TO to new resolution framework

[ https://issues.apache.org/jira/browse/SPARK-33664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244269#comment-17244269 ] Apache Spark commented on SPARK-33664: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33664) Migrate ALTER TABLE ... RENAME TO to new resolution framework

[ https://issues.apache.org/jira/browse/SPARK-33664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33664: Assignee: Apache Spark > Migrate ALTER TABLE ... RENAME TO to new resolution framework >

[jira] [Assigned] (SPARK-33664) Migrate ALTER TABLE ... RENAME TO to new resolution framework

[ https://issues.apache.org/jira/browse/SPARK-33664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33664: Assignee: (was: Apache Spark) > Migrate ALTER TABLE ... RENAME TO to new resolution

[jira] [Created] (SPARK-33664) Migrate ALTER TABLE ... RENAME TO to new resolution framework

Terry Kim created SPARK-33664: - Summary: Migrate ALTER TABLE ... RENAME TO to new resolution framework Key: SPARK-33664 URL: https://issues.apache.org/jira/browse/SPARK-33664 Project: Spark

[jira] [Assigned] (SPARK-33663) Fix misleading message for uncaching when createOrReplaceTempView is called

[ https://issues.apache.org/jira/browse/SPARK-33663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33663: Assignee: (was: Apache Spark) > Fix misleading message for uncaching when

[jira] [Commented] (SPARK-33663) Fix misleading message for uncaching when createOrReplaceTempView is called

[ https://issues.apache.org/jira/browse/SPARK-33663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244229#comment-17244229 ] Apache Spark commented on SPARK-33663: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33663) Fix misleading message for uncaching when createOrReplaceTempView is called

[ https://issues.apache.org/jira/browse/SPARK-33663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33663: Assignee: Apache Spark > Fix misleading message for uncaching when

[jira] [Created] (SPARK-33663) Fix misleading message for uncaching when createOrReplaceTempView is called

Terry Kim created SPARK-33663: - Summary: Fix misleading message for uncaching when createOrReplaceTempView is called Key: SPARK-33663 URL: https://issues.apache.org/jira/browse/SPARK-33663 Project: Spark

[jira] [Updated] (SPARK-24907) Migrate JDBC data source to DataSource API v2

[ https://issues.apache.org/jira/browse/SPARK-24907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-24907: --- Affects Version/s: (was: 2.3.0) (was: 3.0.0)

[jira] [Reopened] (SPARK-33564) Prometheus metrics for Master and Worker isn't working

[ https://issues.apache.org/jira/browse/SPARK-33564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Roberto de Oliveira Castro reopened SPARK-33564: -- Just to understand, the configuration needs to be set up

[jira] [Issue Comment Deleted] (SPARK-33564) Prometheus metrics for Master and Worker isn't working

[ https://issues.apache.org/jira/browse/SPARK-33564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Roberto de Oliveira Castro updated SPARK-33564: - Comment: was deleted (was: Just to understand, the

[jira] [Commented] (SPARK-33564) Prometheus metrics for Master and Worker isn't working

[ https://issues.apache.org/jira/browse/SPARK-33564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244182#comment-17244182 ] Paulo Roberto de Oliveira Castro commented on SPARK-33564: -- Just to understand,

[jira] [Commented] (SPARK-33662) Setting version to 3.2.0-SNAPSHOT

[ https://issues.apache.org/jira/browse/SPARK-33662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244114#comment-17244114 ] Apache Spark commented on SPARK-33662: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-33662) Setting version to 3.2.0-SNAPSHOT

[ https://issues.apache.org/jira/browse/SPARK-33662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33662: Assignee: Apache Spark > Setting version to 3.2.0-SNAPSHOT >

[jira] [Assigned] (SPARK-33662) Setting version to 3.2.0-SNAPSHOT

[ https://issues.apache.org/jira/browse/SPARK-33662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33662: Assignee: (was: Apache Spark) > Setting version to 3.2.0-SNAPSHOT >

[jira] [Created] (SPARK-33662) Setting version to 3.2.0-SNAPSHOT

Dongjoon Hyun created SPARK-33662: - Summary: Setting version to 3.2.0-SNAPSHOT Key: SPARK-33662 URL: https://issues.apache.org/jira/browse/SPARK-33662 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-33661) Unable to load RandomForestClassificationModel trained in Spark 2.x

Marcus Levine created SPARK-33661: - Summary: Unable to load RandomForestClassificationModel trained in Spark 2.x Key: SPARK-33661 URL: https://issues.apache.org/jira/browse/SPARK-33661 Project: Spark

[jira] [Updated] (SPARK-32595) Do truncate and append atomically in JDBCWriteBuilder

[ https://issues.apache.org/jira/browse/SPARK-32595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-32595: --- Affects Version/s: (was: 3.1.0) 3.2.0 > Do truncate and append

[jira] [Updated] (SPARK-32593) JDBC nested columns support

[ https://issues.apache.org/jira/browse/SPARK-32593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-32593: --- Affects Version/s: (was: 3.1.0) 3.2.0 > JDBC nested columns support >

[jira] [Updated] (SPARK-32833) JDBC V2 Datasource aggregate push down

[ https://issues.apache.org/jira/browse/SPARK-32833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-32833: --- Affects Version/s: (was: 3.1.0) 3.2.0 > JDBC V2 Datasource aggregate

[jira] [Commented] (SPARK-33660) Update Kafka Headers Documentation in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-33660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244043#comment-17244043 ] Apache Spark commented on SPARK-33660: -- User 'Gschiavon' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33660) Update Kafka Headers Documentation in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-33660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33660: Assignee: (was: Apache Spark) > Update Kafka Headers Documentation in Structured

[jira] [Assigned] (SPARK-33660) Update Kafka Headers Documentation in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-33660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33660: Assignee: Apache Spark > Update Kafka Headers Documentation in Structured Streaming >

[jira] [Updated] (SPARK-33660) Update Kafka Headers Documentation in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-33660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] German Schiavon Matteo updated SPARK-33660: --- Component/s: docs > Update Kafka Headers Documentation in Structured

[jira] [Created] (SPARK-33660) Update Kafka Headers Documentation in Structured Streaming

German Schiavon Matteo created SPARK-33660: -- Summary: Update Kafka Headers Documentation in Structured Streaming Key: SPARK-33660 URL: https://issues.apache.org/jira/browse/SPARK-33660

[jira] [Commented] (SPARK-32681) PySpark type hints support

[ https://issues.apache.org/jira/browse/SPARK-32681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244012#comment-17244012 ] Maciej Szymkiewicz commented on SPARK-32681: Thanks [~hyukjin.kwon]. As far as I am aware

[jira] [Updated] (SPARK-33565) python/run-tests.py calling python3.8

[ https://issues.apache.org/jira/browse/SPARK-33565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33565: - Fix Version/s: (was: 3.2.0) 3.1.0 > python/run-tests.py calling

[jira] [Updated] (SPARK-33466) Imputer support mode(most_frequent) strategy

[ https://issues.apache.org/jira/browse/SPARK-33466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33466: - Fix Version/s: (was: 3.2.0) 3.1.0 > Imputer support mode(most_frequent)

[jira] [Resolved] (SPARK-33615) Make spark.archives working in Kubernates

[ https://issues.apache.org/jira/browse/SPARK-33615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33615. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30581

[jira] [Assigned] (SPARK-33615) Make spark.archives working in Kubernates

[ https://issues.apache.org/jira/browse/SPARK-33615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33615: Assignee: Hyukjin Kwon > Make spark.archives working in Kubernates >

[jira] [Assigned] (SPARK-27237) Introduce State schema validation among query restart

[ https://issues.apache.org/jira/browse/SPARK-27237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-27237: Assignee: Jungtaek Lim > Introduce State schema validation among query restart >

[jira] [Resolved] (SPARK-27237) Introduce State schema validation among query restart

[ https://issues.apache.org/jira/browse/SPARK-27237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27237. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 24173

[jira] [Comment Edited] (SPARK-33632) to_date doesn't behave as documented

[ https://issues.apache.org/jira/browse/SPARK-33632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243858#comment-17243858 ] Frank Oosterhuis edited comment on SPARK-33632 at 12/4/20, 9:11 AM:

[jira] [Commented] (SPARK-33632) to_date doesn't behave as documented

[ https://issues.apache.org/jira/browse/SPARK-33632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243858#comment-17243858 ] Frank Oosterhuis commented on SPARK-33632: -- [~qwe1398775315] is right, and the spec is actually

[jira] [Updated] (SPARK-33632) to_date doesn't behave as documented

[ https://issues.apache.org/jira/browse/SPARK-33632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Oosterhuis updated SPARK-33632: - Attachment: screenshot-1.png > to_date doesn't behave as documented >

[jira] [Commented] (SPARK-33571) Handling of hybrid to proleptic calendar when reading and writing Parquet data not working correctly

[ https://issues.apache.org/jira/browse/SPARK-33571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243856#comment-17243856 ] Apache Spark commented on SPARK-33571: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Resolved] (SPARK-32681) PySpark type hints support

[ https://issues.apache.org/jira/browse/SPARK-32681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32681. -- Resolution: Done I will mark this as done for now but [~zero323] please feel free to add a

[jira] [Updated] (SPARK-7768) Make user-defined type (UDT) API public

[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-7768: Target Version/s: 3.2.0 (was: 3.1.0) > Make user-defined type (UDT) API public >

[jira] [Commented] (SPARK-7768) Make user-defined type (UDT) API public

[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243847#comment-17243847 ] Hyukjin Kwon commented on SPARK-7768: - I am retargetting to 3.2.0. > Make user-defined type (UDT)

[jira] [Updated] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27495: - Target Version/s: 3.2.0 (was: 3.1.0) > SPIP: Support Stage level resource configuration and

[jira] [Commented] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243846#comment-17243846 ] Hyukjin Kwon commented on SPARK-27495: -- [~tgraves], I switch the target version up to 3.2.0 because

[jira] [Updated] (SPARK-24942) Improve cluster resource management with jobs containing barrier stage

[ https://issues.apache.org/jira/browse/SPARK-24942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24942: - Target Version/s: 3.2.0 (was: 3.1.0) > Improve cluster resource management with jobs

[jira] [Commented] (SPARK-24941) Add RDDBarrier.coalesce() function

[ https://issues.apache.org/jira/browse/SPARK-24941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243845#comment-17243845 ] Hyukjin Kwon commented on SPARK-24941: -- I switched it to 3.2.0 > Add RDDBarrier.coalesce()

[jira] [Updated] (SPARK-24941) Add RDDBarrier.coalesce() function

[ https://issues.apache.org/jira/browse/SPARK-24941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24941: - Target Version/s: 3.2.0 (was: 3.1.0) > Add RDDBarrier.coalesce() function >

[jira] [Commented] (SPARK-24942) Improve cluster resource management with jobs containing barrier stage

[ https://issues.apache.org/jira/browse/SPARK-24942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243844#comment-17243844 ] Hyukjin Kwon commented on SPARK-24942: -- Let me retarget it to 3.2.0. Branch will be cut out soon.

[jira] [Commented] (SPARK-25383) Image data source supports sample pushdown

[ https://issues.apache.org/jira/browse/SPARK-25383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243843#comment-17243843 ] Hyukjin Kwon commented on SPARK-25383: -- Branch 3.2 will be cut out soon. Let me retarget it to

[jira] [Commented] (SPARK-27780) Shuffle server & client should be versioned to enable smoother upgrade

[ https://issues.apache.org/jira/browse/SPARK-27780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243841#comment-17243841 ] Hyukjin Kwon commented on SPARK-27780: -- I will switch to 3.2.0. Branch will be cut out soon. >

[jira] [Updated] (SPARK-25383) Image data source supports sample pushdown

[ https://issues.apache.org/jira/browse/SPARK-25383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25383: - Target Version/s: 3.2.0 (was: 3.1.0) > Image data source supports sample pushdown >

[jira] [Updated] (SPARK-27780) Shuffle server & client should be versioned to enable smoother upgrade

[ https://issues.apache.org/jira/browse/SPARK-27780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27780: - Target Version/s: 3.2.0 (was: 3.1.0) > Shuffle server & client should be versioned to enable

[jira] [Updated] (SPARK-28629) Capture the missing rules in HiveSessionStateBuilder

[ https://issues.apache.org/jira/browse/SPARK-28629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28629: - Target Version/s: 3.2.0 (was: 3.1.0) > Capture the missing rules in HiveSessionStateBuilder >

[jira] [Commented] (SPARK-28629) Capture the missing rules in HiveSessionStateBuilder

[ https://issues.apache.org/jira/browse/SPARK-28629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243840#comment-17243840 ] Hyukjin Kwon commented on SPARK-28629: -- [~smilegator] I am going to retarget it to 3.2.0 but please

[jira] [Updated] (SPARK-31555) Improve cache block migration

[ https://issues.apache.org/jira/browse/SPARK-31555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31555: - Target Version/s: 3.2.0 (was: 3.1.0) > Improve cache block migration >

[jira] [Commented] (SPARK-31555) Improve cache block migration

[ https://issues.apache.org/jira/browse/SPARK-31555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243839#comment-17243839 ] Hyukjin Kwon commented on SPARK-31555: -- [~holden] and [~dongjoon], I switched it to 3.2.0 but feel

[jira] [Updated] (SPARK-33461) Propagating SPARK_CONF_DIR in K8s and tests

[ https://issues.apache.org/jira/browse/SPARK-33461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33461: - Target Version/s: 3.2.0 (was: 3.1.0) > Propagating SPARK_CONF_DIR in K8s and tests >

[jira] [Updated] (SPARK-14922) Alter Table Drop Partition Using Predicate-based Partition Spec

[ https://issues.apache.org/jira/browse/SPARK-14922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-14922: - Target Version/s: 3.2.0 (was: 3.1.0) > Alter Table Drop Partition Using Predicate-based

[jira] [Updated] (SPARK-25075) Build and test Spark against Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25075: - Target Version/s: 3.2.0 (was: 3.1.0) > Build and test Spark against Scala 2.13 >

[jira] [Updated] (SPARK-31976) use MemoryUsage to control the size of block

[ https://issues.apache.org/jira/browse/SPARK-31976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31976: - Target Version/s: 3.2.0 (was: 3.1.0) > use MemoryUsage to control the size of block >

[jira] [Updated] (SPARK-25186) Stabilize Data Source V2 API

[ https://issues.apache.org/jira/browse/SPARK-25186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25186: - Target Version/s: 3.2.0 (was: 3.1.0) > Stabilize Data Source V2 API >

[jira] [Updated] (SPARK-31981) Keep TimestampType when taking an average of a Timestamp

[ https://issues.apache.org/jira/browse/SPARK-31981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31981: - Target Version/s: (was: 3.1.0) > Keep TimestampType when taking an average of a Timestamp >

[jira] [Updated] (SPARK-30334) Add metadata around semi-structured columns to Spark

[ https://issues.apache.org/jira/browse/SPARK-30334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30334: - Target Version/s: 3.2.0 (was: 3.1.0) > Add metadata around semi-structured columns to Spark >

[jira] [Commented] (SPARK-30978) Remove multiple workers on the same host support from Standalone backend

[ https://issues.apache.org/jira/browse/SPARK-30978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243831#comment-17243831 ] Hyukjin Kwon commented on SPARK-30978: -- I changed this to 3.2.0, [~jiangxb1987] > Remove multiple

[jira] [Updated] (SPARK-30324) Simplify API for JSON access in DataFrames/SQL

[ https://issues.apache.org/jira/browse/SPARK-30324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30324: - Target Version/s: 3.2.0 (was: 3.1.0) > Simplify API for JSON access in DataFrames/SQL >

[jira] [Updated] (SPARK-30978) Remove multiple workers on the same host support from Standalone backend

[ https://issues.apache.org/jira/browse/SPARK-30978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30978: - Target Version/s: 3.2.0 (was: 3.1.0) > Remove multiple workers on the same host support from

[jira] [Commented] (SPARK-25752) Add trait to easily whitelist logical operators that produce named output from CleanupAliases

[ https://issues.apache.org/jira/browse/SPARK-25752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243830#comment-17243830 ] Hyukjin Kwon commented on SPARK-25752: -- I changed this to 3.2.0. > Add trait to easily whitelist

[jira] [Updated] (SPARK-25752) Add trait to easily whitelist logical operators that produce named output from CleanupAliases

[ https://issues.apache.org/jira/browse/SPARK-25752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25752: - Target Version/s: 3.2.0 (was: 3.1.0) > Add trait to easily whitelist logical operators that

[jira] [Updated] (SPARK-32846) Support createDataFrame from an RDD of pd.DataFrames

[ https://issues.apache.org/jira/browse/SPARK-32846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32846: - Target Version/s: (was: 3.1.0) > Support createDataFrame from an RDD of pd.DataFrames >

[jira] [Resolved] (SPARK-33640) Extend connection timeout to DB server for DB2IntegrationSuite and its variants

[ https://issues.apache.org/jira/browse/SPARK-33640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33640. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30583