[jira] [Created] (SPARK-32435) Remove heapq3 port from Python 3

Hyukjin Kwon created SPARK-32435: Summary: Remove heapq3 port from Python 3 Key: SPARK-32435 URL: https://issues.apache.org/jira/browse/SPARK-32435 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-29059) [SPIP] Support for Hive Materialized Views in Spark SQL.

[ https://issues.apache.org/jira/browse/SPARK-29059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164751#comment-17164751 ] Amogh Margoor commented on SPARK-29059: --- [~dongjoon] [~maropu] Do you guys think this would be

[jira] [Commented] (SPARK-32425) Spark sequence() fails if start and end of range are identical timestamps

[ https://issues.apache.org/jira/browse/SPARK-32425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164748#comment-17164748 ] L. C. Hsieh commented on SPARK-32425: - I ran a test with current master branch. Didn't see that

[jira] [Commented] (SPARK-32376) Make unionByName null-filling behavior work with struct columns

[ https://issues.apache.org/jira/browse/SPARK-32376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164747#comment-17164747 ] L. C. Hsieh commented on SPARK-32376: - [~mukulmurthy] Is it easy to port your code? If not, I think

[jira] [Resolved] (SPARK-32419) Leverage Conda environment at pip packaging test in GitHub Actions

[ https://issues.apache.org/jira/browse/SPARK-32419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32419. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29212

[jira] [Assigned] (SPARK-32419) Leverage Conda environment at pip packaging test in GitHub Actions

[ https://issues.apache.org/jira/browse/SPARK-32419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32419: Assignee: Hyukjin Kwon > Leverage Conda environment at pip packaging test in GitHub

[jira] [Commented] (SPARK-31847) DAGSchedulerSuite: Rewrite the test framework to cover most of the existing major features of the Spark Scheduler, mock the necessary part wisely, and make the test fr

[ https://issues.apache.org/jira/browse/SPARK-31847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164727#comment-17164727 ] Apache Spark commented on SPARK-31847: -- User 'beliefer' has created a pull request for this issue:

[jira] [Commented] (SPARK-31847) DAGSchedulerSuite: Rewrite the test framework to cover most of the existing major features of the Spark Scheduler, mock the necessary part wisely, and make the test fr

[ https://issues.apache.org/jira/browse/SPARK-31847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164726#comment-17164726 ] Apache Spark commented on SPARK-31847: -- User 'beliefer' has created a pull request for this issue:

[jira] [Updated] (SPARK-32434) Support Scala 2.13 in AbstractCommandBuilder and load-spark-env scripts

[ https://issues.apache.org/jira/browse/SPARK-32434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32434: -- Parent: SPARK-25075 Issue Type: Sub-task (was: Improvement) > Support Scala 2.13 in

[jira] [Commented] (SPARK-32434) Support Scala 2.13 in AbstractCommandBuilder and load-spark-env scripts

[ https://issues.apache.org/jira/browse/SPARK-32434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164704#comment-17164704 ] Apache Spark commented on SPARK-32434: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-32434) Support Scala 2.13 in AbstractCommandBuilder and load-spark-env scripts

[ https://issues.apache.org/jira/browse/SPARK-32434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164703#comment-17164703 ] Apache Spark commented on SPARK-32434: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-32434) Support Scala 2.13 in AbstractCommandBuilder and load-spark-env scripts

[ https://issues.apache.org/jira/browse/SPARK-32434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32434: Assignee: (was: Apache Spark) > Support Scala 2.13 in AbstractCommandBuilder and

[jira] [Assigned] (SPARK-32434) Support Scala 2.13 in AbstractCommandBuilder and load-spark-env scripts

[ https://issues.apache.org/jira/browse/SPARK-32434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32434: Assignee: Apache Spark > Support Scala 2.13 in AbstractCommandBuilder and load-spark-env

[jira] [Created] (SPARK-32434) Support Scala 2.13 in AbstractCommandBuilder and load-spark-env scripts

Dongjoon Hyun created SPARK-32434: - Summary: Support Scala 2.13 in AbstractCommandBuilder and load-spark-env scripts Key: SPARK-32434 URL: https://issues.apache.org/jira/browse/SPARK-32434 Project:

[jira] [Created] (SPARK-32433) Spark Web UI shows Nan undefined in Shuffle Read Size / Records

Arvind Krishnan created SPARK-32433: --- Summary: Spark Web UI shows Nan undefined in Shuffle Read Size / Records Key: SPARK-32433 URL: https://issues.apache.org/jira/browse/SPARK-32433 Project:

[jira] [Updated] (SPARK-32429) Standalone Mode allow setting CUDA_VISIBLE_DEVICES on executor launch

[ https://issues.apache.org/jira/browse/SPARK-32429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-32429: -- Description: It would be nice if standalone mode could allow users to set

[jira] [Assigned] (SPARK-32387) Extract UninterruptibleThread runner logic from KafkaOffsetReader

[ https://issues.apache.org/jira/browse/SPARK-32387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32387: - Assignee: Gabor Somogyi > Extract UninterruptibleThread runner logic from

[jira] [Resolved] (SPARK-32387) Extract UninterruptibleThread runner logic from KafkaOffsetReader

[ https://issues.apache.org/jira/browse/SPARK-32387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32387. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29187

[jira] [Resolved] (SPARK-32287) Flaky Test: ExecutorAllocationManagerSuite.add executors default profile

[ https://issues.apache.org/jira/browse/SPARK-32287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32287. --- Fix Version/s: 3.1.0 3.0.1 Assignee: Thomas Graves

[jira] [Assigned] (SPARK-32430) Allow plugins to inject rules into AQE query stage preparation

[ https://issues.apache.org/jira/browse/SPARK-32430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32430: - Assignee: Andy Grove (was: Thomas Graves) > Allow plugins to inject rules into AQE

[jira] [Assigned] (SPARK-32430) Allow plugins to inject rules into AQE query stage preparation

[ https://issues.apache.org/jira/browse/SPARK-32430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32430: - Assignee: Thomas Graves > Allow plugins to inject rules into AQE query stage

[jira] [Resolved] (SPARK-32430) Allow plugins to inject rules into AQE query stage preparation

[ https://issues.apache.org/jira/browse/SPARK-32430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32430. --- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by

[jira] [Created] (SPARK-32432) Add support for reading ORC/Parquet files with SymlinkTextInputFormat

Noritaka Sekiyama created SPARK-32432: - Summary: Add support for reading ORC/Parquet files with SymlinkTextInputFormat Key: SPARK-32432 URL: https://issues.apache.org/jira/browse/SPARK-32432

[jira] [Commented] (SPARK-32287) Flaky Test: ExecutorAllocationManagerSuite.add executors default profile

[ https://issues.apache.org/jira/browse/SPARK-32287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164516#comment-17164516 ] Apache Spark commented on SPARK-32287: -- User 'tgravescs' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32287) Flaky Test: ExecutorAllocationManagerSuite.add executors default profile

[ https://issues.apache.org/jira/browse/SPARK-32287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32287: Assignee: (was: Apache Spark) > Flaky Test: ExecutorAllocationManagerSuite.add

[jira] [Commented] (SPARK-32287) Flaky Test: ExecutorAllocationManagerSuite.add executors default profile

[ https://issues.apache.org/jira/browse/SPARK-32287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164518#comment-17164518 ] Apache Spark commented on SPARK-32287: -- User 'tgravescs' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32287) Flaky Test: ExecutorAllocationManagerSuite.add executors default profile

[ https://issues.apache.org/jira/browse/SPARK-32287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32287: Assignee: Apache Spark > Flaky Test: ExecutorAllocationManagerSuite.add executors

[jira] [Commented] (SPARK-32287) Flaky Test: ExecutorAllocationManagerSuite.add executors default profile

[ https://issues.apache.org/jira/browse/SPARK-32287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164512#comment-17164512 ] Thomas Graves commented on SPARK-32287: --- I haven't been able to reproduce but the best I can tell

[jira] [Resolved] (SPARK-32406) Support reset one particular configuration

[ https://issues.apache.org/jira/browse/SPARK-32406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32406. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29202

[jira] [Assigned] (SPARK-32406) Support reset one particular configuration

[ https://issues.apache.org/jira/browse/SPARK-32406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32406: - Assignee: Kent Yao > Support reset one particular configuration >

[jira] [Assigned] (SPARK-32408) Enable crossPaths back to prevent side effects

[ https://issues.apache.org/jira/browse/SPARK-32408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32408: - Assignee: Hyukjin Kwon > Enable crossPaths back to prevent side effects >

[jira] [Resolved] (SPARK-32408) Enable crossPaths back to prevent side effects

[ https://issues.apache.org/jira/browse/SPARK-32408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32408. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29218

[jira] [Resolved] (SPARK-27683) Remove usage of TraversableOnce

[ https://issues.apache.org/jira/browse/SPARK-27683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-27683. -- Resolution: Not A Problem I believe this is no longer an issue. > Remove usage of

[jira] [Commented] (SPARK-32431) The .schema() API behaves incorrectly for nested schemas that have column duplicates in case-insensitive mode

[ https://issues.apache.org/jira/browse/SPARK-32431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164499#comment-17164499 ] Maxim Gekk commented on SPARK-32431: I am working on this. > The .schema() API behaves incorrectly

[jira] [Updated] (SPARK-32431) The .schema() API behaves incorrectly for nested schemas that have column duplicates in case-insensitive mode

[ https://issues.apache.org/jira/browse/SPARK-32431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michał Świtakowski updated SPARK-32431: --- Description: The code below throws org.apache.spark.sql.AnalysisException: Found

[jira] [Created] (SPARK-32431) The .schema() API behaves incorrectly for nested schemas that have column duplicates in case-insensitive mode

Michał Świtakowski created SPARK-32431: -- Summary: The .schema() API behaves incorrectly for nested schemas that have column duplicates in case-insensitive mode Key: SPARK-32431 URL:

[jira] [Commented] (SPARK-32430) Allow plugins to inject rules into AQE query stage preparation

[ https://issues.apache.org/jira/browse/SPARK-32430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164483#comment-17164483 ] Apache Spark commented on SPARK-32430: -- User 'andygrove' has created a pull request for this issue:

[jira] [Commented] (SPARK-32430) Allow plugins to inject rules into AQE query stage preparation

[ https://issues.apache.org/jira/browse/SPARK-32430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164481#comment-17164481 ] Apache Spark commented on SPARK-32430: -- User 'andygrove' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32430) Allow plugins to inject rules into AQE query stage preparation

[ https://issues.apache.org/jira/browse/SPARK-32430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32430: Assignee: Apache Spark > Allow plugins to inject rules into AQE query stage preparation

[jira] [Assigned] (SPARK-32430) Allow plugins to inject rules into AQE query stage preparation

[ https://issues.apache.org/jira/browse/SPARK-32430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32430: Assignee: (was: Apache Spark) > Allow plugins to inject rules into AQE query stage

[jira] [Created] (SPARK-32430) Allow plugins to inject rules into AQE query stage preparation

Thomas Graves created SPARK-32430: - Summary: Allow plugins to inject rules into AQE query stage preparation Key: SPARK-32430 URL: https://issues.apache.org/jira/browse/SPARK-32430 Project: Spark

[jira] [Updated] (SPARK-32363) Flaky pip installation test in Jenkins

[ https://issues.apache.org/jira/browse/SPARK-32363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32363: -- Affects Version/s: 3.0.0 > Flaky pip installation test in Jenkins >

[jira] [Updated] (SPARK-32363) Flaky pip installation test in Jenkins

[ https://issues.apache.org/jira/browse/SPARK-32363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32363: -- Fix Version/s: 3.0.1 > Flaky pip installation test in Jenkins >

[jira] [Resolved] (SPARK-32375) Implement TableCatalog for JDBC

[ https://issues.apache.org/jira/browse/SPARK-32375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32375. - Fix Version/s: 3.1.0 Assignee: Maxim Gekk Resolution: Fixed > Implement

[jira] [Commented] (SPARK-32429) Standalone Mode allow setting CUDA_VISIBLE_DEVICES on executor launch

[ https://issues.apache.org/jira/browse/SPARK-32429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164440#comment-17164440 ] Thomas Graves commented on SPARK-32429: --- [~jiangxb1987] [~Ngone51] thoughts on the above proposal?

[jira] [Created] (SPARK-32429) Standalone Mode allow setting CUDA_VISIBLE_DEVICES on executor launch

Thomas Graves created SPARK-32429: - Summary: Standalone Mode allow setting CUDA_VISIBLE_DEVICES on executor launch Key: SPARK-32429 URL: https://issues.apache.org/jira/browse/SPARK-32429 Project:

[jira] [Assigned] (SPARK-32428) [EXAMPLES] Make BinaryClassificationMetricsExample consistently print the metrics on driver's stdout

[ https://issues.apache.org/jira/browse/SPARK-32428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32428: Assignee: Apache Spark > [EXAMPLES] Make BinaryClassificationMetricsExample consistently

[jira] [Assigned] (SPARK-32428) [EXAMPLES] Make BinaryClassificationMetricsExample consistently print the metrics on driver's stdout

[ https://issues.apache.org/jira/browse/SPARK-32428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32428: Assignee: (was: Apache Spark) > [EXAMPLES] Make BinaryClassificationMetricsExample

[jira] [Commented] (SPARK-32428) [EXAMPLES] Make BinaryClassificationMetricsExample consistently print the metrics on driver's stdout

[ https://issues.apache.org/jira/browse/SPARK-32428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164404#comment-17164404 ] Apache Spark commented on SPARK-32428: -- User 'titsuki' has created a pull request for this issue:

[jira] [Created] (SPARK-32428) [EXAMPLES] Make BinaryClassificationMetricsExample consistently print the metrics on driver's stdout

Itsuki Toyota created SPARK-32428: - Summary: [EXAMPLES] Make BinaryClassificationMetricsExample consistently print the metrics on driver's stdout Key: SPARK-32428 URL:

[jira] [Created] (SPARK-32427) Omit USING in CREATE TABLE via JDBC Table Catalog

Maxim Gekk created SPARK-32427: -- Summary: Omit USING in CREATE TABLE via JDBC Table Catalog Key: SPARK-32427 URL: https://issues.apache.org/jira/browse/SPARK-32427 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32426) ui shows sql after variable substitution

[ https://issues.apache.org/jira/browse/SPARK-32426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164300#comment-17164300 ] Apache Spark commented on SPARK-32426: -- User 'cxzl25' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32426) ui shows sql after variable substitution

[ https://issues.apache.org/jira/browse/SPARK-32426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32426: Assignee: (was: Apache Spark) > ui shows sql after variable substitution >

[jira] [Commented] (SPARK-32426) ui shows sql after variable substitution

[ https://issues.apache.org/jira/browse/SPARK-32426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164301#comment-17164301 ] Apache Spark commented on SPARK-32426: -- User 'cxzl25' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32426) ui shows sql after variable substitution

[ https://issues.apache.org/jira/browse/SPARK-32426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32426: Assignee: Apache Spark > ui shows sql after variable substitution >

[jira] [Created] (SPARK-32426) ui shows sql after variable substitution

dzcxzl created SPARK-32426: -- Summary: ui shows sql after variable substitution Key: SPARK-32426 URL: https://issues.apache.org/jira/browse/SPARK-32426 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-32425) Spark sequence() fails if start and end of range are identical timestamps

Lauri Koobas created SPARK-32425: Summary: Spark sequence() fails if start and end of range are identical timestamps Key: SPARK-32425 URL: https://issues.apache.org/jira/browse/SPARK-32425 Project:

[jira] [Updated] (SPARK-32425) Spark sequence() fails if start and end of range are identical timestamps

[ https://issues.apache.org/jira/browse/SPARK-32425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lauri Koobas updated SPARK-32425: - Description: The following Spark SQL query throws an exception {code:java} select 

[jira] [Assigned] (SPARK-32424) Fix silent data change for timestamp parsing if overflow happens

[ https://issues.apache.org/jira/browse/SPARK-32424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32424: Assignee: (was: Apache Spark) > Fix silent data change for timestamp parsing if

[jira] [Assigned] (SPARK-32424) Fix silent data change for timestamp parsing if overflow happens

[ https://issues.apache.org/jira/browse/SPARK-32424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32424: Assignee: Apache Spark > Fix silent data change for timestamp parsing if overflow

[jira] [Commented] (SPARK-32424) Fix silent data change for timestamp parsing if overflow happens

[ https://issues.apache.org/jira/browse/SPARK-32424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164251#comment-17164251 ] Apache Spark commented on SPARK-32424: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Created] (SPARK-32424) Fix silent data change for timestamp parsing if overflow happens

Kent Yao created SPARK-32424: Summary: Fix silent data change for timestamp parsing if overflow happens Key: SPARK-32424 URL: https://issues.apache.org/jira/browse/SPARK-32424 Project: Spark

[jira] [Updated] (SPARK-32409) Remove dependency between spark.metrics.staticSources.enabled and JVMSource registration

[ https://issues.apache.org/jira/browse/SPARK-32409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-32409: Description: SPARK-29654 has introduced the configuration

[jira] [Created] (SPARK-32423) class 'DataFrame' returns instance of type(self) instead of DataFrame

Timothy created SPARK-32423: --- Summary: class 'DataFrame' returns instance of type(self) instead of DataFrame Key: SPARK-32423 URL: https://issues.apache.org/jira/browse/SPARK-32423 Project: Spark