[jira] [Resolved] (SPARK-35479) Format PartitionFilters IN strings in scan nodes

2021-05-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35479. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32615

[jira] [Assigned] (SPARK-35479) Format PartitionFilters IN strings in scan nodes

2021-05-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35479: --- Assignee: Takeshi Yamamuro > Format PartitionFilters IN strings in scan nodes >

[jira] [Assigned] (SPARK-35445) Reduce the execution time of DeduplicateRelations

2021-05-20 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-35445: -- Assignee: wuyi > Reduce the execution time of DeduplicateRelations >

[jira] [Resolved] (SPARK-35445) Reduce the execution time of DeduplicateRelations

2021-05-20 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35445. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32590

[jira] [Resolved] (SPARK-35456) Show invalid value in config entry check error message

2021-05-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35456. -- Fix Version/s: 3.2.0 Resolution: Fixed Fixed in

[jira] [Assigned] (SPARK-35456) Show invalid value in config entry check error message

2021-05-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35456: Assignee: Kent Yao > Show invalid value in config entry check error message >

[jira] [Updated] (SPARK-35481) Create more robust link for Data Source Options

2021-05-20 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35481: Description: Now the link for the Data Source Options using /latest/, but it possibly be broken

[jira] [Updated] (SPARK-35481) Create more robust link for Data Source Options

2021-05-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35481: - Description: Now the link for the Data Source Options using /latest/, but it possibly be

[jira] [Updated] (SPARK-35481) Create more robust link for Data Source Options

2021-05-20 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35481: Description: Now the link for the Data Source Options using /latest/, but it possibly be broken

[jira] [Created] (SPARK-35481) Create more robust link for Data Source Options

2021-05-20 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35481: --- Summary: Create more robust link for Data Source Options Key: SPARK-35481 URL: https://issues.apache.org/jira/browse/SPARK-35481 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35480) percentile_approx function doesn't work with pivot

2021-05-20 Thread Christopher Bryant (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Bryant updated SPARK-35480: --- Description: The percentile_approx PySpark function does not appear to treat the

[jira] [Updated] (SPARK-35480) percentile_approx function doesn't work with pivot

2021-05-20 Thread Christopher Bryant (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Bryant updated SPARK-35480: --- Description: The percentile_approx PySpark function does not appear to treat the

[jira] [Created] (SPARK-35480) percentile_approx function doesn't work with pivot

2021-05-20 Thread Christopher Bryant (Jira)
Christopher Bryant created SPARK-35480: -- Summary: percentile_approx function doesn't work with pivot Key: SPARK-35480 URL: https://issues.apache.org/jira/browse/SPARK-35480 Project: Spark

[jira] [Assigned] (SPARK-35454) Ambiguous self-join doesn't fail after transfroming the dataset to dataframe

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35454: Assignee: Apache Spark > Ambiguous self-join doesn't fail after transfroming the dataset

[jira] [Assigned] (SPARK-35454) Ambiguous self-join doesn't fail after transfroming the dataset to dataframe

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35454: Assignee: (was: Apache Spark) > Ambiguous self-join doesn't fail after transfroming

[jira] [Commented] (SPARK-35454) Ambiguous self-join doesn't fail after transfroming the dataset to dataframe

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348935#comment-17348935 ] Apache Spark commented on SPARK-35454: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Commented] (SPARK-35479) Format PartitionFilters IN strings in scan nodes

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348923#comment-17348923 ] Apache Spark commented on SPARK-35479: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35479) Format PartitionFilters IN strings in scan nodes

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35479: Assignee: Apache Spark > Format PartitionFilters IN strings in scan nodes >

[jira] [Assigned] (SPARK-35479) Format PartitionFilters IN strings in scan nodes

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35479: Assignee: (was: Apache Spark) > Format PartitionFilters IN strings in scan nodes >

[jira] [Created] (SPARK-35479) Format PartitionFilters IN strings in scan nodes

2021-05-20 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-35479: Summary: Format PartitionFilters IN strings in scan nodes Key: SPARK-35479 URL: https://issues.apache.org/jira/browse/SPARK-35479 Project: Spark

[jira] [Assigned] (SPARK-35465) Enable disallow_untyped_defs mypy check except for major modules.

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35465: Assignee: Apache Spark > Enable disallow_untyped_defs mypy check except for major

[jira] [Commented] (SPARK-35465) Enable disallow_untyped_defs mypy check except for major modules.

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348832#comment-17348832 ] Apache Spark commented on SPARK-35465: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35465) Enable disallow_untyped_defs mypy check except for major modules.

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35465: Assignee: (was: Apache Spark) > Enable disallow_untyped_defs mypy check except for

[jira] [Created] (SPARK-35477) Enable disallow_untyped_defs mypy check for pyspark.pandas.utils.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35477: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.utils. Key: SPARK-35477 URL: https://issues.apache.org/jira/browse/SPARK-35477 Project: Spark

[jira] [Created] (SPARK-35478) Enable disallow_untyped_defs mypy check for pyspark.pandas.window.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35478: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.window. Key: SPARK-35478 URL: https://issues.apache.org/jira/browse/SPARK-35478 Project: Spark

[jira] [Created] (SPARK-35476) Enable disallow_untyped_defs mypy check for pyspark.pandas.series.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35476: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.series. Key: SPARK-35476 URL: https://issues.apache.org/jira/browse/SPARK-35476 Project: Spark

[jira] [Created] (SPARK-35475) Enable disallow_untyped_defs mypy check for pyspark.pandas.namespace.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35475: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.namespace. Key: SPARK-35475 URL: https://issues.apache.org/jira/browse/SPARK-35475 Project:

[jira] [Created] (SPARK-35474) Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35474: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing. Key: SPARK-35474 URL: https://issues.apache.org/jira/browse/SPARK-35474 Project:

[jira] [Created] (SPARK-35472) Enable disallow_untyped_defs mypy check for pyspark.pandas.generic.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35472: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.generic. Key: SPARK-35472 URL: https://issues.apache.org/jira/browse/SPARK-35472 Project: Spark

[jira] [Created] (SPARK-35473) Enable disallow_untyped_defs mypy check for pyspark.pandas.groupby.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35473: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.groupby. Key: SPARK-35473 URL: https://issues.apache.org/jira/browse/SPARK-35473 Project: Spark

[jira] [Created] (SPARK-35470) Enable disallow_untyped_defs mypy check for pyspark.pandas.base.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35470: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.base. Key: SPARK-35470 URL: https://issues.apache.org/jira/browse/SPARK-35470 Project: Spark

[jira] [Created] (SPARK-35471) Enable disallow_untyped_defs mypy check for pyspark.pandas.frame.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35471: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.frame. Key: SPARK-35471 URL: https://issues.apache.org/jira/browse/SPARK-35471 Project: Spark

[jira] [Created] (SPARK-35468) Enable disallow_untyped_defs mypy check for pyspark.pandas.typedef.typehints.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35468: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.typedef.typehints. Key: SPARK-35468 URL: https://issues.apache.org/jira/browse/SPARK-35468

[jira] [Created] (SPARK-35469) Enable disallow_untyped_defs mypy check for pyspark.pandas.accessors.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35469: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.accessors. Key: SPARK-35469 URL: https://issues.apache.org/jira/browse/SPARK-35469 Project:

[jira] [Updated] (SPARK-35466) Enable disallow_untyped_defs mypy check for pyspark.pandas.data_type_ops.

2021-05-20 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-35466: -- Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.data_type_ops. (was:

[jira] [Created] (SPARK-35467) Enable disallow_untyped_defs mypy check for pyspark.pandas.spark.accessors.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35467: - Summary: Enable disallow_untyped_defs mypy check for pyspark.pandas.spark.accessors. Key: SPARK-35467 URL: https://issues.apache.org/jira/browse/SPARK-35467

[jira] [Created] (SPARK-35466) Enable disallow_untyped_defs for pyspark.pandas.data_type_ops.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35466: - Summary: Enable disallow_untyped_defs for pyspark.pandas.data_type_ops. Key: SPARK-35466 URL: https://issues.apache.org/jira/browse/SPARK-35466 Project: Spark

[jira] [Created] (SPARK-35465) Enable disallow_untyped_defs mypy check except for major modules.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35465: - Summary: Enable disallow_untyped_defs mypy check except for major modules. Key: SPARK-35465 URL: https://issues.apache.org/jira/browse/SPARK-35465 Project: Spark

[jira] [Created] (SPARK-35464) pandas APIs on Spark: Enable mypy check "disallow_untyped_defs" for main codes.

2021-05-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-35464: - Summary: pandas APIs on Spark: Enable mypy check "disallow_untyped_defs" for main codes. Key: SPARK-35464 URL: https://issues.apache.org/jira/browse/SPARK-35464

[jira] [Resolved] (SPARK-35364) Renaming the existing Koalas related codes.

2021-05-20 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35364. --- Fix Version/s: 3.2.0 Assignee: Haejoon Lee Resolution: Fixed Issue resolved

[jira] [Resolved] (SPARK-35463) Skip checking checksum on a system doesn't have `shasum`

2021-05-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35463. --- Fix Version/s: 3.0.3 3.1.2 3.2.0 Resolution:

[jira] [Assigned] (SPARK-35463) Skip checking checksum on a system doesn't have `shasum`

2021-05-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35463: - Assignee: Dongjoon Hyun > Skip checking checksum on a system doesn't have `shasum` >

[jira] [Resolved] (SPARK-35462) Upgrade Kubernetes-client to 5.4.0 to support K8s 1.21 models

2021-05-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35462. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32612

[jira] [Assigned] (SPARK-35462) Upgrade Kubernetes-client to 5.4.0 to support K8s 1.21 models

2021-05-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35462: - Assignee: Dongjoon Hyun > Upgrade Kubernetes-client to 5.4.0 to support K8s 1.21

[jira] [Commented] (SPARK-35463) Skip checking checksum on a system doesn't have `shasum`

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348799#comment-17348799 ] Apache Spark commented on SPARK-35463: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-35463) Skip checking checksum on a system doesn't have `shasum`

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35463: Assignee: (was: Apache Spark) > Skip checking checksum on a system doesn't have

[jira] [Assigned] (SPARK-35463) Skip checking checksum on a system doesn't have `shasum`

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35463: Assignee: Apache Spark > Skip checking checksum on a system doesn't have `shasum` >

[jira] [Commented] (SPARK-35463) Skip checking checksum on a system doesn't have `shasum`

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348800#comment-17348800 ] Apache Spark commented on SPARK-35463: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-35458) ARM CI failed: failed to validate maven sha512

2021-05-20 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-35458: - Priority: Minor (was: Major) > ARM CI failed: failed to validate maven sha512 >

[jira] [Updated] (SPARK-35463) Skip checking checksum on a system doesn't have `shasum`

2021-05-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35463: -- Target Version/s: 3.0.3, 3.1.2, 3.2.0 > Skip checking checksum on a system doesn't have

[jira] [Resolved] (SPARK-35458) ARM CI failed: failed to validate maven sha512

2021-05-20 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-35458. -- Fix Version/s: 3.0.3 3.1.2 3.2.0 Resolution:

[jira] [Updated] (SPARK-35463) Skip checking checksum on a system doesn't have `shasum`

2021-05-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35463: -- Priority: Blocker (was: Major) > Skip checking checksum on a system doesn't have `shasum` >

[jira] [Assigned] (SPARK-35458) ARM CI failed: failed to validate maven sha512

2021-05-20 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-35458: Assignee: Yikun Jiang > ARM CI failed: failed to validate maven sha512 >

[jira] [Created] (SPARK-35463) Skip checking checksum on a system doesn't have `shasum`

2021-05-20 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-35463: - Summary: Skip checking checksum on a system doesn't have `shasum` Key: SPARK-35463 URL: https://issues.apache.org/jira/browse/SPARK-35463 Project: Spark

[jira] [Commented] (SPARK-18683) REST APIs for standalone Master、Workers and Applications

2021-05-20 Thread Mayank Asthana (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348708#comment-17348708 ] Mayank Asthana commented on SPARK-18683: Looking through the code I found that there is a 

[jira] [Assigned] (SPARK-35462) Upgrade Kubernetes-client to 5.4.0 to support K8s 1.21 models

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35462: Assignee: (was: Apache Spark) > Upgrade Kubernetes-client to 5.4.0 to support K8s

[jira] [Commented] (SPARK-35462) Upgrade Kubernetes-client to 5.4.0 to support K8s 1.21 models

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348681#comment-17348681 ] Apache Spark commented on SPARK-35462: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-35462) Upgrade Kubernetes-client to 5.4.0 to support K8s 1.21 models

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35462: Assignee: Apache Spark > Upgrade Kubernetes-client to 5.4.0 to support K8s 1.21 models >

[jira] [Created] (SPARK-35462) Upgrade Kubernetes-client to 5.4.0 to support K8s 1.21 models

2021-05-20 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-35462: - Summary: Upgrade Kubernetes-client to 5.4.0 to support K8s 1.21 models Key: SPARK-35462 URL: https://issues.apache.org/jira/browse/SPARK-35462 Project: Spark

[jira] [Updated] (SPARK-35461) Error when reading dictionary-encoded Parquet int column when read schema is bigint

2021-05-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35461: -- Issue Type: Improvement (was: Bug) > Error when reading dictionary-encoded Parquet int

[jira] [Commented] (SPARK-35461) Error when reading dictionary-encoded Parquet int column when read schema is bigint

2021-05-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348678#comment-17348678 ] Dongjoon Hyun commented on SPARK-35461: --- For a record, Apache Spark file-based data sources have

[jira] [Updated] (SPARK-35461) Error when reading dictionary-encoded Parquet int column when read schema is bigint

2021-05-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35461: -- Affects Version/s: (was: 3.1.1) (was: 3.0.2)

[jira] [Commented] (SPARK-35461) Error when reading dictionary-encoded Parquet int column when read schema is bigint

2021-05-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348677#comment-17348677 ] Dongjoon Hyun commented on SPARK-35461: --- This is a well-known limitation after we build a test

[jira] [Commented] (SPARK-35314) Support arithmetic operations against bool IndexOpsMixin

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348675#comment-17348675 ] Apache Spark commented on SPARK-35314: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-35314) Support arithmetic operations against bool IndexOpsMixin

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35314: Assignee: Apache Spark > Support arithmetic operations against bool IndexOpsMixin >

[jira] [Assigned] (SPARK-35314) Support arithmetic operations against bool IndexOpsMixin

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35314: Assignee: (was: Apache Spark) > Support arithmetic operations against bool

[jira] [Commented] (SPARK-35314) Support arithmetic operations against bool IndexOpsMixin

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348672#comment-17348672 ] Apache Spark commented on SPARK-35314: -- User 'xinrong-databricks' has created a pull request for

[jira] [Updated] (SPARK-35314) Support arithmetic operations against bool IndexOpsMixin

2021-05-20 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35314: - Summary: Support arithmetic operations against bool IndexOpsMixin (was: Support arithmetic

[jira] [Commented] (SPARK-35461) Error when reading dictionary-encoded Parquet int column when read schema is bigint

2021-05-20 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348667#comment-17348667 ] Chao Sun commented on SPARK-35461: -- Actually this also fails when turning off the vectorized reader:

[jira] [Created] (SPARK-35461) Error when reading dictionary-encoded Parquet int column when read schema is bigint

2021-05-20 Thread Chao Sun (Jira)
Chao Sun created SPARK-35461: Summary: Error when reading dictionary-encoded Parquet int column when read schema is bigint Key: SPARK-35461 URL: https://issues.apache.org/jira/browse/SPARK-35461 Project:

[jira] [Updated] (SPARK-33743) Change datatype mapping in JDBC mssqldialect: DATETIME to DATETIME2

2021-05-20 Thread Lu Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Xu updated SPARK-33743: -- Description: *datetime v/s datetime2* Spark datetime type is

[jira] [Updated] (SPARK-35460) invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

2021-05-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35460: -- Affects Version/s: (was: 3.1.1) 3.2.0 > invalid

[jira] [Updated] (SPARK-33743) Change datatype mapping in JDBC mssqldialect: DATETIME to DATETIME2

2021-05-20 Thread Lu Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Xu updated SPARK-33743: -- Description: *datetime v/s datetime2* Spark datetime type is

[jira] [Updated] (SPARK-33743) Change datatype mapping in JDBC mssqldialect: DATETIME to DATETIME2

2021-05-20 Thread Lu Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Xu updated SPARK-33743: -- Description: *datetime v/s datetime2* Spark datetime type is

[jira] [Updated] (SPARK-33743) Change datatype mapping in JDBC mssqldialect: DATETIME to DATETIME2

2021-05-20 Thread Lu Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Xu updated SPARK-33743: -- Description: *datetime v/s datetime2* Spark datetime type is

[jira] [Updated] (SPARK-33743) Change datatype mapping in JDBC mssqldialect: DATETIME to DATETIME2

2021-05-20 Thread Lu Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Xu updated SPARK-33743: -- Description: *datetime v/s datetime2* Spark datetime type is

[jira] [Commented] (SPARK-35256) Subexpression elimination leading to a performance regression

2021-05-20 Thread Ondrej Kokes (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348576#comment-17348576 ] Ondrej Kokes commented on SPARK-35256: -- [~Kimahriman] I think you're right - I've just built the PR

[jira] [Commented] (SPARK-35460) invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348544#comment-17348544 ] Apache Spark commented on SPARK-35460: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35460) invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35460: Assignee: (was: Apache Spark) > invalid `spark.kubernetes.executor.podNamePrefix`

[jira] [Commented] (SPARK-35460) invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348543#comment-17348543 ] Apache Spark commented on SPARK-35460: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35460) invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35460: Assignee: Apache Spark > invalid `spark.kubernetes.executor.podNamePrefix` causes app

[jira] [Created] (SPARK-35460) invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

2021-05-20 Thread Kent Yao (Jira)
Kent Yao created SPARK-35460: Summary: invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang Key: SPARK-35460 URL: https://issues.apache.org/jira/browse/SPARK-35460 Project: Spark

[jira] [Commented] (SPARK-29223) Kafka source: offset by timestamp - allow specifying timestamp for "all partitions"

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348508#comment-17348508 ] Apache Spark commented on SPARK-29223: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Commented] (SPARK-33867) java.time.Instant and java.time.LocalDate not handled in org.apache.spark.sql.jdbc.JdbcDialect#compileValue

2021-05-20 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348323#comment-17348323 ] Takeshi Yamamuro commented on SPARK-33867: -- Please see the "Fix Version/s" in this jira and

[jira] [Commented] (SPARK-35373) Verify checksums of downloaded artifacts in build/mvn

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348318#comment-17348318 ] Apache Spark commented on SPARK-35373: -- User 'Yikun' has created a pull request for this issue:

[jira] [Commented] (SPARK-33867) java.time.Instant and java.time.LocalDate not handled in org.apache.spark.sql.jdbc.JdbcDialect#compileValue

2021-05-20 Thread Cristi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348298#comment-17348298 ] Cristi commented on SPARK-33867: looks like it: 

[jira] [Resolved] (SPARK-35459) Move AvroRowReaderSuite to a separate file

2021-05-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35459. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32607

[jira] [Commented] (SPARK-33867) java.time.Instant and java.time.LocalDate not handled in org.apache.spark.sql.jdbc.JdbcDialect#compileValue

2021-05-20 Thread LiaoHanwen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348294#comment-17348294 ] LiaoHanwen commented on SPARK-33867: Is this fixed on branch-3.1? > java.time.Instant and

[jira] [Resolved] (SPARK-35424) Remove some useless code in ExternalBlockHandler

2021-05-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35424. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32571

[jira] [Assigned] (SPARK-35424) Remove some useless code in ExternalBlockHandler

2021-05-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35424: Assignee: weixiuli > Remove some useless code in ExternalBlockHandler >

[jira] [Assigned] (SPARK-35459) Move AvroRowReaderSuite to a separate file

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35459: Assignee: Max Gekk (was: Apache Spark) > Move AvroRowReaderSuite to a separate file >

[jira] [Commented] (SPARK-35459) Move AvroRowReaderSuite to a separate file

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348195#comment-17348195 ] Apache Spark commented on SPARK-35459: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-35459) Move AvroRowReaderSuite to a separate file

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348194#comment-17348194 ] Apache Spark commented on SPARK-35459: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35459) Move AvroRowReaderSuite to a separate file

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35459: Assignee: Apache Spark (was: Max Gekk) > Move AvroRowReaderSuite to a separate file >

[jira] [Updated] (SPARK-35378) Eagerly execute non-root Command so that query command with CTE

2021-05-20 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-35378: --- Summary: Eagerly execute non-root Command so that query command with CTE (was: Eagerly execute

[jira] [Assigned] (SPARK-35287) RemoveRedundantProjects removes non-redundant projects

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35287: Assignee: (was: Apache Spark) > RemoveRedundantProjects removes non-redundant

[jira] [Commented] (SPARK-35287) RemoveRedundantProjects removes non-redundant projects

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348184#comment-17348184 ] Apache Spark commented on SPARK-35287: -- User 'sarutak' has created a pull request for this issue:

[jira] [Created] (SPARK-35459) Move AvroRowReaderSuite to a separate file

2021-05-20 Thread Max Gekk (Jira)
Max Gekk created SPARK-35459: Summary: Move AvroRowReaderSuite to a separate file Key: SPARK-35459 URL: https://issues.apache.org/jira/browse/SPARK-35459 Project: Spark Issue Type: Test

[jira] [Assigned] (SPARK-35287) RemoveRedundantProjects removes non-redundant projects

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35287: Assignee: Apache Spark > RemoveRedundantProjects removes non-redundant projects >

[jira] [Assigned] (SPARK-35446) Override getJDBCType in MySQLDialect to map FloatType to FLOAT

2021-05-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35446: Assignee: (was: Apache Spark) > Override getJDBCType in MySQLDialect to map

  1   2   >