[jira] [Created] (SPARK-35927) Remove type collection AllTimestampTypes

2021-06-28 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-35927: -- Summary: Remove type collection AllTimestampTypes Key: SPARK-35927 URL: https://issues.apache.org/jira/browse/SPARK-35927 Project: Spark Issue Type: Sub-

[jira] [Assigned] (SPARK-35873) Cleanup the version logic from the pandas API on Spark

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35873: Assignee: Apache Spark > Cleanup the version logic from the pandas API on Spark > ---

[jira] [Commented] (SPARK-35873) Cleanup the version logic from the pandas API on Spark

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371169#comment-17371169 ] Apache Spark commented on SPARK-35873: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-35873) Cleanup the version logic from the pandas API on Spark

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35873: Assignee: (was: Apache Spark) > Cleanup the version logic from the pandas API on Spar

[jira] [Commented] (SPARK-35926) Support YearMonthIntervalType in width-bucket function

2021-06-28 Thread PengLei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371167#comment-17371167 ] PengLei commented on SPARK-35926: - I'm going to do this. > Support YearMonthIntervalTyp

[jira] [Created] (SPARK-35926) Support YearMonthIntervalType in width-bucket function

2021-06-28 Thread PengLei (Jira)
PengLei created SPARK-35926: --- Summary: Support YearMonthIntervalType in width-bucket function Key: SPARK-35926 URL: https://issues.apache.org/jira/browse/SPARK-35926 Project: Spark Issue Type: Sub-

[jira] [Commented] (SPARK-35925) Support DayTimeIntervalType in width-bucket function

2021-06-28 Thread PengLei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371163#comment-17371163 ] PengLei commented on SPARK-35925: - [~maxgekk]  Do you think it's necessary? I'm going to

[jira] [Resolved] (SPARK-35916) Support subtraction among Date/Timestamp/TimestampWithoutTZ

2021-06-28 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35916. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33115 [https:

[jira] [Created] (SPARK-35925) Support DayTimeIntervalType in width-bucket function

2021-06-28 Thread PengLei (Jira)
PengLei created SPARK-35925: --- Summary: Support DayTimeIntervalType in width-bucket function Key: SPARK-35925 URL: https://issues.apache.org/jira/browse/SPARK-35925 Project: Spark Issue Type: Sub-ta

[jira] [Commented] (SPARK-35915) Kafka doesn't recover from data loss

2021-06-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371160#comment-17371160 ] Hyukjin Kwon commented on SPARK-35915: -- cc [~kabhwan] FYI > Kafka doesn't recover

[jira] [Commented] (SPARK-35483) Add a new GA test job for the docker integration tests

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371127#comment-17371127 ] Apache Spark commented on SPARK-35483: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-35924) Add Java 17 ea build test to GitHub action

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371076#comment-17371076 ] Apache Spark commented on SPARK-35924: -- User 'williamhyun' has created a pull reque

[jira] [Assigned] (SPARK-35924) Add Java 17 ea build test to GitHub action

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35924: Assignee: Apache Spark > Add Java 17 ea build test to GitHub action > ---

[jira] [Assigned] (SPARK-35924) Add Java 17 ea build test to GitHub action

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35924: Assignee: (was: Apache Spark) > Add Java 17 ea build test to GitHub action >

[jira] [Updated] (SPARK-35924) Add Java 17 ea build test to GitHub action

2021-06-28 Thread William Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Hyun updated SPARK-35924: - Component/s: Tests > Add Java 17 ea build test to GitHub action > --

[jira] [Created] (SPARK-35924) Add Java 17 ea build test to GitHub action

2021-06-28 Thread William Hyun (Jira)
William Hyun created SPARK-35924: Summary: Add Java 17 ea build test to GitHub action Key: SPARK-35924 URL: https://issues.apache.org/jira/browse/SPARK-35924 Project: Spark Issue Type: Improv

[jira] [Commented] (SPARK-35483) Add a new GA test job for the docker integration tests

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371063#comment-17371063 ] Apache Spark commented on SPARK-35483: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-35483) Add a new GA test job for the docker integration tests

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371062#comment-17371062 ] Apache Spark commented on SPARK-35483: -- User 'dongjoon-hyun' has created a pull req

[jira] [Resolved] (SPARK-35922) Upgrade maven-shade-plugin to 3.2.4

2021-06-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35922. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33122 [https://

[jira] [Assigned] (SPARK-35922) Upgrade maven-shade-plugin to 3.2.4

2021-06-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35922: - Assignee: Dongjoon Hyun > Upgrade maven-shade-plugin to 3.2.4 > ---

[jira] [Commented] (SPARK-34302) Migrate ALTER TABLE .. CHANGE COLUMN to new resolution framework

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371041#comment-17371041 ] Apache Spark commented on SPARK-34302: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-34302) Migrate ALTER TABLE .. CHANGE COLUMN to new resolution framework

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371040#comment-17371040 ] Apache Spark commented on SPARK-34302: -- User 'dongjoon-hyun' has created a pull req

[jira] [Resolved] (SPARK-35920) Upgrade to Chill 0.10.0

2021-06-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35920. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33119 [https://

[jira] [Assigned] (SPARK-35920) Upgrade to Chill 0.10.0

2021-06-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35920: - Assignee: Dongjoon Hyun > Upgrade to Chill 0.10.0 > --- > >

[jira] [Assigned] (SPARK-35923) Coalesce empty partition with mixed CoalescedPartitionSpec and PartialReducerPartitionSpec

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35923: Assignee: (was: Apache Spark) > Coalesce empty partition with mixed CoalescedPartitio

[jira] [Assigned] (SPARK-35923) Coalesce empty partition with mixed CoalescedPartitionSpec and PartialReducerPartitionSpec

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35923: Assignee: Apache Spark > Coalesce empty partition with mixed CoalescedPartitionSpec and

[jira] [Commented] (SPARK-35923) Coalesce empty partition with mixed CoalescedPartitionSpec and PartialReducerPartitionSpec

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371038#comment-17371038 ] Apache Spark commented on SPARK-35923: -- User 'ulysses-you' has created a pull reque

[jira] [Created] (SPARK-35923) Coalesce empty partition with mixed CoalescedPartitionSpec and PartialReducerPartitionSpec

2021-06-28 Thread XiDuo You (Jira)
XiDuo You created SPARK-35923: - Summary: Coalesce empty partition with mixed CoalescedPartitionSpec and PartialReducerPartitionSpec Key: SPARK-35923 URL: https://issues.apache.org/jira/browse/SPARK-35923

[jira] [Assigned] (SPARK-35876) array_zip unexpected column names

2021-06-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35876: Assignee: Kousuke Saruta > array_zip unexpected column names > --

[jira] [Resolved] (SPARK-35876) array_zip unexpected column names

2021-06-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35876. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33106 [https://gi

[jira] [Assigned] (SPARK-35922) Upgrade maven-shade-plugin to 3.2.4

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35922: Assignee: (was: Apache Spark) > Upgrade maven-shade-plugin to 3.2.4 > ---

[jira] [Assigned] (SPARK-35922) Upgrade maven-shade-plugin to 3.2.4

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35922: Assignee: (was: Apache Spark) > Upgrade maven-shade-plugin to 3.2.4 > ---

[jira] [Assigned] (SPARK-35922) Upgrade maven-shade-plugin to 3.2.4

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35922: Assignee: Apache Spark > Upgrade maven-shade-plugin to 3.2.4 > --

[jira] [Commented] (SPARK-35922) Upgrade maven-shade-plugin to 3.2.4

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371014#comment-17371014 ] Apache Spark commented on SPARK-35922: -- User 'dongjoon-hyun' has created a pull req

[jira] [Created] (SPARK-35922) Upgrade maven-shade-plugin to 3.2.4

2021-06-28 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-35922: - Summary: Upgrade maven-shade-plugin to 3.2.4 Key: SPARK-35922 URL: https://issues.apache.org/jira/browse/SPARK-35922 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-34537) Repartition miss/duplicated data

2021-06-28 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-34537. --- Resolution: Not A Problem > Repartition miss/duplicated data > > >

[jira] [Commented] (SPARK-35914) Driver can't distribute task to executor because NullPointerException

2021-06-28 Thread Helt Long (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371009#comment-17371009 ] Helt Long commented on SPARK-35914: --- I guess this problem is related to hadoop-version

[jira] [Assigned] (SPARK-35921) ${spark.yarn.isHadoopProvided} in config.properties is not edited if build with SBT

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35921: Assignee: Apache Spark (was: Kousuke Saruta) > ${spark.yarn.isHadoopProvided} in config.

[jira] [Assigned] (SPARK-35921) ${spark.yarn.isHadoopProvided} in config.properties is not edited if build with SBT

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35921: Assignee: Kousuke Saruta (was: Apache Spark) > ${spark.yarn.isHadoopProvided} in config.

[jira] [Commented] (SPARK-35921) ${spark.yarn.isHadoopProvided} in config.properties is not edited if build with SBT

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371007#comment-17371007 ] Apache Spark commented on SPARK-35921: -- User 'sarutak' has created a pull request f

[jira] [Resolved] (SPARK-35802) Error loading the stages/stage/ page in spark UI

2021-06-28 Thread Helt Long (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Helt Long resolved SPARK-35802. --- Resolution: Not A Bug I try hadoop-2.7.5, the problem is not existed. So I confirm it's because I u

[jira] [Assigned] (SPARK-34302) Migrate ALTER TABLE .. CHANGE COLUMN to new resolution framework

2021-06-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34302: --- Assignee: Terry Kim (was: Max Gekk) > Migrate ALTER TABLE .. CHANGE COLUMN to new resoluti

[jira] [Resolved] (SPARK-34302) Migrate ALTER TABLE .. CHANGE COLUMN to new resolution framework

2021-06-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34302. - Resolution: Fixed Issue resolved by pull request 33113 [https://github.com/apache/spark/pull/331

[jira] [Resolved] (SPARK-35888) Add dataSize field in CoalescedPartitionSpec

2021-06-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35888. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33079 [https://gith

[jira] [Assigned] (SPARK-35888) Add dataSize field in CoalescedPartitionSpec

2021-06-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35888: --- Assignee: XiDuo You > Add dataSize field in CoalescedPartitionSpec > --

[jira] [Updated] (SPARK-35921) ${spark.yarn.isHadoopProvided} in config.properties is not edited if build with SBT

2021-06-28 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35921: --- Summary: ${spark.yarn.isHadoopProvided} in config.properties is not edited if build with SBT

[jira] [Created] (SPARK-35921) The value of spark.yarn.isHadoopProvided property in config.properties is not edited if build with SBT

2021-06-28 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35921: -- Summary: The value of spark.yarn.isHadoopProvided property in config.properties is not edited if build with SBT Key: SPARK-35921 URL: https://issues.apache.org/jira/browse/SPA

[jira] [Commented] (SPARK-26764) [SPIP] Spark Relational Cache

2021-06-28 Thread Adrian Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370993#comment-17370993 ] Adrian Wang commented on SPARK-26764: - [~zshao] Thanks for the interest. We created

[jira] [Commented] (SPARK-35899) Add a utility to convert connector expressions to Catalyst expressions

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370991#comment-17370991 ] Apache Spark commented on SPARK-35899: -- User 'aokolnychyi' has created a pull reque

[jira] [Commented] (SPARK-35899) Add a utility to convert connector expressions to Catalyst expressions

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370990#comment-17370990 ] Apache Spark commented on SPARK-35899: -- User 'aokolnychyi' has created a pull reque

[jira] [Resolved] (SPARK-33898) Support SHOW CREATE TABLE in v2

2021-06-28 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-33898. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32931 [https://github.com

[jira] [Assigned] (SPARK-33898) Support SHOW CREATE TABLE in v2

2021-06-28 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-33898: Assignee: PengLei > Support SHOW CREATE TABLE in v2 > --- > >

[jira] [Resolved] (SPARK-35344) Support creating a Column of numpy literal value in pandas-on-Spark

2021-06-28 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35344. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue resolved

[jira] [Commented] (SPARK-35920) Upgrade to Chill 0.10.0

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370971#comment-17370971 ] Apache Spark commented on SPARK-35920: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-35920) Upgrade to Chill 0.10.0

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35920: Assignee: (was: Apache Spark) > Upgrade to Chill 0.10.0 > --- > >

[jira] [Commented] (SPARK-35920) Upgrade to Chill 0.10.0

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370970#comment-17370970 ] Apache Spark commented on SPARK-35920: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-35920) Upgrade to Chill 0.10.0

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35920: Assignee: Apache Spark > Upgrade to Chill 0.10.0 > --- > >

[jira] [Created] (SPARK-35920) Upgrade to Chill 0.10.0

2021-06-28 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-35920: - Summary: Upgrade to Chill 0.10.0 Key: SPARK-35920 URL: https://issues.apache.org/jira/browse/SPARK-35920 Project: Spark Issue Type: Task Componen

[jira] [Created] (SPARK-35919) Support pathlib.PurePath-like objects in DataFrameReader / DataFrameWriter

2021-06-28 Thread Andrew Grigorev (Jira)
Andrew Grigorev created SPARK-35919: --- Summary: Support pathlib.PurePath-like objects in DataFrameReader / DataFrameWriter Key: SPARK-35919 URL: https://issues.apache.org/jira/browse/SPARK-35919 Proj

[jira] [Assigned] (SPARK-35917) Disable push-based shuffle until the feature is complete

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35917: Assignee: Apache Spark > Disable push-based shuffle until the feature is complete > -

[jira] [Assigned] (SPARK-35917) Disable push-based shuffle until the feature is complete

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35917: Assignee: (was: Apache Spark) > Disable push-based shuffle until the feature is compl

[jira] [Commented] (SPARK-35917) Disable push-based shuffle until the feature is complete

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370877#comment-17370877 ] Apache Spark commented on SPARK-35917: -- User 'otterc' has created a pull request fo

[jira] [Updated] (SPARK-35881) [SQL] AQE does not support columnar execution for the final query stage

2021-06-28 Thread Andy Grove (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Grove updated SPARK-35881: --- Description: In AdaptiveSparkPlanExec, a query is broken down into stages and these stages are exec

[jira] [Created] (SPARK-35918) Consolidate logic between AvroSerializer/AvroDeserializer for schema mismatch handling and error messages

2021-06-28 Thread Erik Krogen (Jira)
Erik Krogen created SPARK-35918: --- Summary: Consolidate logic between AvroSerializer/AvroDeserializer for schema mismatch handling and error messages Key: SPARK-35918 URL: https://issues.apache.org/jira/browse/SPARK-

[jira] [Assigned] (SPARK-35910) Update remoteBlockBytes based on merged block info

2021-06-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35910: - Assignee: Kent Yao > Update remoteBlockBytes based on merged block info > -

[jira] [Resolved] (SPARK-35910) Update remoteBlockBytes based on merged block info

2021-06-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35910. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33109 [https://

[jira] [Commented] (SPARK-35859) Cleanup type hints.

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370842#comment-17370842 ] Apache Spark commented on SPARK-35859: -- User 'ueshin' has created a pull request fo

[jira] [Assigned] (SPARK-35859) Cleanup type hints.

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35859: Assignee: Apache Spark > Cleanup type hints. > --- > > Ke

[jira] [Assigned] (SPARK-35859) Cleanup type hints.

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35859: Assignee: (was: Apache Spark) > Cleanup type hints. > --- > >

[jira] [Commented] (SPARK-35859) Cleanup type hints.

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370840#comment-17370840 ] Apache Spark commented on SPARK-35859: -- User 'ueshin' has created a pull request fo

[jira] [Commented] (SPARK-35259) ExternalBlockHandler metrics have misleading unit in the name

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370792#comment-17370792 ] Apache Spark commented on SPARK-35259: -- User 'xkrogen' has created a pull request f

[jira] [Assigned] (SPARK-35259) ExternalBlockHandler metrics have misleading unit in the name

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35259: Assignee: Apache Spark > ExternalBlockHandler metrics have misleading unit in the name >

[jira] [Commented] (SPARK-35259) ExternalBlockHandler metrics have misleading unit in the name

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370790#comment-17370790 ] Apache Spark commented on SPARK-35259: -- User 'xkrogen' has created a pull request f

[jira] [Assigned] (SPARK-35259) ExternalBlockHandler metrics have misleading unit in the name

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35259: Assignee: (was: Apache Spark) > ExternalBlockHandler metrics have misleading unit in

[jira] [Updated] (SPARK-35259) ExternalBlockHandler metrics have misleading unit in the name

2021-06-28 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated SPARK-35259: Description: Today {{ExternalBlockHandler}} exposes a few {{Timer}} metrics: {code} // Time la

[jira] [Commented] (SPARK-26764) [SPIP] Spark Relational Cache

2021-06-28 Thread Zheng Shao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370770#comment-17370770 ] Zheng Shao commented on SPARK-26764: [~adrian-wang] It has been over 2 years since t

[jira] [Commented] (SPARK-29038) SPIP: Support Spark Materialized View

2021-06-28 Thread Zheng Shao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370768#comment-17370768 ] Zheng Shao commented on SPARK-29038: [~cltlfcjin] and [~AidenZhang]. I also recently

[jira] [Updated] (SPARK-35917) Disable push-based shuffle until the feature is complete

2021-06-28 Thread Chandni Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh updated SPARK-35917: -- Description: Push-based shuffle is partially merged in apache master but some of the tasks ar

[jira] [Created] (SPARK-35917) Disable push-based shuffle until the feature is complete

2021-06-28 Thread Chandni Singh (Jira)
Chandni Singh created SPARK-35917: - Summary: Disable push-based shuffle until the feature is complete Key: SPARK-35917 URL: https://issues.apache.org/jira/browse/SPARK-35917 Project: Spark Is

[jira] [Resolved] (SPARK-35898) Converting arrays with RowToColumnConverter triggers assertion

2021-06-28 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-35898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-35898. --- Fix Version/s: 3.1.3 3.2.0 Resolution: Fixed > Converting

[jira] [Assigned] (SPARK-35898) Converting arrays with RowToColumnConverter triggers assertion

2021-06-28 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-35898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-35898: - Assignee: Tom van Bussel > Converting arrays with RowToColumnConverter triggers

[jira] [Commented] (SPARK-35802) Error loading the stages/stage/ page in spark UI

2021-06-28 Thread Helt Long (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370626#comment-17370626 ] Helt Long commented on SPARK-35802: --- Sound like it's key point, I will try higher hado

[jira] [Commented] (SPARK-35802) Error loading the stages/stage/ page in spark UI

2021-06-28 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370590#comment-17370590 ] Kousuke Saruta commented on SPARK-35802: Hadoop version is 2.9.0 on my environme

[jira] [Assigned] (SPARK-35916) Support subtraction among Date/Timestamp/TimestampWithoutTZ

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35916: Assignee: Gengliang Wang (was: Apache Spark) > Support subtraction among Date/Timestamp/

[jira] [Commented] (SPARK-35916) Support subtraction among Date/Timestamp/TimestampWithoutTZ

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370578#comment-17370578 ] Apache Spark commented on SPARK-35916: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-35916) Support subtraction among Date/Timestamp/TimestampWithoutTZ

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35916: Assignee: Apache Spark (was: Gengliang Wang) > Support subtraction among Date/Timestamp/

[jira] [Created] (SPARK-35916) Support subtraction among Date/Timestamp/TimestampWithoutTZ

2021-06-28 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-35916: -- Summary: Support subtraction among Date/Timestamp/TimestampWithoutTZ Key: SPARK-35916 URL: https://issues.apache.org/jira/browse/SPARK-35916 Project: Spark

[jira] [Commented] (SPARK-35912) [SQL] JSON read behavior is different depending on the cache setting when nullable is false.

2021-06-28 Thread Fu Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370512#comment-17370512 ] Fu Chen commented on SPARK-35912: - Working on this > [SQL] JSON read behavior is differ

[jira] [Created] (SPARK-35915) Kafka doesn't recover from data loss

2021-06-28 Thread Yuval Yellin (Jira)
Yuval Yellin created SPARK-35915: Summary: Kafka doesn't recover from data loss Key: SPARK-35915 URL: https://issues.apache.org/jira/browse/SPARK-35915 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-35914) Driver can't distribute task to executor because NullPointerException

2021-06-28 Thread Helt Long (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Helt Long updated SPARK-35914: -- Attachment: webui stuck.png > Driver can't distribute task to executor because NullPointerException >

[jira] [Updated] (SPARK-35914) Driver can't distribute task to executor because NullPointerException

2021-06-28 Thread Helt Long (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Helt Long updated SPARK-35914: -- Attachment: stuck log.png > Driver can't distribute task to executor because NullPointerException > --

[jira] [Assigned] (SPARK-35904) Collapse above RebalancePartitions

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35904: Assignee: Yuming Wang (was: Apache Spark) > Collapse above RebalancePartitions > ---

[jira] [Assigned] (SPARK-35904) Collapse above RebalancePartitions

2021-06-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35904: Assignee: Apache Spark (was: Yuming Wang) > Collapse above RebalancePartitions > ---

[jira] [Updated] (SPARK-35904) Collapse above RebalancePartitions

2021-06-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-35904: Fix Version/s: (was: 3.2.0) > Collapse above RebalancePartitions > ---

[jira] [Reopened] (SPARK-35904) Collapse above RebalancePartitions

2021-06-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-35904: - Reverted at https://github.com/apache/spark/commit/108635af1708173a72bec0e36bf3f2cea5b088c4 > Coll

[jira] [Updated] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-06-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35672: - Fix Version/s: 3.1.3 > Spark fails to launch executors with very large user classpath lists on Y

[jira] [Commented] (SPARK-35802) Error loading the stages/stage/ page in spark UI

2021-06-28 Thread Helt Long (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370491#comment-17370491 ] Helt Long commented on SPARK-35802: --- [~sarutak] I used CDH 5.7.1, and used Spark on Y

[jira] [Comment Edited] (SPARK-35802) Error loading the stages/stage/ page in spark UI

2021-06-28 Thread Helt Long (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370491#comment-17370491 ] Helt Long edited comment on SPARK-35802 at 6/28/21, 8:33 AM: -

[jira] [Updated] (SPARK-35802) Error loading the stages/stage/ page in spark UI

2021-06-28 Thread Helt Long (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Helt Long updated SPARK-35802: -- Description: I try to load the sparkUI page for a specific stage, I get the following error: {quote}Un

[jira] [Created] (SPARK-35914) Driver can't distribute task to executor because NullPointerException

2021-06-28 Thread Helt Long (Jira)
Helt Long created SPARK-35914: - Summary: Driver can't distribute task to executor because NullPointerException Key: SPARK-35914 URL: https://issues.apache.org/jira/browse/SPARK-35914 Project: Spark

  1   2   >