[jira] [Commented] (SPARK-35707) optimize sparse GEMM by skipping bound checking

[ https://issues.apache.org/jira/browse/SPARK-35707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360607#comment-17360607 ] Apache Spark commented on SPARK-35707: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-35707) optimize sparse GEMM by skipping bound checking

[ https://issues.apache.org/jira/browse/SPARK-35707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35707: Assignee: (was: Apache Spark) > optimize sparse GEMM by skipping bound checking > ---

[jira] [Commented] (SPARK-35707) optimize sparse GEMM by skipping bound checking

[ https://issues.apache.org/jira/browse/SPARK-35707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360606#comment-17360606 ] Apache Spark commented on SPARK-35707: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-35707) optimize sparse GEMM by skipping bound checking

[ https://issues.apache.org/jira/browse/SPARK-35707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35707: Assignee: Apache Spark > optimize sparse GEMM by skipping bound checking > --

[jira] [Commented] (SPARK-29626) notEqual() should return true when the one is null, the other is not null

[ https://issues.apache.org/jira/browse/SPARK-29626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360632#comment-17360632 ] dc-heros commented on SPARK-29626: -- ok thanks, I will create a pull request to add notE

[jira] [Assigned] (SPARK-35673) Spark fails on unrecognized hint in subquery

[ https://issues.apache.org/jira/browse/SPARK-35673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35673: --- Assignee: Fu Chen > Spark fails on unrecognized hint in subquery >

[jira] [Resolved] (SPARK-35673) Spark fails on unrecognized hint in subquery

[ https://issues.apache.org/jira/browse/SPARK-35673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35673. - Fix Version/s: 3.1.3 3.2.0 3.0.3 Resolution: Fixed

[jira] [Commented] (SPARK-29626) notEqual() should return true when the one is null, the other is not null

[ https://issues.apache.org/jira/browse/SPARK-29626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360635#comment-17360635 ] dc-heros commented on SPARK-29626: -- should this func added to pyspark too?   > notEqu

[jira] [Commented] (SPARK-35706) Consider making the ':' in STRUCT data type definition optional

[ https://issues.apache.org/jira/browse/SPARK-35706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360643#comment-17360643 ] Apache Spark commented on SPARK-35706: -- User 'jerqi' has created a pull request for

[jira] [Issue Comment Deleted] (SPARK-29626) notEqual() should return true when the one is null, the other is not null

[ https://issues.apache.org/jira/browse/SPARK-29626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dc-heros updated SPARK-29626: - Comment: was deleted (was: ok thanks, I will create a pull request to add notEqualNullSafe  ) > notEqu

[jira] [Issue Comment Deleted] (SPARK-29626) notEqual() should return true when the one is null, the other is not null

[ https://issues.apache.org/jira/browse/SPARK-29626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dc-heros updated SPARK-29626: - Comment: was deleted (was: should this func added to pyspark too?  ) > notEqual() should return true w

[jira] [Created] (SPARK-35708) Add BaseTest for DataTypeOps

Yikun Jiang created SPARK-35708: --- Summary: Add BaseTest for DataTypeOps Key: SPARK-35708 URL: https://issues.apache.org/jira/browse/SPARK-35708 Project: Spark Issue Type: Sub-task Com

[jira] [Commented] (SPARK-35708) Add BaseTest for DataTypeOps

[ https://issues.apache.org/jira/browse/SPARK-35708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360654#comment-17360654 ] Apache Spark commented on SPARK-35708: -- User 'Yikun' has created a pull request for

[jira] [Assigned] (SPARK-35708) Add BaseTest for DataTypeOps

[ https://issues.apache.org/jira/browse/SPARK-35708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35708: Assignee: Apache Spark > Add BaseTest for DataTypeOps > > >

[jira] [Assigned] (SPARK-35708) Add BaseTest for DataTypeOps

[ https://issues.apache.org/jira/browse/SPARK-35708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35708: Assignee: (was: Apache Spark) > Add BaseTest for DataTypeOps > --

[jira] [Assigned] (SPARK-35708) Add BaseTest for DataTypeOps

[ https://issues.apache.org/jira/browse/SPARK-35708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35708: Assignee: Apache Spark > Add BaseTest for DataTypeOps > > >

[jira] [Created] (SPARK-35709) Remove links to Nomad integration in the Documentation

Paweł Ptaszyński created SPARK-35709: Summary: Remove links to Nomad integration in the Documentation Key: SPARK-35709 URL: https://issues.apache.org/jira/browse/SPARK-35709 Project: Spark

[jira] [Updated] (SPARK-35709) Remove links to Nomad integration in the Documentation

[ https://issues.apache.org/jira/browse/SPARK-35709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paweł Ptaszyński updated SPARK-35709: - Priority: Trivial (was: Major) > Remove links to Nomad integration in the Documentation

[jira] [Updated] (SPARK-35707) optimize sparse GEMM by skipping bound checking

[ https://issues.apache.org/jira/browse/SPARK-35707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-35707: - Affects Version/s: 3.1.3 3.0.3 > optimize sparse GEMM by skipping bound c

[jira] [Assigned] (SPARK-35709) Remove links to Nomad integration in the Documentation

[ https://issues.apache.org/jira/browse/SPARK-35709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35709: Assignee: (was: Apache Spark) > Remove links to Nomad integration in the Documentatio

[jira] [Commented] (SPARK-35709) Remove links to Nomad integration in the Documentation

[ https://issues.apache.org/jira/browse/SPARK-35709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360681#comment-17360681 ] Apache Spark commented on SPARK-35709: -- User 'pptaszynski' has created a pull reque

[jira] [Commented] (SPARK-35709) Remove links to Nomad integration in the Documentation

[ https://issues.apache.org/jira/browse/SPARK-35709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360680#comment-17360680 ] Apache Spark commented on SPARK-35709: -- User 'pptaszynski' has created a pull reque

[jira] [Assigned] (SPARK-35709) Remove links to Nomad integration in the Documentation

[ https://issues.apache.org/jira/browse/SPARK-35709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35709: Assignee: Apache Spark > Remove links to Nomad integration in the Documentation > ---

[jira] [Commented] (SPARK-35652) Different Behaviour join vs joinWith in self joining

[ https://issues.apache.org/jira/browse/SPARK-35652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360689#comment-17360689 ] dc-heros commented on SPARK-35652: -- I think we can fix this issues with the approach fr

[jira] [Commented] (SPARK-35652) Different Behaviour join vs joinWith in self joining

[ https://issues.apache.org/jira/browse/SPARK-35652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360685#comment-17360685 ] dc-heros commented on SPARK-35652: -- I will make a pull request soon > Different Behavi

[jira] [Created] (SPARK-35710) Support DPP + AQE when no reused broadcast exchange

Ke Jia created SPARK-35710: -- Summary: Support DPP + AQE when no reused broadcast exchange Key: SPARK-35710 URL: https://issues.apache.org/jira/browse/SPARK-35710 Project: Spark Issue Type: Improveme

[jira] [Updated] (SPARK-35710) Support DPP + AQE when no reused broadcast exchange

[ https://issues.apache.org/jira/browse/SPARK-35710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ke Jia updated SPARK-35710: --- Description: Support DPP + AQE when no reused broadcast exchange. > Support DPP + AQE when no reused broadca

[jira] [Assigned] (SPARK-35710) Support DPP + AQE when no reused broadcast exchange

[ https://issues.apache.org/jira/browse/SPARK-35710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35710: Assignee: Apache Spark > Support DPP + AQE when no reused broadcast exchange > --

[jira] [Commented] (SPARK-35710) Support DPP + AQE when no reused broadcast exchange

[ https://issues.apache.org/jira/browse/SPARK-35710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360692#comment-17360692 ] Apache Spark commented on SPARK-35710: -- User 'JkSelf' has created a pull request fo

[jira] [Commented] (SPARK-35710) Support DPP + AQE when no reused broadcast exchange

[ https://issues.apache.org/jira/browse/SPARK-35710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360690#comment-17360690 ] Apache Spark commented on SPARK-35710: -- User 'JkSelf' has created a pull request fo

[jira] [Assigned] (SPARK-35710) Support DPP + AQE when no reused broadcast exchange

[ https://issues.apache.org/jira/browse/SPARK-35710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35710: Assignee: (was: Apache Spark) > Support DPP + AQE when no reused broadcast exchange >

[jira] [Commented] (SPARK-35695) QueryExecutionListener does not see any observed metrics fired before persist/cache

[ https://issues.apache.org/jira/browse/SPARK-35695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360710#comment-17360710 ] Apache Spark commented on SPARK-35695: -- User 'tanelk' has created a pull request fo

[jira] [Assigned] (SPARK-35695) QueryExecutionListener does not see any observed metrics fired before persist/cache

[ https://issues.apache.org/jira/browse/SPARK-35695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35695: Assignee: (was: Apache Spark) > QueryExecutionListener does not see any observed metr

[jira] [Assigned] (SPARK-35695) QueryExecutionListener does not see any observed metrics fired before persist/cache

[ https://issues.apache.org/jira/browse/SPARK-35695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35695: Assignee: Apache Spark > QueryExecutionListener does not see any observed metrics fired b

[jira] [Commented] (SPARK-35652) Different Behaviour join vs joinWith in self joining

[ https://issues.apache.org/jira/browse/SPARK-35652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360747#comment-17360747 ] Apache Spark commented on SPARK-35652: -- User 'dgd-contributor' has created a pull r

[jira] [Assigned] (SPARK-35652) Different Behaviour join vs joinWith in self joining

[ https://issues.apache.org/jira/browse/SPARK-35652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35652: Assignee: Apache Spark > Different Behaviour join vs joinWith in self joining > -

[jira] [Assigned] (SPARK-35652) Different Behaviour join vs joinWith in self joining

[ https://issues.apache.org/jira/browse/SPARK-35652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35652: Assignee: (was: Apache Spark) > Different Behaviour join vs joinWith in self joining

[jira] [Created] (SPARK-35711) Support casting of timestamp without time zone to timestamp type

Gengliang Wang created SPARK-35711: -- Summary: Support casting of timestamp without time zone to timestamp type Key: SPARK-35711 URL: https://issues.apache.org/jira/browse/SPARK-35711 Project: Spark

[jira] [Assigned] (SPARK-35711) Support casting of timestamp without time zone to timestamp type

[ https://issues.apache.org/jira/browse/SPARK-35711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35711: Assignee: Apache Spark (was: Gengliang Wang) > Support casting of timestamp without time

[jira] [Assigned] (SPARK-35711) Support casting of timestamp without time zone to timestamp type

[ https://issues.apache.org/jira/browse/SPARK-35711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35711: Assignee: Gengliang Wang (was: Apache Spark) > Support casting of timestamp without time

[jira] [Commented] (SPARK-35711) Support casting of timestamp without time zone to timestamp type

[ https://issues.apache.org/jira/browse/SPARK-35711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360758#comment-17360758 ] Apache Spark commented on SPARK-35711: -- User 'gengliangwang' has created a pull req

[jira] [Commented] (SPARK-35711) Support casting of timestamp without time zone to timestamp type

[ https://issues.apache.org/jira/browse/SPARK-35711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360759#comment-17360759 ] Apache Spark commented on SPARK-35711: -- User 'gengliangwang' has created a pull req

[jira] [Commented] (SPARK-35701) Contention on SQLConf.sqlConfEntries and SQLConf.staticConfKeys

[ https://issues.apache.org/jira/browse/SPARK-35701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360881#comment-17360881 ] Apache Spark commented on SPARK-35701: -- User 'haiyangsun-db' has created a pull req

[jira] [Assigned] (SPARK-35701) Contention on SQLConf.sqlConfEntries and SQLConf.staticConfKeys

[ https://issues.apache.org/jira/browse/SPARK-35701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35701: Assignee: Apache Spark (was: Haiyang Sun) > Contention on SQLConf.sqlConfEntries and SQL

[jira] [Assigned] (SPARK-35701) Contention on SQLConf.sqlConfEntries and SQLConf.staticConfKeys

[ https://issues.apache.org/jira/browse/SPARK-35701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35701: Assignee: Apache Spark (was: Haiyang Sun) > Contention on SQLConf.sqlConfEntries and SQL

[jira] [Assigned] (SPARK-35701) Contention on SQLConf.sqlConfEntries and SQLConf.staticConfKeys

[ https://issues.apache.org/jira/browse/SPARK-35701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35701: Assignee: Haiyang Sun (was: Apache Spark) > Contention on SQLConf.sqlConfEntries and SQL

[jira] [Created] (SPARK-35712) Simplify ResolveAggregateFunctions

Wenchen Fan created SPARK-35712: --- Summary: Simplify ResolveAggregateFunctions Key: SPARK-35712 URL: https://issues.apache.org/jira/browse/SPARK-35712 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-35712) Simplify ResolveAggregateFunctions

[ https://issues.apache.org/jira/browse/SPARK-35712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360892#comment-17360892 ] Apache Spark commented on SPARK-35712: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-35712) Simplify ResolveAggregateFunctions

[ https://issues.apache.org/jira/browse/SPARK-35712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35712: Assignee: (was: Apache Spark) > Simplify ResolveAggregateFunctions >

[jira] [Assigned] (SPARK-35712) Simplify ResolveAggregateFunctions

[ https://issues.apache.org/jira/browse/SPARK-35712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35712: Assignee: Apache Spark > Simplify ResolveAggregateFunctions > ---

[jira] [Created] (SPARK-35713) Bugfix for thread leak in JobCancellationSuite

Wan Kun created SPARK-35713: --- Summary: Bugfix for thread leak in JobCancellationSuite Key: SPARK-35713 URL: https://issues.apache.org/jira/browse/SPARK-35713 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-35713) Bugfix for thread leak in JobCancellationSuite

[ https://issues.apache.org/jira/browse/SPARK-35713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-35713: Description: When we call Thread.interrupt() method, that thread's interrupt status will be set but it ma

[jira] [Updated] (SPARK-35713) Bug fix for thread leak in JobCancellationSuite

[ https://issues.apache.org/jira/browse/SPARK-35713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-35713: Summary: Bug fix for thread leak in JobCancellationSuite (was: Bugfix for thread leak in JobCancellationS

[jira] [Created] (SPARK-35714) Bug fix for deadlock during the executor shutdown

Wan Kun created SPARK-35714: --- Summary: Bug fix for deadlock during the executor shutdown Key: SPARK-35714 URL: https://issues.apache.org/jira/browse/SPARK-35714 Project: Spark Issue Type: Improveme

[jira] [Updated] (SPARK-35714) Bug fix for deadlock during the executor shutdown

[ https://issues.apache.org/jira/browse/SPARK-35714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-35714: Description: When a executor received a TERM signal, it (the second TERM signal) will lock java.lang.Shut

[jira] [Commented] (SPARK-35713) Bug fix for thread leak in JobCancellationSuite

[ https://issues.apache.org/jira/browse/SPARK-35713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360976#comment-17360976 ] Apache Spark commented on SPARK-35713: -- User 'wankunde' has created a pull request

[jira] [Assigned] (SPARK-35714) Bug fix for deadlock during the executor shutdown

[ https://issues.apache.org/jira/browse/SPARK-35714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35714: Assignee: (was: Apache Spark) > Bug fix for deadlock during the executor shutdown > -

[jira] [Assigned] (SPARK-35713) Bug fix for thread leak in JobCancellationSuite

[ https://issues.apache.org/jira/browse/SPARK-35713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35713: Assignee: Apache Spark > Bug fix for thread leak in JobCancellationSuite > --

[jira] [Commented] (SPARK-35714) Bug fix for deadlock during the executor shutdown

[ https://issues.apache.org/jira/browse/SPARK-35714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360977#comment-17360977 ] Apache Spark commented on SPARK-35714: -- User 'wankunde' has created a pull request

[jira] [Assigned] (SPARK-35714) Bug fix for deadlock during the executor shutdown

[ https://issues.apache.org/jira/browse/SPARK-35714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35714: Assignee: Apache Spark > Bug fix for deadlock during the executor shutdown >

[jira] [Assigned] (SPARK-35713) Bug fix for thread leak in JobCancellationSuite

[ https://issues.apache.org/jira/browse/SPARK-35713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35713: Assignee: (was: Apache Spark) > Bug fix for thread leak in JobCancellationSuite > ---

[jira] [Commented] (SPARK-35714) Bug fix for deadlock during the executor shutdown

[ https://issues.apache.org/jira/browse/SPARK-35714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360981#comment-17360981 ] Apache Spark commented on SPARK-35714: -- User 'wankunde' has created a pull request

[jira] [Resolved] (SPARK-35711) Support casting of timestamp without time zone to timestamp type

[ https://issues.apache.org/jira/browse/SPARK-35711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35711. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32864 [https:

[jira] [Assigned] (SPARK-35653) [SQL] CatalystToExternalMap interpreted path fails for Map with case classes as keys or values

[ https://issues.apache.org/jira/browse/SPARK-35653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35653: --- Assignee: Emil Ejbyfeldt > [SQL] CatalystToExternalMap interpreted path fails for Map with

[jira] [Resolved] (SPARK-35653) [SQL] CatalystToExternalMap interpreted path fails for Map with case classes as keys or values

[ https://issues.apache.org/jira/browse/SPARK-35653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35653. - Fix Version/s: 3.0.3 3.1.3 3.2.0 Resolution: Fixed

[jira] [Created] (SPARK-35715) Option "--files" with local:// prefix is not honoured for Spark on kubernetes

Pardhu Madipalli created SPARK-35715: Summary: Option "--files" with local:// prefix is not honoured for Spark on kubernetes Key: SPARK-35715 URL: https://issues.apache.org/jira/browse/SPARK-35715

[jira] [Updated] (SPARK-35715) Option "--files" with local:// prefix is not honoured for Spark on kubernetes

[ https://issues.apache.org/jira/browse/SPARK-35715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pardhu Madipalli updated SPARK-35715: - Description: When we provide a local file as a dependency using "--files" option, the fi

[jira] [Updated] (SPARK-35715) Option "--files" with local:// prefix is not honoured for Spark on kubernetes

[ https://issues.apache.org/jira/browse/SPARK-35715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pardhu Madipalli updated SPARK-35715: - Description: When we provide a local file as a dependency using "--files" option, the fi

[jira] [Commented] (SPARK-35715) Option "--files" with local:// prefix is not honoured for Spark on kubernetes

[ https://issues.apache.org/jira/browse/SPARK-35715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361086#comment-17361086 ] Erik Krogen commented on SPARK-35715: - Not sure about k8s, but at least for YARN thi

[jira] [Commented] (SPARK-34861) Support nested column in Spark vectorized readers

[ https://issues.apache.org/jira/browse/SPARK-34861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361090#comment-17361090 ] Chao Sun commented on SPARK-34861: -- Synced with [~chengsu] offline and I will take over

[jira] [Resolved] (SPARK-35296) Dataset.observe fails with an assertion

[ https://issues.apache.org/jira/browse/SPARK-35296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35296. - Fix Version/s: 3.0.3 3.1.3 3.2.0 Resolution: Fixed

[jira] [Assigned] (SPARK-35296) Dataset.observe fails with an assertion

[ https://issues.apache.org/jira/browse/SPARK-35296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35296: --- Assignee: Kousuke Saruta > Dataset.observe fails with an assertion > --

[jira] [Created] (SPARK-35716) Support casting of timestamp without time zone to date type

Gengliang Wang created SPARK-35716: -- Summary: Support casting of timestamp without time zone to date type Key: SPARK-35716 URL: https://issues.apache.org/jira/browse/SPARK-35716 Project: Spark

[jira] [Commented] (SPARK-35716) Support casting of timestamp without time zone to date type

[ https://issues.apache.org/jira/browse/SPARK-35716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361116#comment-17361116 ] Apache Spark commented on SPARK-35716: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-35716) Support casting of timestamp without time zone to date type

[ https://issues.apache.org/jira/browse/SPARK-35716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35716: Assignee: Apache Spark (was: Gengliang Wang) > Support casting of timestamp without time

[jira] [Assigned] (SPARK-35716) Support casting of timestamp without time zone to date type

[ https://issues.apache.org/jira/browse/SPARK-35716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35716: Assignee: Gengliang Wang (was: Apache Spark) > Support casting of timestamp without time

[jira] [Commented] (SPARK-35716) Support casting of timestamp without time zone to date type

[ https://issues.apache.org/jira/browse/SPARK-35716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361117#comment-17361117 ] Apache Spark commented on SPARK-35716: -- User 'gengliangwang' has created a pull req

[jira] [Commented] (SPARK-32709) Write Hive ORC/Parquet bucketed table with hivehash (for Hive 1,2)

[ https://issues.apache.org/jira/browse/SPARK-32709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361119#comment-17361119 ] Shashank Pedamallu commented on SPARK-32709: [~chengsu], thank you so much f

[jira] [Assigned] (SPARK-32920) Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage

[ https://issues.apache.org/jira/browse/SPARK-32920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-32920: --- Assignee: Venkata krishnan Sowrirajan > Add support in Spark driver to coor

[jira] [Updated] (SPARK-32920) Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage

[ https://issues.apache.org/jira/browse/SPARK-32920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-32920: Fix Version/s: 3.2.0 > Add support in Spark driver to coordinate the finalization

[jira] [Resolved] (SPARK-32920) Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage

[ https://issues.apache.org/jira/browse/SPARK-32920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-32920. - Resolution: Fixed > Add support in Spark driver to coordinate the finalization o

[jira] [Commented] (SPARK-24579) SPIP: Standardize Optimized Data Exchange between Spark and DL/AI frameworks

[ https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361175#comment-17361175 ] binwei yang commented on SPARK-24579: - We implemented the native Apache Arrow data s

[jira] [Resolved] (SPARK-35716) Support casting of timestamp without time zone to date type

[ https://issues.apache.org/jira/browse/SPARK-35716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35716. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32869 [https://github.com

[jira] [Assigned] (SPARK-35692) Use int to replace long for EXECUTOR_ID_COUNTER in Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-35692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35692: - Assignee: Kent Yao > Use int to replace long for EXECUTOR_ID_COUNTER in Kubernetes > --

[jira] [Resolved] (SPARK-35692) Use int to replace long for EXECUTOR_ID_COUNTER in Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-35692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35692. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32837 [https://

[jira] [Assigned] (SPARK-33350) Add support to DiskBlockManager to create merge directory and to get the local shuffle merged data

[ https://issues.apache.org/jira/browse/SPARK-33350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-33350: --- Assignee: Ye Zhou > Add support to DiskBlockManager to create merge directo

[jira] [Resolved] (SPARK-33350) Add support to DiskBlockManager to create merge directory and to get the local shuffle merged data

[ https://issues.apache.org/jira/browse/SPARK-33350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-33350. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 320

[jira] [Commented] (SPARK-24295) Purge Structured streaming FileStreamSinkLog metadata compact file data.

[ https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361245#comment-17361245 ] Parsa Shabani commented on SPARK-24295: --- Hello all, Any updates on this? [~kabhwa

[jira] [Comment Edited] (SPARK-24295) Purge Structured streaming FileStreamSinkLog metadata compact file data.

[ https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361245#comment-17361245 ] Parsa Shabani edited comment on SPARK-24295 at 6/10/21, 10:43 PM:

[jira] [Assigned] (SPARK-35593) Support shuffle data recovery on the reused PVCs

[ https://issues.apache.org/jira/browse/SPARK-35593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35593: - Assignee: Dongjoon Hyun > Support shuffle data recovery on the reused PVCs > --

[jira] [Resolved] (SPARK-35593) Support shuffle data recovery on the reused PVCs

[ https://issues.apache.org/jira/browse/SPARK-35593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35593. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32730 [https://

[jira] [Updated] (SPARK-35616) Make astype data-type-based

[ https://issues.apache.org/jira/browse/SPARK-35616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35616: - Description: There are many type checks in `astype` methods. Since `DataTypeOps` is introduced,

[jira] [Commented] (SPARK-35439) Children subexpr should come first than parent subexpr in subexpression elimination

[ https://issues.apache.org/jira/browse/SPARK-35439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361297#comment-17361297 ] Apache Spark commented on SPARK-35439: -- User 'viirya' has created a pull request fo

[jira] [Created] (SPARK-35717) pandas_udf crashes in conjunction with .filter()

F. H. created SPARK-35717: - Summary: pandas_udf crashes in conjunction with .filter() Key: SPARK-35717 URL: https://issues.apache.org/jira/browse/SPARK-35717 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-35439) Children subexpr should come first than parent subexpr in subexpression elimination

[ https://issues.apache.org/jira/browse/SPARK-35439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361298#comment-17361298 ] Apache Spark commented on SPARK-35439: -- User 'viirya' has created a pull request fo

[jira] [Resolved] (SPARK-35709) Remove links to Nomad integration in the Documentation

[ https://issues.apache.org/jira/browse/SPARK-35709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-35709. -- Fix Version/s: 3.2.0 Assignee: Paweł Ptaszyński Resolution: Fixed Reso

[jira] [Assigned] (SPARK-35475) Enable disallow_untyped_defs mypy check for pyspark.pandas.namespace.

[ https://issues.apache.org/jira/browse/SPARK-35475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35475: Assignee: (was: Apache Spark) > Enable disallow_untyped_defs mypy check for pyspark.p

[jira] [Assigned] (SPARK-35475) Enable disallow_untyped_defs mypy check for pyspark.pandas.namespace.

[ https://issues.apache.org/jira/browse/SPARK-35475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35475: Assignee: Apache Spark > Enable disallow_untyped_defs mypy check for pyspark.pandas.names

[jira] [Commented] (SPARK-35475) Enable disallow_untyped_defs mypy check for pyspark.pandas.namespace.

[ https://issues.apache.org/jira/browse/SPARK-35475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361309#comment-17361309 ] Apache Spark commented on SPARK-35475: -- User 'ueshin' has created a pull request fo

[jira] [Commented] (SPARK-35475) Enable disallow_untyped_defs mypy check for pyspark.pandas.namespace.

[ https://issues.apache.org/jira/browse/SPARK-35475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361311#comment-17361311 ] Apache Spark commented on SPARK-35475: -- User 'ueshin' has created a pull request fo

  1   2   >