[jira] [Resolved] (SPARK-35641) TPCDS Q32 running failed

[ https://issues.apache.org/jira/browse/SPARK-35641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35641. -- Resolution: Invalid > TPCDS Q32 running failed > > >

[jira] [Commented] (SPARK-35676) pyspark.sql.functions GroupBy agg CountDistinct() return bad value

[ https://issues.apache.org/jira/browse/SPARK-35676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362700#comment-17362700 ] Hyukjin Kwon commented on SPARK-35676: -- {{distinc().count()}} includes nulls but {{countDistinct}}

[jira] [Commented] (SPARK-35688) GeneratePredicate eliminate will fail in some case

[ https://issues.apache.org/jira/browse/SPARK-35688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362699#comment-17362699 ] Hyukjin Kwon commented on SPARK-35688: -- [~cjuexuan] would you mind providing fully self-contained

[jira] [Commented] (SPARK-35700) spark.sql.orc.filterPushdown not working with Spark 3.1.1 for tables with varchar data type

[ https://issues.apache.org/jira/browse/SPARK-35700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362698#comment-17362698 ] Hyukjin Kwon commented on SPARK-35700: -- cc [~Qin Yao] [~cloud_fan] [~dongjoon] FYI >

[jira] [Commented] (SPARK-35717) pandas_udf crashes in conjunction with .filter()

[ https://issues.apache.org/jira/browse/SPARK-35717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362696#comment-17362696 ] Hyukjin Kwon commented on SPARK-35717: -- [~hoeze] I would like to try reproducing this one. Would

[jira] [Commented] (SPARK-35739) [Spark Sql] Add Java-comptable Dataset.join overloads

[ https://issues.apache.org/jira/browse/SPARK-35739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362694#comment-17362694 ] Hyukjin Kwon commented on SPARK-35739: -- Can you use Array instead of list? otherwise I think it's

[jira] [Commented] (SPARK-35739) [Spark Sql] Add Java-comptable Dataset.join overloads

[ https://issues.apache.org/jira/browse/SPARK-35739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362695#comment-17362695 ] Hyukjin Kwon commented on SPARK-35739: -- Feel free to go ahead with a PR if you find some time

[jira] [Commented] (SPARK-35744) Performance degradation in avro SpecificRecordBuilders

[ https://issues.apache.org/jira/browse/SPARK-35744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362693#comment-17362693 ] Hyukjin Kwon commented on SPARK-35744: -- cc [~Gengliang.Wang] FYI > Performance degradation in avro

[jira] [Resolved] (SPARK-35745) Serie to Scalar pandas_udf in GroupedData.agg() breaks the following monotonically_increasing_id()

[ https://issues.apache.org/jira/browse/SPARK-35745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35745. -- Resolution: Not A Problem > Serie to Scalar pandas_udf in GroupedData.agg() breaks the

[jira] [Commented] (SPARK-35745) Serie to Scalar pandas_udf in GroupedData.agg() breaks the following monotonically_increasing_id()

[ https://issues.apache.org/jira/browse/SPARK-35745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362692#comment-17362692 ] Hyukjin Kwon commented on SPARK-35745: -- {quote} The problem is avoided by using

[jira] [Updated] (SPARK-27714) Support Join Reorder based on Genetic Algorithm when the # of joined tables > 12

[ https://issues.apache.org/jira/browse/SPARK-27714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27714: Target Version/s: (was: 3.0.0) > Support Join Reorder based on Genetic Algorithm when the # of

[jira] [Created] (SPARK-35750) Rename "pandas APIs on Spark" to "pandas API on Spark" in the documents

Haejoon Lee created SPARK-35750: --- Summary: Rename "pandas APIs on Spark" to "pandas API on Spark" in the documents Key: SPARK-35750 URL: https://issues.apache.org/jira/browse/SPARK-35750 Project: Spark

[jira] [Updated] (SPARK-35591) Rename "Koalas" to "pandas APIs on Spark" in the documents

[ https://issues.apache.org/jira/browse/SPARK-35591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35591: Summary: Rename "Koalas" to "pandas APIs on Spark" in the documents (was: Rename "Koalas" to

[jira] [Commented] (SPARK-35622) DataFrame's count function do not need groupBy and avoid shuffle

[ https://issues.apache.org/jira/browse/SPARK-35622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362683#comment-17362683 ] Hyukjin Kwon commented on SPARK-35622: -- IIRC, it already works same or similarly with RDD's count.

[jira] [Commented] (SPARK-35622) DataFrame's count function do not need groupBy and avoid shuffle

[ https://issues.apache.org/jira/browse/SPARK-35622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362681#comment-17362681 ] dgd_contributor commented on SPARK-35622: - hi, could you explain more of this issue? where

[jira] [Commented] (SPARK-35652) Different Behaviour join vs joinWith in self joining

[ https://issues.apache.org/jira/browse/SPARK-35652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362679#comment-17362679 ] Apache Spark commented on SPARK-35652: -- User 'dgd-contributor' has created a pull request for this

[jira] [Resolved] (SPARK-35736) Parse any day-time interval types in SQL

[ https://issues.apache.org/jira/browse/SPARK-35736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35736. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32893

[jira] [Assigned] (SPARK-35736) Parse any day-time interval types in SQL

[ https://issues.apache.org/jira/browse/SPARK-35736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-35736: Assignee: Kousuke Saruta > Parse any day-time interval types in SQL >

[jira] [Assigned] (SPARK-35714) Bug fix for deadlock during the executor shutdown

[ https://issues.apache.org/jira/browse/SPARK-35714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-35714: Assignee: Wan Kun > Bug fix for deadlock during the executor shutdown >

[jira] [Resolved] (SPARK-35714) Bug fix for deadlock during the executor shutdown

[ https://issues.apache.org/jira/browse/SPARK-35714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-35714. -- Fix Version/s: 3.0.3 3.1.3 3.2.0 Resolution:

[jira] [Created] (SPARK-35749) Parse single unit field interval literals as day-time interval types

Kousuke Saruta created SPARK-35749: -- Summary: Parse single unit field interval literals as day-time interval types Key: SPARK-35749 URL: https://issues.apache.org/jira/browse/SPARK-35749 Project:

[jira] [Commented] (SPARK-35720) Support casting of String to timestamp without time zone type

[ https://issues.apache.org/jira/browse/SPARK-35720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362591#comment-17362591 ] Apache Spark commented on SPARK-35720: -- User 'pingsutw' has created a pull request for this issue:

[jira] [Commented] (SPARK-35720) Support casting of String to timestamp without time zone type

[ https://issues.apache.org/jira/browse/SPARK-35720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362590#comment-17362590 ] Apache Spark commented on SPARK-35720: -- User 'pingsutw' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35720) Support casting of String to timestamp without time zone type

[ https://issues.apache.org/jira/browse/SPARK-35720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35720: Assignee: Apache Spark > Support casting of String to timestamp without time zone type >

[jira] [Assigned] (SPARK-35720) Support casting of String to timestamp without time zone type

[ https://issues.apache.org/jira/browse/SPARK-35720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35720: Assignee: (was: Apache Spark) > Support casting of String to timestamp without time

[jira] [Resolved] (SPARK-35719) Support type conversion between timestamp and timestamp without time zone type

[ https://issues.apache.org/jira/browse/SPARK-35719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35719. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32878

[jira] [Updated] (SPARK-35700) spark.sql.orc.filterPushdown not working with Spark 3.1.1 for tables with varchar data type

[ https://issues.apache.org/jira/browse/SPARK-35700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arghya Saha updated SPARK-35700: Description: We are not able to upgrade to Spark 3.1.1 from Spark 2.4.x as the join on varchar

[jira] [Commented] (SPARK-35415) Change information to map type for SHOW TABLE EXTENDED command

[ https://issues.apache.org/jira/browse/SPARK-35415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362536#comment-17362536 ] Apache Spark commented on SPARK-35415: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-35748) Fix StreamingJoinHelper to be able to handle day-time interval

[ https://issues.apache.org/jira/browse/SPARK-35748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35748: Assignee: Apache Spark (was: Kousuke Saruta) > Fix StreamingJoinHelper to be able to

[jira] [Commented] (SPARK-35748) Fix StreamingJoinHelper to be able to handle day-time interval

[ https://issues.apache.org/jira/browse/SPARK-35748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362524#comment-17362524 ] Apache Spark commented on SPARK-35748: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-35748) Fix StreamingJoinHelper to be able to handle day-time interval

[ https://issues.apache.org/jira/browse/SPARK-35748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362523#comment-17362523 ] Apache Spark commented on SPARK-35748: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35748) Fix StreamingJoinHelper to be able to handle day-time interval

[ https://issues.apache.org/jira/browse/SPARK-35748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35748: Assignee: Kousuke Saruta (was: Apache Spark) > Fix StreamingJoinHelper to be able to

[jira] [Assigned] (SPARK-35748) Fix StreamingJoinHelper to be able to handle day-time interval

[ https://issues.apache.org/jira/browse/SPARK-35748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35748: Assignee: Apache Spark (was: Kousuke Saruta) > Fix StreamingJoinHelper to be able to

[jira] [Created] (SPARK-35748) Fix StreamingJoinHelper to be able to handle day-time interval

Kousuke Saruta created SPARK-35748: -- Summary: Fix StreamingJoinHelper to be able to handle day-time interval Key: SPARK-35748 URL: https://issues.apache.org/jira/browse/SPARK-35748 Project: Spark

[jira] [Assigned] (SPARK-35658) Document Parquet encryption feature in Spark

[ https://issues.apache.org/jira/browse/SPARK-35658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35658: Assignee: Apache Spark > Document Parquet encryption feature in Spark >

[jira] [Assigned] (SPARK-35658) Document Parquet encryption feature in Spark

[ https://issues.apache.org/jira/browse/SPARK-35658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35658: Assignee: (was: Apache Spark) > Document Parquet encryption feature in Spark >

[jira] [Commented] (SPARK-35658) Document Parquet encryption feature in Spark

[ https://issues.apache.org/jira/browse/SPARK-35658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362481#comment-17362481 ] Apache Spark commented on SPARK-35658: -- User 'ggershinsky' has created a pull request for this