[jira] [Commented] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

2023-03-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17706665#comment-17706665 ] Yuming Wang commented on SPARK-40610: - Users can {{set

[jira] [Commented] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

2023-03-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696716#comment-17696716 ] Apache Spark commented on SPARK-40610: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

2023-03-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696715#comment-17696715 ] Apache Spark commented on SPARK-40610: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

2023-03-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696634#comment-17696634 ] Yuming Wang commented on SPARK-40610: - This is because of

[jira] [Commented] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

2023-03-01 Thread icyjhl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17694993#comment-17694993 ] icyjhl commented on SPARK-40610: Hi [~yumwang], it's string. > Spark fall back to use getPartitions

[jira] [Commented] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

2023-02-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691387#comment-17691387 ] Yuming Wang commented on SPARK-40610: - [~icyjhl] What's your dt data type? date, string or

[jira] [Commented] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

2022-10-07 Thread icyjhl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17614323#comment-17614323 ] icyjhl commented on SPARK-40610: Hi [~Zing], this works fine in spark2 and spark 3.0.1, anything we

[jira] [Commented] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

2022-10-03 Thread zzzzming95 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17612241#comment-17612241 ] ming95 commented on SPARK-40610: [~icyjhl]    If `dt` is string type, you can cast

[jira] [Commented] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

2022-10-02 Thread zzzzming95 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17612135#comment-17612135 ] ming95 commented on SPARK-40610: Spark will perform partition pruning when executing the sql of