[
https://issues.apache.org/jira/browse/HUDI-8037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Y Ethan Guo updated HUDI-8037:
------------------------------
Sprint: Hudi 1.0 Blockers+Bugs Sprint
> Partition query for transformed value incorrectly prunes valid partitions
> -------------------------------------------------------------------------
>
> Key: HUDI-8037
> URL: https://issues.apache.org/jira/browse/HUDI-8037
> Project: Apache Hudi
> Issue Type: Bug
> Components: spark, spark-sql
> Reporter: Jonathan Vexler
> Assignee: Jonathan Vexler
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.1.0, 0.15.1
>
>
> With timestamp keygen you can have a partition column with timestamps, but
> then use the keygen so it will create partitions based on days so that all
> records that have a timestamp on 7-31-2024 will go to the same parititon even
> though the values in the partition column differ by hours and minutes etc.
> This causes a problem with partition pruning. lets say you query "select *
> from table where partition < 7-31-2024 at 7am and partition > 7-31-2024 at
> 6am ". Since the file structure has the partition of just 7-31-2024, that
> will be interpreted as 7-31-2024 at 12am. So the partition will be pruned
> from the search space.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
