bkosuru opened a new issue, #5983:
URL: https://github.com/apache/hudi/issues/5983
Partition pruning does not seem to be working for Incremental Query.
Data Size 1TB
Table type COW
Data in HDFS
Regular query : 8.4 mins
Incremental Query: 90 mins for the same data
Noticed this in 0.10.1. Not sure if this happens in older versions
I set the following option on the reader.
.option("hoodie.file.index.enable", false)
The writer config we use is here - https://github.com/apache/hudi/issues/5741
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
