Timothy Brown created HUDI-9612:
-----------------------------------
Summary: Improve schema resolver performance by skipping
delete-only files
Key: HUDI-9612
URL: https://issues.apache.org/jira/browse/HUDI-9612
Project: Apache Hudi
Issue Type: Improvement
Reporter: Timothy Brown
The TableSchemaResolver can have varied performance depending on the last
commit to the table. The code will currently try to inspect log files with only
deletes for example which will not yield a schema and then we must inspect
another file. In the worst case, you can iterate through all the files and
still not find a valid schema.
This can be updated to first ensure the commit has a non-zero number of inserts
or updates, then inspect files that have non-zero inserts or updates only.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
