[
https://issues.apache.org/jira/browse/HUDI-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen closed HUDI-9612.
----------------------------
Reviewers: Danny Chen
Resolution: Fixed
Fixed via master branch: eb8e829b876d1e670d5651bd6f258ddced6b6df1
> Improve schema resolver performance by skipping delete-only files
> -----------------------------------------------------------------
>
> Key: HUDI-9612
> URL: https://issues.apache.org/jira/browse/HUDI-9612
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: Timothy Brown
> Assignee: Timothy Brown
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> The TableSchemaResolver can have varied performance depending on the last
> commit to the table. The code will currently try to inspect log files with
> only deletes for example which will not yield a schema and then we must
> inspect another file. In the worst case, you can iterate through all the
> files and still not find a valid schema.
> This can be updated to first ensure the commit has a non-zero number of
> inserts or updates, then inspect files that have non-zero inserts or updates
> only.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
