rangareddy commented on issue #5980: URL: https://github.com/apache/hudi/issues/5980#issuecomment-5103004999
This issue was reviewed as part of the JIRA-migrated backlog triage. Findings: This is the same 0.8.x → 0.10.x upsert regression family as #4682, and like that one it was never bisected to a commit, so there is no root cause on record. Two things make it unactionable as filed. The re-benchmark on 0.12.1 that was agreed on 2022-10-31 never happened -- a follow-up on 2023-03-31 also went unanswered -- so there is no measurement against a version anyone still maintains. And as noted in the original report, upgrading past 0.10.1 was not possible for you; 0.10.x has long been end-of-life and Hudi does not backport performance work to it, so a fix could not have reached you there in any case. For anyone landing here from a search: #4682 established that a stale or incorrect `hoodie.copyonwrite.record.size.estimate` combined with a timestamp-based partition scheme was enough to reproduce this shape of slowdown, and that removing that config restored the earlier performance. That is worth checking before assuming a version regression. Closing as stale. If you do re-benchmark on a current release (1.0.x / 1.1.x / 1.2.0) and still see a regression, please open a fresh issue with both sets of timings, the write configs, and the Spark UI stage breakdown. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
