codope commented on issue #7643: URL: https://github.com/apache/hudi/issues/7643#issuecomment-1403217907
@BruceKellan Sorry, i was away on a break and didn't get a chance to look into this. First of all, I don't think it's a regression as the query is slow even with hudi connector of Trino version 400 using Hudi 0.11.1. There was a regression in hive connector due to a change in hudi code and we have fixed that in [master](https://github.com/apache/hudi/commit/a882f440d37b4adb0ff194dad579c11dc44bbc78). Now, in my setup of hudi connector, I found that the query is slow because there is single split manager thread doing all the listing. It's also evident in your setup (`hudi-split-manager-0`). This is quite inefficient. I need to improve this, do more like how hive connector's background split loader works. This is a change in Trino codebase and not Hudi. I will work on it next week. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
