[GitHub] [hudi] BruceKellan commented on issue #7643: [SUPPORT] Too slow while using trino-hudi connector while querying partitioned tables.

Thu, 12 Jan 2023 04:42:59 -0800 


BruceKellan commented on issue #7643:
URL: https://github.com/apache/hudi/issues/7643#issuecomment-1380282920

   > @BruceKellan we addressed performance regression in 0.12.2. Can you give 
it a try?
   
   <img width="502" alt="image" 
src="https://user-images.githubusercontent.com/13477122/212068968-d535849a-8ad0-4034-8769-c0decba4c293.png";>
   
   I upgrade the version of hudi in trino-hudi to 0.12.2 and recompiled, but 
it's also too slow. We try to locate the problem by enabling debug level 
logging and get some information.
   
   While running trino sql to query:
   ```sql
   select count(1) from hudi.website.hudi_reproduce where day between 20230101 
and 20230104 and type between 1 and 9;
   ```
   
   The hudi connector also get all paritions from hive metastore one by one. 
Maybe this is the reason.
   <img width="1534" alt="image" 
src="https://user-images.githubusercontent.com/13477122/212069245-c4c3903c-b85c-4174-acd1-1f0bdab2c977.png";>
   [server.log](https://github.com/apache/hudi/files/10401595/server.log)
   @alexeykudinkin @codope WDYT?
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to