hudi-bot opened a new issue, #14945: URL: https://github.com/apache/hudi/issues/14945
Hi, community I currently try to use spark-hudi integration in spark-thrift-server, and after test create hudi table for a while, I found it would finally result in META-SPACE OOM(in my case, jvm option -XX:MaxMetaspaceSize=256m assigned). After track the source, I found that every time a CreateHoodieTableCommand performed, `HiveClientUtils.newClientForMetadata` will be invoked, thus a IsolatedClientLoader will be created, in my scene, the OOM will occured after about 10 create statement executed. Why not use `sessionState.catalog.externalCatalog.asInstanceOf[ExternalCatalogWithListener].unwrapped.asInstanceOf[HiveExternalCatalog].client ` instead ? Does it has anything side effect? env: hudi master/spark-3.1.2/hive-2.3.6 Thanks. ## JIRA info - Link: https://issues.apache.org/jira/browse/HUDI-2960 - Type: Bug - Affects version(s): - 0.10.0 --- ## Comments 31/Jan/22 18:13;shivnarayan;CC [[email protected]] [~rxu] ;;; -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
