PhantomHunt commented on issue #9344: URL: https://github.com/apache/hudi/issues/9344#issuecomment-1663332820
We have a Job running on EC2 ubuntu machine that upserts data into 2 hudi tables parallelly in 2 threads (using threadPoolExecutor in the concurrent library of Python) at a time. There are 17 tables in total. When upsertion in any one of the tables is finished, threadPoolExecutor takes in another table to process in the available free thread. The Job terminates when upsertion in all 17 tables finishes. This job runs every 5 mins via cronjob. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
