vinothchandar commented on issue #1731: URL: https://github.com/apache/hudi/issues/1731#issuecomment-643471123
@tooptoop4 > including time to trigger spark-submit, unless there is faster way if you are targetting something like spark-submit, then its not under hudi's control. spark-submit alone can take anywhere from 30 seconds to minutes depending on various factors on your cluster, size of the jar you are uploading etc.. on Hudi upsert it self, using Hudi to upsert 1 row at a time, is not an intended design. I recommend atleast batching input data for a minute or so, and issue upsert as a batch. In other words, Hudi (or anything like Hudi) is not really designed like MySQL.. Hope that helps.. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
