kazdy commented on issue #7778: URL: https://github.com/apache/hudi/issues/7778#issuecomment-1418741003
Hi @nsivabalan thanks for your hints, I'll take these into consideration. Regarding this one: > But in streaming read, the amount of data read might spike up(if we plan to do the same) and the user may not have provisioned higher resources for the job. I think we can support rate limiting in streaming reads, Spark has an interface for it so I can implement it. This will be something similar to what we already have in deltastreamer (limiting on no of instants per microbatch). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
