BALAJI VARADARAJAN created HUDI-269:
---------------------------------------
Summary: Provide ability to throttle DeltaStreamer sync runs
Key: HUDI-269
URL: https://issues.apache.org/jira/browse/HUDI-269
Project: Apache Hudi (incubating)
Issue Type: Improvement
Components: deltastreamer
Reporter: BALAJI VARADARAJAN
Copied from [https://github.com/apache/incubator-hudi/issues/922]
In some scenario in our cluster, we may want delta streamer to slow down a bit.
so it's nice to have a parameter to control the min sync interval of each sync
in continuous mode.
this param is default to 0, so this should not affect current logic.
minor pr: [#921|https://github.com/apache/incubator-hudi/pull/921]
the main reason we want to slow it down is that aws s3 is charged by s3
get/put/list requests. we don't want to pay for too many requests for a really
slow change table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
