Lokesh Lingarajan created HUDI-6629:
---------------------------------------
Summary: Changes for s3/gcs IncrSource job to taken into
sourceLimit during ingestion
Key: HUDI-6629
URL: https://issues.apache.org/jira/browse/HUDI-6629
Project: Apache Hudi
Issue Type: Improvement
Reporter: Lokesh Lingarajan
Current S3/GCS incremental jobs do not take into account sourceLimit and this
causes oom issues in the production. We need to enhance the implementation of
these jobs to slice the dataset based on sourceLimit specified in bytes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
