Aditya Goenka created HUDI-6739:
-----------------------------------
Summary: Avoid checking timeline for successful commits for spark
structured streaming when offset is 0
Key: HUDI-6739
URL: https://issues.apache.org/jira/browse/HUDI-6739
Project: Apache Hudi
Issue Type: Improvement
Components: spark
Reporter: Aditya Goenka
Fix For: 1.0.0
For Spark Structured Streaming writer, for new writer we are checking the
active timeline to identify the last successful commit and fix idempotency.
No need to check the checkpoint when the offset is 0 as that will mean it will
be the new writer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
