William Lo created GOBBLIN-1959:
-----------------------------------
Summary: Introduce Gobblin partial success semantic which fails
job on partial success
Key: GOBBLIN-1959
URL: https://issues.apache.org/jira/browse/GOBBLIN-1959
Project: Apache Gobblin
Issue Type: Improvement
Components: gobblin-core
Reporter: William Lo
Assignee: Abhishek Tiwari
Currently, the partial success semantics in Gobblin where it follows a commit
on task success policy will return a successful job running status even though
some tasks fail. This is because in most Gobblin jobs, there would be movement
of the watermark after the tasks succeed so we want to follow that semantic to
not lead to duplicate ingestion.
However, there are some Gobblin jobs (distcp) which do not follow the watermark
semantic, thus a partial success should return a failed status so that the job
callers can understand that not every task has succeeded, thus invoke a retry
with amended amounts of work.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
