Cameron Lee created SAMZA-2684:
----------------------------------
Summary: Revisit config rewriting flows in Samza
Key: SAMZA-2684
URL: https://issues.apache.org/jira/browse/SAMZA-2684
Project: Samza
Issue Type: Improvement
Reporter: Cameron Lee
I noticed that there are multiple stages that a "config" can be in through the
Samza flow.
# Submission config passed to job coordinator
# "Original" config which is loaded by ConfigLoader
# Rewritten config which is the "original" config with rewriters applied
# Job model config which is calculated when job model is determined
It is not good to have so many different stages of config. In the code, we use
the class Config, but it isn't clear which stage a Config corresponds to.
Ideally, we should be passing the Stage 4 config around as much as possible,
but that isn't the case in some code (e.g. ClusterBasedJobCoordinator creates
monitors with Stage 3 config).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
