Hey David, Yes, the config is rewritten by the JobRunner each time the job is restarted as dictated by the "resetJobConfig" property (which defaults to true) here: https://github.com/apache/samza/blob/944dd02e1d00bcce59f1fcc33ecbb2a8acd95870/samza-core/src/main/scala/org/apache/samza/job/JobRunner.scala#L85
The version parameter is not the version of the config, but the version of the coordinator stream protocol. It will only increment if there's a change to the way Samza reads/writes coordinator stream messages. -Jake On Thu, Aug 25, 2016 at 8:44 AM, David Yu <david...@optimizely.com> wrote: > After digging around a bit using kafka-console-consumer.sh, I'm able to > peek into the coordinator stream and see the config entries. > > Looks like a redeployment would just append the new configs to this topic. > And I assume the bootstrapping process will have to consume the entire > stream from the beginning to get a consolidated/up-to-date job model. Am I > understanding this correctly? > > Also, according to the doc, the keys of the coordinator stream is > ["<version-number>", "<message-type>", "<key>"]. From what I saw, the > <version-number> doesn't seem to change. It stays at 1 even after I > redeployed the job with new configs. I was assuming that it would get > incremented with every deployment/update. > > Thanks, > David > > On Wed, Aug 24, 2016 at 5:39 PM David Yu <david...@optimizely.com> wrote: > > > Hi, > > > > I'm trying to understand role of the coordinator stream during a job > > redeployment. > > > > From the Samza documentation, I'm seeing the following about the > > coordinator stream: > > > > The Job Coordinator bootstraps configuration from the coordinator stream > > each time upon job start-up. It periodically catches up with any new data > > written to the coordinator stream and updates the Job Model. > > > > However, it is unclear to me how this will work with "--config-path" when > > we need to redeploy a job. Does the provided config first gets persisted > to > > the coordinator stream, updating the previous model, or will it simply be > > ignored? > > > > Thanks, > > David > > >
