Hey Guys, We've been discussing a portion of Samza's configuration system over here:
https://issues.apache.org/jira/browse/SAMZA-348 Since the proposed change is large, I'm looking for any feedback folks in the broader community might have. Samza's config leaves quite a bit to be desired. Our config system is used in three places: * The client (run-job.sh, JobRunner). * The framework (SamzaAppMaster, SamzaContainer). * The task (StreamTask). This design proposal focuses on how configuration is resolved and passed around within the framework. No attempt is made to change the Config object, or how the StreamTask API sees the configuration. The goal of the proposal is to: 1. Make it possible to support dynamic configuration changes without restarting a Samza job. 2. Dis-entangle generic job coordination logic from the YARN AM, so the generic logic can be used for other job schedulers (e.g. Mesos). 3. Unify job configuration, offset management, and job control (start, stop, restart) into a single abstraction. 4. Allow Samza jobs to have a large number of configuration key/value pairs. Feedback welcome. Cheers, Chris
