simingweng opened a new issue #3211: add command line arg to CheckpointManager to take overridden config URL: https://github.com/apache/incubator-heron/issues/3211 Currently, there is no command line argument that CheckpointManager supports to take in the `heron-conf/override.yaml`, which holds all the configuration property that user specifies, for example, via `-D` property argument to the Heron API server deployment. This means that users who deploy Heron on Kubernetes will be locked to `org.apache.heron.statefulstorage.localfs.LocalFileSystemStorage` as the storage backend of any stateful topology. But, local filesystem is ephemeral in Docker container, therefore it defeats the purpose of a stateful topology, since all the states get lost when the topology gets re-submitted and its pods get recreated. User can work around this issue by creating their own Heron docker image and override the configuration file, but that is static approach, which won't work well because certain configuration varies from deployment to deployment, and is unknown until deploy time, such as the DistributedLog namespace including zookeeper service/url. So, the better solution is to make CheckpointManager aware of the `override.yaml` via CLI argument, and Executor can specify it when it starts the CheckpointManager process in each Heron Container Pod.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
