Alex Loddengaard wrote:
Just to be clear, you want to persist a configuration change to your entire
cluster without bringing it down, and you're hoping to use the Configuration
API to do so. Did I get your question correct?
I don't know of a way to do this without restarting the cluster, because I'm
pretty sure Configuration changes will only affect the current job. Does
anyone else have a suggestion?
Alex
Its not doable with the current architecture
-confs never get reread
-they aren't persisted
If you did persist them, it would get into a mess if every node in the
cluster was reading the same shared file from an NFS mount; the last
node to start up and set a value would set it for everything else.
Better to have a persistent central configuration management ("CM")
infrastructure and drive hadoop that way.
Even with CM tooling, the idea of persisting all changes back from the
apps is generally considered a bad thing. What is preferred is to have a
gui for managing system state that you can view, rollback, compare
configuration changes too, as having the app do it means you don't know
what's going on -and your app can leave your system in a mess, which is
exactly what CM tools try to prevent.
-steve