Thanks Gour! Your step-by-step explanation on how to update Slider cluster is very helpful. Now I understand what it takes to update a Slider cluster. Sorry that I missed the "upgrade" doc earlier.
Just a recap of what we discussed so far: * With multiple components in a slider cluster, "Flex" is to up/down scale number of instances of one component. It won't load new configuration or binary for unaffected component or instances. * To add or remove a component for a running cluster, "rolling upgrade" in below fashion: ** create local app zip file with updated metainfo.xml, update local appConfig.json ** update hdfs package with command: slider install-package --replacepkg --package myapp.zip --name mycluster ** update slider AM : slider upgrade mycluster -template appConfig.json --resources resources.json --queue my_queue ** wait for slider AM to add or remove the component to or from mycluster Is this correct? Thanks, Paul On Sun, Aug 9, 2015 at 2:30 PM, Gour Saha <[email protected]> wrote: > Paul, hopefully these will answer your questions. > > Question 1. > Steve is right. There is no better way to do this yet. However if you care > about persistent data created in a cluster, then you do not need to > destroy and recreate it. You can just stop, use the following 2 commands > and start it back up. > > slider package (using --replacepkg) - to update the metainfo > slider update - to update the appconfig and resources > (Note if you do not intend to update one of them, you still need to > provide the current form of that file. Otherwise I think it resets it to > empty, just try it out on a test cluster) > > Question 2. > As Steve mentioned, rolling upgrade is the solution to your dynamic > add/remove component req. "Upgrading App Packages" doc available under > "Using" menu, should help you get everything you need - > http://slider.incubator.apache.org/docs/slider_specs/application_pkg_upgrad > e.html > > > Let us know if you have further questions. > > -Gour > > > On 8/7/15, 4:44 PM, "Paul Han" <[email protected]> wrote: > > >Thanks Steve for the quick response! > > > >The "live update" feature is exciting. Combing that with "rolling update" > >could really give Slider cluster the stability needed in production-grade > >env. > > > >Gour, any update on this? I didn't see any doc published yet. Wonder if we > >can get a sneak preview? > > > >Thanks, > >Paul > > > > > > > > > >On Wed, Aug 5, 2015 at 1:41 PM, Steve Loughran <[email protected]> > >wrote: > > > >> > >> > On 5 Aug 2015, at 10:53, Paul Han <[email protected]> wrote: > >> > > >> > Hi, Slider Team > >> > > >> > We are using Slider to deploy our multi-tenanted services to Yarn. For > >> each > >> > tenant, I add a new component to Slider cluster. The services are > >> identical > >> > except for the tenant it serve for, which is passed to the component > >>as a > >> > string parameter. > >> > > >> > the appConfig file looks like something below: > >> > { > >> > "global": { > >> > "tenantName": "TO_BE_OVER_WRITTEN_BY_COMPONENT", > >> > }, > >> > "components": { > >> > "comp_a": { > >> > "tenantName": "tenant_a", > >> > }, > >> > "comp_b": { > >> > "tenantName": "tenant_b", > >> > } > >> > } > >> > } > >> > > >> > metainfo.xml > >> > ... > >> > <components> > >> > > >> > <component> > >> > <name>comp_a</name> > >> > <category>MASTER</category> > >> > <compExports>Servers-host_port</compExports> > >> > <commandScript> > >> > <script>scripts/foo-service.py</script> > >> > <scriptType>PYTHON</scriptType> > >> > </commandScript> > >> > </component> > >> > > >> > <component> > >> > <name>comp_b</name> > >> > <category>MASTER</category> > >> > <compExports>Servers-host_port</compExports> > >> > <commandScript> > >> > <script>scripts/foo-service.py</script> > >> > <scriptType>PYTHON</scriptType> > >> > </commandScript> > >> > </component> > >> > > >> > </components> > >> > ... > >> > > >> > I have two questions: > >> > 1. Is this the best I can do? > >> > you can see that to deploy identical services(binary, command scripts > >>are > >> > all same) for multiple tenants, I need to modify metainfo.xml and > >> > appConfig.json to have dup entries with just a param difference. > >>Slider > >> > cluster has to be destroy and recreated. > >> > > >> > >> right now, yes. > >> > >> > 2. Once this is working, to add a new tenant to the cluster, I have to > >> > modify the those files again and restart Slider cluster. The > >>persistent > >> > state such as app cache for services of existing tenants will be lost > >>and > >> > rebuilt again. That's very expensive operation for us. Is it possible > >>to > >> > add a component dynamically? Saw similar email threads about "Slider > >> flex", > >> > it doesn't do this, right? > >> > > >> > >> We've done a lot of work in 0.80 for live updates of a cluster, in which > >> each container is updated as we go along...this implicitly handles some > >> changes in the files, including the package with the metadata > >> > >> At the same time, I think we left out reload of the entire application > >> configs during flex simply to avoid confusion about what got updates > >>-and > >> what didn't. There's no fundamental reason, however, for flex not to do > >>a > >> reread of the appconf json. > >> > >> Gour -what do you think? How will rolling upgrades react to this? > >> > >> -steve > >> > >> > >
