Author: gourksaha
Date: Tue May 19 05:39:49 2015
New Revision: 1680175
URL: http://svn.apache.org/r1680175
Log:
SLIDER-855 Site documentation for upgrade/reconfig support in Slider
Added:
incubator/slider/site/trunk/content/docs/slider_specs/application_pkg_upgrade.md
Modified:
incubator/slider/site/trunk/templates/nav.html
Added:
incubator/slider/site/trunk/content/docs/slider_specs/application_pkg_upgrade.md
URL:
http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/docs/slider_specs/application_pkg_upgrade.md?rev=1680175&view=auto
==============================================================================
---
incubator/slider/site/trunk/content/docs/slider_specs/application_pkg_upgrade.md
(added)
+++
incubator/slider/site/trunk/content/docs/slider_specs/application_pkg_upgrade.md
Tue May 19 05:39:49 2015
@@ -0,0 +1,204 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# Apache Slider App Packages Rolling Upgrade
+
+##Introduction
+
+From version `0.80.0-incubating` onwards, Slider supports application package
rolling upgrade.
+Application package owners can now upgrade to a newer version of their
application
+binary and/or configuration, without any downtime.
+
+By saying no downtime, it is primarily the responsibility of the application
owners to orchestrate the
+upgrade. It needs to be done in a planned fashion, such that the application
appears to be running, as per
+the end-users are concerned. Hence, if **_Apache HBase_** is undergoing an
upgrade from version `0.98.4`
+to `1.1.0`, then sufficient **_regionservers_** should be running at all
times, which can handle the load
+at the time of the upgrade. Application owners also need to ensure that their
application is compatible
+in a mixed version mode. So, while the **_regionservers_** are getting
upgraded, the **_master_** will
+still be the older version. Hence `1.1.0` **_regionserver_** should be
backward compatible with `0.98.4`
+**_masters_**.
+
+Currently, _Ambari Slider View_ supports automated deployment and management
of application packages.
+Automated application package upgrade will be supported soon.
+
+## Phases of Upgrade/Downgrade
+
+#### YARN core (libraries and configurations) upgrade/downgrade
+Running Slider apps will continue to run, with no downtime
+
+#### Slider client upgrade/downgrade
+Does not affect running Slider apps at all. New version of client can co-exist
with older versions.
+
+#### Slider Application Master upgrade/downgrade of running applications
+Applications started prior to the start of YARN core upgrade/downgrade, will
continue to run with the
+older versions of Slider and Hadoop core libraries. To upgrade running Slider
AMs the application needs
+to be stopped and restarted with the new version of the client. There is no
support for rolling upgrade
+of Slider AM.
+
+#### Applications deployed by Slider (binaries and configurations)
upgrade/downgrade
+This is the what this document is primarily about.
+
+## Upgrade Slider Application Packages
+
+A run-book style list of atomic steps exposed by Slider. These steps will be
automated
+by Apache Ambari in a future release. It can be easily orchestrated by a shell
script
+or executed manually in the correct order.
+
+### An application owner brings a new package and creates a new instance
+
+Upload a new application package *slider-hbase-app-package_v1.0.zip*
+
+ slider package --install --name MyHBase_Facebook --version 1.0 --package
~/slider-hbase-app-package_v1.0.zip
+
+Create an application *MyHBase_Facebook_Finance*
+
+ slider create MyHBase_Facebook_Finance --template
~/myHBase_template_v1.0.json --resources ~/myHBase_resources_v1.0.json
+
+Application *MyHBase_Facebook_Finance* goes through its usual lifecycle
+
+ slider flex MyHBase_Facebook_Finance --component HBASE_MASTER 5
+
+### The app owner can list all containers of an app instance with filtering
options
+
+ slider list MyHBase_Facebook_Finance --containers
+
+ slider list MyHBase_Facebook_Finance --containers --components HBASE_MASTER
+
+ slider list MyHBase_Facebook_Finance --containers --components
HBASE_MASTER HBASE_REGIONSERVER --version 1.0
+
+ slider list MyHBase_Facebook_Finance --containers --version 1.0
+
+### At some point, the app owner has a newer version of the package and needs
to upgrade
+
+Uploads the newer version 2.0 of the app package
+
+ slider package --install --name MyHBase_Facebook --version 2.0 --package
~/slider-hbase-app-package_v2.0.zip
+
+This command upgrades the internal state of Slider App Master with the new
config and resource
+specifications. After this, upgrade command needs to be issued against all
containers to upgrade
+them to the new app version e.g. v2.0 in this case. Note, it is recommended to
not issue the
+flex command, although Slider does not explicitly block it.
+
+ slider upgrade MyHBase_Facebook_Finance --template
~/myHBase_template_v2.0.json --resources ~/myHBase_resources_v2.0.json
+
+At this point the upgrade orchestrator (Ambari or a script) will rely on the
slider `list` command with
+`--containers` option (with additional filter options `--components` and/or
`--version`) to find all
+the containers that are running for the application. The orchestrator needs to
decide how it wants to
+span/schedule the upgrade commands of every single container. This will
primarily depend on the
+type of the application. For e.g. say *HBase* might want to call upgrade on
all *Master* containers
+first before calling upgrade on any of the *RegionServer* containers. *Storm*
might want to provide
+a 5 min delay between the upgrade of the *Nimbus* and *Supervisor*
container(s).
+
+Upgrade all containers with ids `id1, id2, .. idn`. It is possible that all
these n containers can
+actually be down at the same time. If this is not desired, then issue n
upgrade command with a
+single container id at a time.
+
+ slider upgrade MyHBase_Facebook_Finance --containers id1 id2 .. idn
+
+ e.g. slider upgrade MyHBase_Facebook_Finance --containers
container_e03_1427412101162_0001_01_000002
container_e03_1427412101162_0001_01_000003
+
+Upgrade all containers of all roles `role1, role2, .. rolen`. Again, all
containers of all the n
+roles can be down at the same time. If this is not desired, use the
`--containers` option with
+one container at a time.
+
+ slider upgrade MyHBase_Facebook_Finance --components role1 role2 .. rolen
+
+ e.g. slider upgrade MyHBase_Facebook_Finance --components HBASE_MASTER
HBASE_REGIONSERVER
+
+### When do we know that upgrade is complete?
+
+The upgrade command need to be executed against all containers and upgrade can
be considered
+complete only if the command `slider list MyHBase_Facebook_Finance
--containers --version 1.0`
+returns no containers.
+
+The app owner can also call `slider list MyHBase_Facebook_Finance
--containers` to explicitly
+check that all expected containers are running and with the desired new
version (e.g. 2.0 in
+this case).
+
+If any issues are encountered in Slider and/or the application during upgrade,
the
+application can be downgraded to its original version.
+
+## Downgrade Applications Deployed and Upgraded by Slider
+
+A Slider deployed application might be required to be downgraded to its
original version for
+any reason - whether the upgrade was completed in entirety or was in-flight.
There are no new
+commands exposed to perform downgrade. It basically involves calling the
upgrade command again
+with the older version (e.g. version 1.0 in this case) of the application
config and resource
+specifications. It needs to be followed by upgrade commands issued against all
containers which
+were upgraded to the newer version.
+
+This command although called upgrade, actually downgrades the internal state
of Slider App
+Master with the old v1.0 config and resource specifications
+
+ slider upgrade MyHBase_Facebook_Finance --template
~/myHBase_template_v1.0.json --resources ~/myHBase_resources_v1.0.json
+
+Take the list of all containers returned by the list command -
+
+ slider list MyHBase_Facebook_Finance --containers --version 2.0
+
+And issue upgrade command on all of them (in desired orchestrated fashion) -
+
+ slider upgrade MyHBase_Facebook_Finance --containers id1 id2 .. idn
+
+## Pre and post upgrade hooks
+
+### Pre-upgrade hook (optional)
+The pre-upgrade steps, if provided, will allow applications to execute simple
housekeeping tasks
+before Slider actually calls stop operation in an upgrade scenario
(specifically if they need to
+be performed in every single container and 1000s of them are running). An
example could be to
+send a message to a queue that the current instance of memcached is going down
so that the load
+balancer rules can be dynamically updated. Performing long-running tasks or
tasks that needs to
+be executed by only 1 instance of a component (n instances of which are
running) should be
+performed manually before calling upgrade (or maybe before calling the upgrade
command). The
+pre-upgrade hook is not a good candidate for such operations. Additional
parameters might have
+to be exposed, like timeout, which can be used to wait at most n seconds (say)
after which Slider
+will call the application stop hook even if the pre-upgrade operation is not
completed.
+
+### Post-upgrade hook (optional) - not yet supported
+This allows applications to perform simple housekeeping tasks prior to calling
start on the new
+version of the application component. This is helpful only if such tasks are
required to be
+performed in every single container and 1000s of them are running. This is
required only in the
+upgrade scenario and should not be called on new containers created using flex
up in non-upgrade
+scenarios. This is tricky, how will Slider determine this? I donât think we
need a post-upgrade
+hook? Clarify with Rakesh and remove?
+
+## Upgrade support of simple apps with no packages
+- Upgrade of such simple apps does not involve binaries
+- Upgrade of such simple apps basically means re-configuration only
+- The following can be done to perform the reconfiguration (or upgrade if you
prefer to call so)
+ - Download the app package that was internally created by Slider from HDFS
(First class Slider
+ command-line support will not be provided for download. So refer to
documentation or get
+ Slider/Yarn admin support to find the app package location in HDFS.)
+ - Make necessary changes to metainfo.json and repackage the app
+ - Make necessary updates to appConfig.json and resources.json
+ - Follow the upgrade steps provided in this document to upgrade the simple
app
+
+## Assumptions/Recommendations
+
+- Letâs take the scenario when the upgrade command has been called (with new
app config and
+ resources) and the orchestrator is in the middle of calling upgrade command
on all containers
+ in a certain order/schedule. At this point, it is important to understand
that if a container
+ (on which the upgrade command was not called yet) fails, then its
replacement container will
+ come up with the newer version of the application.
+- It is recommended to not call `flex` (to increase or decrease the no of
containers assigned
+ to a specific role) after the upgrade command has been issued and until
upgrade command is
+ issued on all currently running containers (as per current resource
specification). Basically
+ avoid calling flex until `slider list --containers --version <old_ver>`
returns no more
+ containers. Note, flex command is not explicitly blocked by Slider, hence
unexpected
+ behavior of the application might occur if called.
+
+
Modified: incubator/slider/site/trunk/templates/nav.html
URL:
http://svn.apache.org/viewvc/incubator/slider/site/trunk/templates/nav.html?rev=1680175&r1=1680174&r2=1680175&view=diff
==============================================================================
--- incubator/slider/site/trunk/templates/nav.html (original)
+++ incubator/slider/site/trunk/templates/nav.html Tue May 19 05:39:49 2015
@@ -71,6 +71,7 @@
<li id="nav_use_resources"><a
href="/docs/configuration/resources.html">Resource Specification</a></li>
<li role="presentation" class="divider"></li>
<li id="nav_use_app_package"><a href="/docs/slider_specs/index.html">Creating
App Packages</a></li>
+<li id="nav_use_app_package_upgrade"><a
href="/docs/slider_specs/application_pkg_upgrade.html">Upgrading App
Packages</a></li>
<li role="presentation" class="divider"></li>
<li id="nav_use_client_config"><a
href="/docs/client-configuration.html">Client Configuration</a></li>
<li id="nav_use_exitcodes"><a href="/docs/exitcodes.html">Client Exit
Codes</a></li>
