Author: gourksaha
Date: Fri May 22 18:31:02 2015
New Revision: 1681180
URL: http://svn.apache.org/r1681180
Log:
SLIDER-245 Update sample app jmemcached documentation with sample add-on
package app
Modified:
incubator/slider/site/trunk/content/docs/slider_specs/hello_world_slider_app.md
Modified:
incubator/slider/site/trunk/content/docs/slider_specs/hello_world_slider_app.md
URL:
http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/docs/slider_specs/hello_world_slider_app.md?rev=1681180&r1=1681179&r2=1681180&view=diff
==============================================================================
---
incubator/slider/site/trunk/content/docs/slider_specs/hello_world_slider_app.md
(original)
+++
incubator/slider/site/trunk/content/docs/slider_specs/hello_world_slider_app.md
Fri May 22 18:31:02 2015
@@ -225,8 +225,8 @@ Note that the `parameter.py` file only r
env.set_params(params)
check_process_status(params.pid_file)
- if __name__ == "__main__":
- Memcached().execute()
+ if __name__ == "__main__":
+ Memcached().execute()
Thats pretty much it. The script basically does the following:
@@ -246,4 +246,182 @@ which, expands to
port and memory are based on the input provided. And, class path includes
YARN container location for the active container where the tarball got expanded.
-So, without Slider/YARN, one should be able to execute the command above and
see memcached up and running. It might be a better option to try the command
first to ensure that you have the right tarballs and environment to run
memcached.
\ No newline at end of file
+So, without Slider/YARN, one should be able to execute the command above and
see memcached up and running. It might be a better option to try the command
first to ensure that you have the right tarballs and environment to run
memcached.
+
+## Add on package
+
+You can deploy the application with add on packages as well. Add on packages
are extension libraries, configurations, or scripts that the master application
can use. For example, HBase has Phoenix providing SQL access to it and Ranger
providing Authorization added to it. With Slider, you can deploy HBase as the
master application, and Phoenix and/or Ranger as the add on package to it
+
+To do that, you need to package your add on packages in a similar way to the
master package. For example, in order to deploy Phoenix with HBase, you need to
create an add on package as below:
+
+```
+unzip -l "$@" Phoenix.zip
+Archive: Archive.zip
+ Length Date Time Name
+ -------- ---- ---- ----
+ 2143 04-19-15 12:03 metainfo.xml
+ 0 11-19-14 15:17 package/
+ 0 04-19-15 11:41 package/files/
+ 1908 12-04-14 17:07 package/files/end2endTest.py
+ 840 12-04-14 17:07 package/files/hadoop-metrics2-hbase.properties
+ 2271 12-04-14 17:07 package/files/hadoop-metrics2-phoenix.properties
+ 1136 12-04-14 17:07 package/files/hbase-site.xml
+ 2342 12-04-14 17:07 package/files/log4j.properties
+ 3762 12-04-14 17:07 package/files/performance.py
+ 38015961 12-04-14 17:07 package/files/phoenix-4.2.2-client.jar
+ 4039324 12-04-14 17:07 package/files/phoenix-4.2.2-server.jar
+ 3171 12-04-14 17:07 package/files/phoenix_utils.py
+ 1669 12-04-14 17:07 package/files/psql.py
+ 1820 12-04-14 17:07 package/files/readme.txt
+ 2314 12-04-14 17:07 package/files/sqlline.py
+ 0 04-19-15 21:57 package/scripts/
+ 2287 04-19-15 21:57 package/scripts/addon_hbase_master.py
+ 1659 04-19-15 11:45 package/scripts/addon_hbase_regionserver.py
+ 1079 04-18-15 17:36 package/scripts/params.py
+ 0 04-17-15 18:42 package/templates/
+ -------- -------
+ 42106790 21 files
+```
+
+Please note that different from the master package, the add on package doesn't
have appConfig.json and resources.json, because appConfig.json is not supported
for now, while you can define all the variables you need in the master
package's appConfig.json.
+For add on package, you should/can not add any new components to the master
application, so resources.json is not needed either.
+
+What is remaining is the metainfo.xml, in which you should/can not add any new
components. However, you can apply some components in the add on package's
metainfo.json belonging to all components in the master package's metainfo.json
by specifying 'ALL' in the component name
+
+```
+<metainfo>
+ <schemaVersion>2.0</schemaVersion>
+ <applicationPackage>
+ <name>PHOENIX</name>
+ <comment>
+ Apache Phoenix is a relational database layer over HBase delivered as a
client-embedded JDBC driver targeting low latency queries over HBase data.
+ </comment>
+ <version>2.0</version>
+ <type>ADDON-PACKAGE</type>
+ <minHadoopVersion>1.0</minHadoopVersion>
+
+ <components>
+ <component>
+ <name>HBASE_REGIONSERVER</name>
+ <commandScript>
+ <script>scripts/addon_hbase_regionserver.py</script>
+ <scriptType>PYTHON</scriptType>
+ <timeout>600</timeout>
+ </commandScript>
+ </component>
+ <component>
+ <name>HBASE_MASTER</name>
+ <commandScript>
+ <script>scripts/addon_hbase_master.py</script>
+ <scriptType>PYTHON</scriptType>
+ <timeout>600</timeout>
+ </commandScript>
+ </component>
+ </components>
+
+ <osSpecifics>
+ <osSpecific>
+ <osType>any</osType>
+ <packages>
+ <package>
+ <type>tarball</type>
+ <name>files/phoenix-4.2.2-server.jar</name>
+ </package>
+ </packages>
+ </osSpecific>
+ </osSpecifics>
+ </applicationPackage>
+</metainfo>
+```
+
+Please note the type of the application package is specified as
'ADDON-PACKAGE'.
+
+The add on package is expected to implement INSTALL for each component,
without other commands defined for the master application. Note, you can still
use Slider's extensive library that can be used to implement the commands. More
details can be found [here](/docs/slider_specs/writing_app_command_scripts).
+
+Below is an example of the python script for HbaseMaster:
+```
+import sys
+import os
+from shutil import copyfile
+from resource_management import *
+
+
+class HbaseMaster(Script):
+ def install(self, env):
+ config = Script.get_config()
+
+ src = config['commandParams']['addonPackageRoot'] +
"/package/files/phoenix-4.2.2-client.jar"
+ dst = config['configurations']['global']['app_root'] +
"/lib/phoenix-4.2.2-client.jar"
+ copyfile(src, dst)
+
+ filestocopytobin =
["end2endTest.py","hadoop-metrics2-hbase.properties","hadoop-metrics2-phoenix.properties","hbase-site.xml","log4j.properties","performance.py","phoenix_utils.py","psql.py","readme.txt","sqlline.py"]
+
+ for file in filestocopytobin:
+ src = config['commandParams']['addonPackageRoot'] + "/package/files/" +
file
+ dst = config['configurations']['global']['app_root'] + "/bin/" + file
+ copyfile(src, dst)
+ bin_file = config['configurations']['global']['app_root'] + "/bin/" +
"sqlline.py"
+ os.chmod(bin_file, 0555)
+
+if __name__ == "__main__":
+ HbaseMaster().execute()
+ pass
+```
+
+Below is an example of the python script for HbaseRegionServer:
+```
+import sys
+from shutil import copyfile
+from resource_management import *
+
+
+class HbaseRegionserver(Script):
+ def install(self, env):
+ config = Script.get_config()
+ src = config['commandParams']['addonPackageRoot'] +
"/package/files/phoenix-4.2.2-server.jar"
+ dst = config['configurations']['global']['app_root'] +
"/lib/phoenix-4.2.2-server.jar"
+ copyfile(src, dst)
+
+ def configure(self, env):
+ import params
+ env.set_params(params)
+
+
+ def start(self, env):
+ import params
+ env.set_params(params)
+ self.configure(env) # for security
+
+
+ def stop(self, env):
+ import params
+ env.set_params(params)
+
+
+ def status(self, env):
+ import status_params
+ env.set_params(status_params)
+
+if __name__ == "__main__":
+ HbaseRegionserver().execute()
+ pass
+```
+
+Similarly, you can provide params.py to provide all configuration needed to
run the install command
+
+```
+from resource_management import *
+
+# server configurations
+config = Script.get_config()
+
+hbase_root = config['configurations']['global']['app_root']
+jar_location = config['commandParams']['addonPackageRoot'] +
"/package/files/phoenix-4.2.2-server.jar"
+
+print ('jar_location' + jar_location)
+```
+
+When submitting an application with add on package with Slider, you need to
use '--addon' option to specify the add on package name and path to its zipped
package
+```
+slider create [application_name] --template [path to appConfig.json]
--resources [path to resources.json] --addon PHOENIX [path to Phoenix.zip]
+```
