Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hama Wiki" for change notification.
The "GettingStartedYARN" page has been changed by MinhoKim: https://wiki.apache.org/hama/GettingStartedYARN?action=diff&rev1=6&rev2=7 = Getting Started with Hama on YARN = - == Requirements == + == Preparations == Current Hama and Hadoop requires JRE 1.6 or higher and ssh to be set up between nodes in the cluster: @@ -15, +15 @@ This tutorial requires Hadoop 2.x already correctly installed. If you haven't done this yet, please follow the official documentation https://hadoop.apache.org/docs/stable/ + + == Configuration == + + Most of the configs are the same for Hama on YARN as for other deployment modes. See the [[GettingStarted/Properties|configuration page]] for more information. There are configs that are specific to Hama on YARN. + + === Properties === + ||'''Property Name'''||'''Default'''||'''Meaning'''|| + ||bsp.framework.name||yarn||In order to run Hama on YARN, it must be set this property. This property means run applications on YARN.|| + ||hama.appmaster.memory.mb||100mb||The amount of memory used by the BSPApplicationMaster. The total amount of memory used by the ApplicationMaster is calculated as follows. ''memoryInMb = 3 * BSP_TASK_NUM + hama.appmaster.memory.mb''. This is because the application master spawns 1-3 thread per launched task that each should take 1mb, plus a minimum of base memory usage of 100. If you face memory issues, you can set this to a higher value.|| + + == Launching Hama on YARN == + + Ensure that copy yarn-site.xml in in HADOOP_CONF_DIR or YARN_CONF_DIR to HAMA_CONF_DIR. Because this configuration file is used to connect to the YARN. + + Launch Hama application which is serialize printing example: + {{{ + $HAMA_HOME/bin/hama jar hama-yarn-0.7.0-SNAPSHOT.jar org.apache.hama.bsp.YarnSerializePrinting + }}} + + You should see "Hello BSP Message" which each container spawned in HDFS where you defined output path. == How to write a Hama-YARN job == @@ -41, +61 @@ {{{ job.setMemoryUsedPerTaskInMb(50); }}} - - == How to configure a job == - - There are some configuration values that the job needs to have in order to submit sucessfully to YARN infrastructure. - - The importantest configurations are the {{{yarn.resourcemanager.address}}} and {{{fs.default.name}}}. This should point to the address (hostname+port) where your ResourceManager and HDFS Namonode runs, for example {{{localhost:8040}}}, {{{hdfs://localhost:9000}}}. - - Another important configuration value is the amount of memory used by the BSPApplicationMaster. You can configure a base amount of memory for the application master with this configuration key - {{{ - hama.appmaster.memory.mb - }}} - - By default, this is set to 100mb. - - The total amount of memory used by the ApplicationMaster is calculated as follows - - {{{ - int memoryInMb = 3 * this.getNumBspTask() + conf.getInt("hama.appmaster.memory.mb", 100) - }}} - - This is because the application master spawns 1-3 threads per launched task that each should take 1mb, plus a minimum of base memory usage of 100. - If you face memory issues, you can set this to a higher value. == How to submit a job == @@ -111, +109 @@ to submit a Hama job. You can just change the {{{BSPJob}}} to {{{YARNBSPJob}}}. - == Hama-YARN example == - - TODO this must be revised once the example has moved the jar. - - {{{ - bin/hama jar hama-yarn-0.7.0-SNAPSHOT.jar org.apache.hama.bsp.YarnSerializePrinting - }}} - - Once running, you should see in the spawned application master log that it is launching containers. - When the containers launched, you can see in the logs that there is a little "Hello World" from the other tasks. -
