Hi Gour, I tried modifying the Hadoop conf entry in slider-env.sh file.I have updated it as below:
export HADOOP_CONF_DIR=/etc/hadoop/conf But this also didnt yield any favourable results. As Tim mentioned, I shall try setting some other application probably HBase to see if the problem is Solr specific or Slider in general. In the meantime, do you have any other suggestions? When I run the command slider install-package command, it completed with status 0 and I could see the files under .slider folder on HDFS. The issue is with slider create command where its not able to connect to the resource manager, though RM deamon is running fine. Thanks & Regards Vijay On 27 August 2015 at 18:41, Gour Saha <[email protected]> wrote: > Since you have HDP, instead of setting the props in slider-client.xml, I > would suggest setting HADOOP_CONF_DIR in slider-env.sh (available in the > same folder) to /etc/hadoop/conf and see if that helps. > > -Gour > > On 8/27/15, 2:48 PM, "Timothy Potter" <[email protected]> wrote: > > >Hi Vijay, > > > >Are you able to create any other apps using slider create in your env? > >If so, that rules out any general issues and points to something > >specific to the Slider packaging for Solr. > > > >Tim > > > >On Thu, Aug 27, 2015 at 3:30 PM, Vijay Bhoomireddy > ><[email protected]> wrote: > >> Hi, > >> > >> > >> > >> I am facing some issue while setting up Solr on Hadoop using Slider. I > >>am > >> using HDP 2.3which has got Slider 0.80. I am following LucidWorks blog > >>at > >> https://github.com/LucidWorks/solr-slider for the same. I already have > a > >> single node Hortonworks HDP cluster. When I try to setup Solr on my HDP > >> cluster using Slider, I am facing some issues. > >> > >> > >> > >> As per the blog, I have performed the below steps: > >> > >> > >> > >> 1. I have setup a single node HDP cluster for which the hostname > >>is > >> myhdpcluster.com with all the essential services including ZooKeeper > and > >> Slider running on it. > >> > >> 2. Updated the resource manager address and port in > >>slider-client.xml > >> present under /usr/hdp/current/slider-client/conf > >> > >> <property> > >> > >> <name>yarn.resourcemanager.address</name> > >> > >> <value> myhdpcluster.com:8032</value> > >> > >> </property> > >> > >> 3. Cloned the LucidWorks git and moved it under > >>/home/hdfs/solr-slider > >> > >> 4. Downloaded solr latest stable distribution and renamed it as > >> solr.tgz and placed it under > >>/home/hdfs/solr-slider/package/files/solr.tgz > >> > >> 5. Next ran the following command from within the > >> /home/hdfs/solr-slider folder > >> > >> zip -r solr-on-yarn.zip metainfo.xml package/ > >> > >> 6. Next ran the following command as hdfs user > >> > >> slider install-package --replacepkg --name solr --package > >> /home/hdfs/solr-slider/solr-on-yarn.zip > >> > >> 7. Modified the following settings in the > >> /home/hdfs/solr-slider/appConfig-default.json file > >> > >> "java_home": MY_JAVA_HOME_LOCATION > >> > >> "site.global.app_root": "${AGENT_WORK_ROOT}/app/install/solr-5.2.1", > >> (Should this be changed to any other value?) > >> > >> "site.global.zk_host": " myhdpcluster.com:2181", > >> > >> 8. Set yarn.component.instances to 1 in resources-default.json > >>file > >> > >> 9. Next ran the following command > >> > >> slider create solr --template > >>/home/hdfs/solr-slider/appConfig-default.json > >> --resources /home/hdfs/solr-slider/resources-default.json > >> > >> > >> > >> During this step, I am seeing an message INFO client.RMProxy - > >>Connecting to > >> ResourceManager at myhdpcluster.com/10.0.2.15:8032 > >> > >> > >> INFO ipc.Client - Retrying connect to server: > >> myhdpcluster.com/10.0.2.15:8032. Already tried 0 time(s); > >> > >> > >> > >> This message keeps repeating for 50 times and then pauses for a couple > >>of > >> seconds and then prints the same message in a loop eternally. Not sure > >>on > >> where the problem is. I verified the Resource Manager URL and its > >>pointing > >> correctly. In my case, it's myhdpcluster.com:8032 I even tried passing > >> --manager param to the slider create solr command, but without any luck. > >> > >> I am able to login to Ambari console and see all the services running > >>fine , > >> including YARN related and ZooKeeper. Also, I could login to Resource > >> Manager's web UI as well and its working fine. > >> > >> > >> > >> Here is the complete content of the appConfig-default.json file. I > >>haven't > >> worked with Slider so far, so not very sure if some mistake has crept > >>into > >> this file while modifying it as per the changes mentioned by Lucidworks > >>on > >> the Github page. However, I tried to simulate the steps mentioned on the > >> Github Solr-Slider page. As you can see from below file, I am using > >>Java 7. > >> > >> > >> > >> { > >> > >> "schema": "http://example.org/specification/v2.0.0";, > >> > >> "metadata": { > >> > >> }, > >> > >> "global": { > >> > >> "application.def": "/home/hdfs/solr-slider/solr-on-yarn.zip", > >> > >> "java_home": "/usr/lib/jvm/jre-1.7.0-openjdk.x86_64", > >> > >> "site.global.app_root": > >> "${AGENT_WORK_ROOT}/app/install/solr-5.2.1-SNAPSHOT", > >> > >> "site.global.zk_host": "myhdpcluster.com:2181", > >> > >> "site.global.solr_host": "${SOLR_HOST}", > >> > >> "site.global.listen_port": "${SOLR.ALLOCATED_PORT}", > >> > >> "site.global.xmx_val": "1g", > >> > >> "site.global.xms_val": "1g", > >> > >> "site.global.gc_tune": "-XX:NewRatio=3 -XX:SurvivorRatio=4 > >> -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 > >> -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ConcGCThreads=4 > >> -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark > >> -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly > >> -XX:CMSInitiatingOccupancyFraction=50 > >>-XX:CMSMaxAbortablePrecleanTime=6000 > >> -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -verbose:gc > >> -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps > >> -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime", > >> > >> "site.global.zk_timeout": "15000", > >> > >> "site.global.server_module": "--module=http", > >> > >> "site.global.stop_key": "solrrocks", > >> > >> "site.global.solr_opts": "" > >> > >> }, > >> > >> "components": { > >> > >> "slider-appmaster": { > >> > >> "jvm.heapsize": "512M" > >> > >> }, > >> > >> "SOLR": { > >> > >> } > >> > >> } > >> > >> } > >> > >> > >> > >> Can anyone please let me know what could have gone wrong with my > >> configuration and what changes need to be made to get it working? Many > >> thanks in advance. > >> > >> > >> > >> Thanks & Regards > >> > >> Vijay > >> > >> > >> -- > >> The contents of this e-mail are confidential and for the exclusive use > >>of > >> the intended recipient. If you receive this e-mail in error please > >>delete > >> it from your system immediately and notify us either by e-mail or > >> telephone. You should not copy, forward or otherwise disclose the > >>content > >> of the e-mail. The views expressed in this communication may not > >> necessarily be the view held by WHISHWORKS. > > > > -- The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS.
