Did you get rid of the entries in slider-client.xml? I am surprised that your RM is running at port 8032. The typical default port is 8050. Can you check /etc/hadoop/conf/yarn-site.xml and see what is yarn.resourcemanager.address set to?
-Gour On 8/28/15, 3:51 AM, "Vijaya Narayana Reddy Bhoomi Reddy" <[email protected]> wrote: >Hi Gour, > >I tried modifying the Hadoop conf entry in slider-env.sh file.I have >updated it as below: > >export HADOOP_CONF_DIR=/etc/hadoop/conf > >But this also didnt yield any favourable results. > >As Tim mentioned, I shall try setting some other application probably >HBase >to see if the problem is Solr specific or Slider in general. In the >meantime, do you have any other suggestions? > >When I run the command slider install-package command, it completed with >status 0 and I could see the files under .slider folder on HDFS. The issue >is with slider create command where its not able to connect to the >resource >manager, though RM deamon is running fine. > > >Thanks & Regards >Vijay > > > > > >On 27 August 2015 at 18:41, Gour Saha <[email protected]> wrote: > >> Since you have HDP, instead of setting the props in slider-client.xml, I >> would suggest setting HADOOP_CONF_DIR in slider-env.sh (available in the >> same folder) to /etc/hadoop/conf and see if that helps. >> >> -Gour >> >> On 8/27/15, 2:48 PM, "Timothy Potter" <[email protected]> wrote: >> >> >Hi Vijay, >> > >> >Are you able to create any other apps using slider create in your env? >> >If so, that rules out any general issues and points to something >> >specific to the Slider packaging for Solr. >> > >> >Tim >> > >> >On Thu, Aug 27, 2015 at 3:30 PM, Vijay Bhoomireddy >> ><[email protected]> wrote: >> >> Hi, >> >> >> >> >> >> >> >> I am facing some issue while setting up Solr on Hadoop using Slider. >>I >> >>am >> >> using HDP 2.3which has got Slider 0.80. I am following LucidWorks >>blog >> >>at >> >> https://github.com/LucidWorks/solr-slider for the same. I already >>have >> a >> >> single node Hortonworks HDP cluster. When I try to setup Solr on my >>HDP >> >> cluster using Slider, I am facing some issues. >> >> >> >> >> >> >> >> As per the blog, I have performed the below steps: >> >> >> >> >> >> >> >> 1. I have setup a single node HDP cluster for which the >>hostname >> >>is >> >> myhdpcluster.com with all the essential services including ZooKeeper >> and >> >> Slider running on it. >> >> >> >> 2. Updated the resource manager address and port in >> >>slider-client.xml >> >> present under /usr/hdp/current/slider-client/conf >> >> >> >> <property> >> >> >> >> <name>yarn.resourcemanager.address</name> >> >> >> >> <value> myhdpcluster.com:8032</value> >> >> >> >> </property> >> >> >> >> 3. Cloned the LucidWorks git and moved it under >> >>/home/hdfs/solr-slider >> >> >> >> 4. Downloaded solr latest stable distribution and renamed it as >> >> solr.tgz and placed it under >> >>/home/hdfs/solr-slider/package/files/solr.tgz >> >> >> >> 5. Next ran the following command from within the >> >> /home/hdfs/solr-slider folder >> >> >> >> zip -r solr-on-yarn.zip metainfo.xml package/ >> >> >> >> 6. Next ran the following command as hdfs user >> >> >> >> slider install-package --replacepkg --name solr --package >> >> /home/hdfs/solr-slider/solr-on-yarn.zip >> >> >> >> 7. Modified the following settings in the >> >> /home/hdfs/solr-slider/appConfig-default.json file >> >> >> >> "java_home": MY_JAVA_HOME_LOCATION >> >> >> >> "site.global.app_root": >>"${AGENT_WORK_ROOT}/app/install/solr-5.2.1", >> >> (Should this be changed to any other value?) >> >> >> >> "site.global.zk_host": " myhdpcluster.com:2181", >> >> >> >> 8. Set yarn.component.instances to 1 in resources-default.json >> >>file >> >> >> >> 9. Next ran the following command >> >> >> >> slider create solr --template >> >>/home/hdfs/solr-slider/appConfig-default.json >> >> --resources /home/hdfs/solr-slider/resources-default.json >> >> >> >> >> >> >> >> During this step, I am seeing an message INFO client.RMProxy - >> >>Connecting to >> >> ResourceManager at myhdpcluster.com/10.0.2.15:8032 >> >> >> >> >> >> INFO ipc.Client - Retrying connect to server: >> >> myhdpcluster.com/10.0.2.15:8032. Already tried 0 time(s); >> >> >> >> >> >> >> >> This message keeps repeating for 50 times and then pauses for a >>couple >> >>of >> >> seconds and then prints the same message in a loop eternally. Not >>sure >> >>on >> >> where the problem is. I verified the Resource Manager URL and its >> >>pointing >> >> correctly. In my case, it's myhdpcluster.com:8032 I even tried >>passing >> >> --manager param to the slider create solr command, but without any >>luck. >> >> >> >> I am able to login to Ambari console and see all the services running >> >>fine , >> >> including YARN related and ZooKeeper. Also, I could login to Resource >> >> Manager's web UI as well and its working fine. >> >> >> >> >> >> >> >> Here is the complete content of the appConfig-default.json file. I >> >>haven't >> >> worked with Slider so far, so not very sure if some mistake has crept >> >>into >> >> this file while modifying it as per the changes mentioned by >>Lucidworks >> >>on >> >> the Github page. However, I tried to simulate the steps mentioned on >>the >> >> Github Solr-Slider page. As you can see from below file, I am using >> >>Java 7. >> >> >> >> >> >> >> >> { >> >> >> >> "schema": "http://example.org/specification/v2.0.0";, >> >> >> >> "metadata": { >> >> >> >> }, >> >> >> >> "global": { >> >> >> >> "application.def": "/home/hdfs/solr-slider/solr-on-yarn.zip", >> >> >> >> "java_home": "/usr/lib/jvm/jre-1.7.0-openjdk.x86_64", >> >> >> >> "site.global.app_root": >> >> "${AGENT_WORK_ROOT}/app/install/solr-5.2.1-SNAPSHOT", >> >> >> >> "site.global.zk_host": "myhdpcluster.com:2181", >> >> >> >> "site.global.solr_host": "${SOLR_HOST}", >> >> >> >> "site.global.listen_port": "${SOLR.ALLOCATED_PORT}", >> >> >> >> "site.global.xmx_val": "1g", >> >> >> >> "site.global.xms_val": "1g", >> >> >> >> "site.global.gc_tune": "-XX:NewRatio=3 -XX:SurvivorRatio=4 >> >> -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 >> >> -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ConcGCThreads=4 >> >> -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark >> >> -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly >> >> -XX:CMSInitiatingOccupancyFraction=50 >> >>-XX:CMSMaxAbortablePrecleanTime=6000 >> >> -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -verbose:gc >> >> -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps >> >> -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime", >> >> >> >> "site.global.zk_timeout": "15000", >> >> >> >> "site.global.server_module": "--module=http", >> >> >> >> "site.global.stop_key": "solrrocks", >> >> >> >> "site.global.solr_opts": "" >> >> >> >> }, >> >> >> >> "components": { >> >> >> >> "slider-appmaster": { >> >> >> >> "jvm.heapsize": "512M" >> >> >> >> }, >> >> >> >> "SOLR": { >> >> >> >> } >> >> >> >> } >> >> >> >> } >> >> >> >> >> >> >> >> Can anyone please let me know what could have gone wrong with my >> >> configuration and what changes need to be made to get it working? >>Many >> >> thanks in advance. >> >> >> >> >> >> >> >> Thanks & Regards >> >> >> >> Vijay >> >> >> >> >> >> -- >> >> The contents of this e-mail are confidential and for the exclusive >>use >> >>of >> >> the intended recipient. If you receive this e-mail in error please >> >>delete >> >> it from your system immediately and notify us either by e-mail or >> >> telephone. You should not copy, forward or otherwise disclose the >> >>content >> >> of the e-mail. The views expressed in this communication may not >> >> necessarily be the view held by WHISHWORKS. >> > >> >> > >-- >The contents of this e-mail are confidential and for the exclusive use of >the intended recipient. If you receive this e-mail in error please delete >it from your system immediately and notify us either by e-mail or >telephone. You should not copy, forward or otherwise disclose the content >of the e-mail. The views expressed in this communication may not >necessarily be the view held by WHISHWORKS.
