I think you have the wrong port. 8050 is the port in HDP Sandbox. But Ambari uses 8052 as default. Please check your install and configure cluster entity appropriately. Its available in yarn-site.xml and the property is yarn.resourcemanager.address.
Thanks! On Thu, Aug 7, 2014 at 5:30 PM, Ed Kohlwey <[email protected]> wrote: > I don't have a copy of the sandbox handy. If you look at the top of the > logs falcon prints a commit id in a banner of stars when it starts. Can you > check the version in the sandbox? It may provide a hint. > > Sent from my mobile device. Please excuse any typos or shorthand. > On Aug 7, 2014 6:16 PM, "Tyler D" <[email protected]> wrote: > > > On Thu, Aug 7, 2014 at 12:00 PM, Ed Kohlwey <[email protected]> wrote: > > > > > I'm attempting to deploy a cluster entity through an ambari-managed > > falcon. > > > This is a clean install from 1.6.1. > > > > > > When submitting the cluster definition, I get the following message: > > > > > > $ sudo -u admin falcon entity -submit -file cluster.xml -type cluster > > > > > > Error: Invalid Execute server or port: <hostname redacted>:8050 > > > > > > Cannot initialize Cluster. Please check your configuration for > > > mapreduce.framework.name and the correspond server addresses. > > > > > > > > > mapreduce.framework.name is set yarn (per ambari deployment defaults). > > My > > > configuration line looks like this: > > > > > > <interface type="execute" endpoint="<hostname redacted>:8050" > > > version="2.4.0" /> > > > > > > Enabling debugging in the hadoop packages in the log4j.xml shows that > the > > > connection is being considered but not established. > > > > > > 2014-08-07 15:23:03,513 DEBUG - > > > [1141105573@qtp-216944274-0:admin:POST//entities/submit/cluster > > > a1b37e55-34fb-48f3-830d-e8e736f79c75] ~ Trying ClientProtocolProvider : > > > org.apache.hadoop.mapred.YarnClientProtocolProvider (Cluster:90) > > > > > > at > > > > > > > > > org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34) > > > > > > at > > > > > > > > > org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34) > > > > > > at > > > > > > > > > org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34) > > > > > > 2014-08-07 15:23:03,971 INFO - > > > [1141105573@qtp-216944274-0:admin:POST//entities/submit/cluster > > > a1b37e55-34fb-48f3-830d-e8e736f79c75] ~ Failed to use > > > org.apache.hadoop.mapred.YarnClientProtocolProvider due to error: null > > > (Cluster:113) > > > > > > > > > Any ideas of what could be wrong are appreciated. I am able to launch > > other > > > yarn jobs on this cluster successfully via Hive. > > > > > > > > > > > I am having this same exact issue. In my case, Falcon is installed > through > > HDP 2.1 distribution, which uses Ambari 1.6.1. > > > > At first I thought the problem was a version or port misconfiguration in > my > > cluster entity spec that I was trying to use. > > > > I tried giving Falcon various different cluster xml configurations. All > > attempts failed. > > > > I included the xml I tried in this mail. Also posted to > > http://pastebin.com/Ce07vyGv > > > > Hopefully this helps people isolate the problem/eliminate non-problems. > > > > All of these interface of type=execute lines failed to register with the > > cluster (which makes me think this xml probably isn't the problem): > > > > <?xml version="1.0"?> > > <cluster colo="USWestOregon" description="oregonHadoopCluster" > > name="primaryCluster" xmlns="uri:falcon:cluster:0.1"> > > <interfaces> > > <interface type="readonly" > > endpoint="hftp://vm-centos6-hdp21-a1.hdp.hadoop:50070"; version="2.4.0" /> > > <interface type="write" > > endpoint="hdfs://vm-centos6-hdp21-a1.hdp.hadoop:8020" version="2.4.0" /> > > <!--<interface type="execute" > > endpoint="vm-centos6-hdp21-a1.hdp.hadoop:8050" version="2.2.0" />--> > > <!-- ^^FAILS: gives error like... > > $ falcon entity -type cluster -submit -file > > /home/ambari-qa/falconChurnDemo/oregonCluster.xml` > > Error:Invalid Execute server or port: > > vm-centos6-hdp21-a1.hdp.hadoop:8050 > > Cannot initialize Cluster. Please check your configuration > for > > mapreduce.framework.name and the correspond server addresses. > > (FalconWebException:66) > > --> > > <!--<interface type="execute" > > endpoint="vm-centos6-hdp21-a1.hdp.hadoop:8088" version="2.2.0" /> --><!-- > > ALSO FAILS --> > > <!--<interface type="execute" > > endpoint="vm-centos6-hdp21-a1.hdp.hadoop:8050" version="2.4.0" /> --><!-- > > ALSO FAILS --> > > <!--<interface type="execute" > > endpoint="vm-centos6-hdp21-a1.hdp.hadoop:8050" version="0.20.2" /> > --><!-- > > ALSO FAILS --> > > <!--<interface type="execute" > > endpoint="vm-centos6-hdp21-a1.hdp.hadoop:8030" version="2.4.0" /> --><!-- > > ALSO FAILS --> > > <!--<interface type="execute" > > endpoint="vm-centos6-hdp21-a1.hdp.hadoop:8025" version="2.4.0" />--><!-- > > ALSO FAILS --> > > <!--<interface type="execute" > > endpoint="vm-centos6-hdp21-a1.hdp.hadoop:8141" version="2.4.0" />--><!-- > > ALSO FAILS --> > > <!--<interface type="execute" endpoint="rm:8050" > version="0.20.2" > > />--><!-- ALSO FAILS --> > > <!--<interface type="execute" endpoint="localhost:8050" > > version="0.20.2" />--><!-- ALSO FAILS --> > > <interface type="execute" > > endpoint="vm-centos6-hdp21-a1.hdp.hadoop:8021" version="2.4.0" /><!-- > ALSO > > FAILS --> > > <interface type="workflow" endpoint=" > > http://vm-centos6-hdp21-a1.hdp.hadoop:11000/oozie/"; version="4.0.0" /> > > <interface type="messaging" > > endpoint="tcp://vm-centos6-hdp21-a1.hdp.hadoop:61616?daemon=true" > > version="5.1.6" /> > > </interfaces> > > <locations> > > <location name="staging" > path="/apps/falcon/primaryCluster/staging" > > /> > > <location name="temp" path="/tmp" /> > > <location name="working" > path="/apps/falcon/primaryCluster/working" > > /> > > </locations> > > </cluster> > > > > And of course here's the error: > > Error:Invalid Execute server or port: vm-centos6-hdp21-a1.hdp.hadoop:8050 > > Cannot initialize Cluster. Please check your configuration for > > mapreduce.framework.name and the correspond server addresses. > > (FalconWebException:66) > > > > > > > > Something I thought was weird is when I execute the bin/falcon-status.sh > > and/or bin/service-status shell scripts: they always come back saying > that > > the Falcon server is not running. This conflicts, however, with the > > situation because the Falcon process is running, the web interface at > > http://vm-centos6-hdp21-a1.hdp.hadoop:15000/ is up, and Ambari reports > > the > > service as being up. Here is me running those commands: > > > > [ambari-qa@vm-centos6-hdp21-a1 falcon]$ bin/falcon-status > > Hadoop is installed, adding hadoop classpath to falcon classpath > > falcon is not running. > > [ambari-qa@vm-centos6-hdp21-a1 falcon]$ echo $? > > 255 > > [ambari-qa@vm-centos6-hdp21-a1 falcon]$ bin/service-status.sh > > Invalid option for app: . Valid choices are falcon and prism > > [ambari-qa@vm-centos6-hdp21-a1 falcon]$ bin/service-status.sh falcon > > Hadoop is installed, adding hadoop classpath to falcon classpath > > falcon is not running. > > [ambari-qa@vm-centos6-hdp21-a1 falcon]$ bin/service-status.sh prism > > Hadoop is installed, adding hadoop classpath to falcon classpath > > mkdir: cannot create directory `/usr/lib/falcon/server/webapp/prism': > > Permission denied > > /usr/lib/falcon/bin/falcon-config.sh: line 99: cd: > > /usr/lib/falcon/server/webapp/prism: No such file or directory > > java.io.FileNotFoundException: /usr/lib/falcon/server/webapp/prism.war > (No > > such file or directory) > > at java.io.FileInputStream.open(Native Method) > > at java.io.FileInputStream.<init>(FileInputStream.java:146) > > at java.io.FileInputStream.<init>(FileInputStream.java:101) > > at sun.tools.jar.Main.run(Main.java:259) > > at sun.tools.jar.Main.main(Main.java:1177) > > /usr/lib/falcon/bin/falcon-config.sh: line 101: cd: OLDPWD not set > > prism is not running. > > > > > > > > Also want to mention that I tried Hortonworks's latest Sandbox vm ( > > > http://hortonassets.s3.amazonaws.com/2.1/vmware/Hortonworks_Sandbox_2.1.ova > > ), and when I execute the bin/falcon-status.sh and/or bin/service-status > > shell scripts: they come back as expected, saying that the Falcon server > is > > up and running and gives the url. > > > > > > Thanks Ed for sending out this mail... I guess for me, whats next could > be > > to investigate the Falcon version differences between the HDP2.1 cluster > I > > installed (with the problems) and the Falcon version in the Sandbox vm > > since it appears ... better. > > > -- Regards, Venkatesh “Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away.” - Antoine de Saint-Exupéry
