Hi,
I am using EC2 to start a hadoop cluster (cloudera's distribution) and setup
hive on it (specifically, the hive client is on the master/jobtracker). I am
using the latest version of hive (with hadoop 0.18.3) and have setup a mysql
metastore on an EBS mount for persistence. The HDFS is also persisting on
several EBS mounts. Everything ran fine the very first time I setup hive and
added a couple of tables. Then I shutdown the cluster and started a new one
and mounted the metastore EBS mount on the new cluster's master. After that
if I try to run a hive query I get the following error:
----
Job Submission failed with exception 'java.net.ConnectException(Call to
domU-amazon-internal-ip.compute-1.internal/amazon-internal-ip:9000 failed on
connection exception: java.net.ConnectException: Connection refused)'
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.ExecDriver
----
The problem is that amazon-internal-ip refers to the master of the previous
cluster I had shutdown. For some reason, hive isn't picking up the new
cluster's configuration.
I tried setting the mapred.job.tracker and fs.default.name with the new
ip's, but it didn't work. Similarly, adding these parameters to
hive-site.xml config file did not work. I also checked hadoop config files
and the parameters are correctly specified with the new ip's. Also, running
a normal hadoop job works fine.
Does anyone know where is hive picking up these parameters from?
Any help would be appreciated. Thanks.
--
gaurav.
----
For more info, here is the detailed error message from hive.log, it says
pretty much the same thing -
2009-07-17 18:40:59,081 ERROR ql.Driver (SessionState.java:printError(279))
- FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.ExecDriver
2009-07-17 18:47:01,530 ERROR exec.ExecDriver
(SessionState.java:printError(279)) - Job Submission failed with exception
'java.net.ConnectException(Call to
domU-12-31-38-00-C8-32.compute-1.internal/10.252.207.192:9000 failed on
connection exception: java.net.ConnectException: Connection refused)'
java.net.ConnectException: Call to
domU-12-31-38-00-C8-32.compute-1.internal/10.252.207.192:9000 failed on
connection exception: java.net.ConnectException: Connection refused
at org.apache.hadoop.ipc.Client.wrapException(Client.java:743)
at org.apache.hadoop.ipc.Client.call(Client.java:719)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216)
at org.apache.hadoop.dfs.$Proxy0.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:348)
at org.apache.hadoop.dfs.DFSClient.createRPCNamenode(DFSClient.java:103)
at org.apache.hadoop.dfs.DFSClient.<init>(DFSClient.java:172)
at
org.apache.hadoop.dfs.DistributedFileSystem.initialize(DistributedFileSystem.java:67)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1328)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:56)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1343)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:213)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:175)
at
org.apache.hadoop.hive.ql.exec.ExecDriver.addInputPaths(ExecDriver.java:685)
at
org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:378)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:335)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:241)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:122)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:165)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:258)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:402)
at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:301)
at org.apache.hadoop.ipc.Client$Connection.access$1700(Client.java:178)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:820)
at org.apache.hadoop.ipc.Client.call(Client.java:705)
... 27 more
----
