Hi Tim, I don't know the answer to your specific problem but IIRC all ports on EC2 machines (within the same security group?) are open and reachable within the EC2 environment. You only have to open the ports (via ec2-authorize) if you want to reach them from outside EC2. So typically for hadoop clusters, you open the ports corresponding to the web-admin consoles (ports 50030, 50060, 50070, etc) so you can see the web console from your browser. I've not used EC2 for a year now so things might have changed.
Also do use the 'public' hostnames for configuring purposes - they resolve to internal IPs inside EC2 and external IPs from outside EC2. More on this: http://mail-archives.apache.org/mod_mbox/hadoop-core-user/200905.mbox/%3cdfd95197f3ae8c45b0a96c2f4ba3a2556bf123e...@sc-mbxc1.thefacebook.com%3e - Harish On Sat, Jun 20, 2009 at 6:12 PM, tim robertson <timrobertson...@gmail.com>wrote: > Hi all, > > I am using Hadoop to build a read only store for voldemort on EC2 and > for some reason can't get it to talk across the nodes. > I know this is a specific EC2 linux setup question, but I was hoping > someone could help me as I am sure all the apps build on Hadoop are > doing this - I'm not very hot on linux. > > The client is calling > > hdfs://ip-10-244-191-175.ec2.internal:54310/user/root/output/fullPD/stage2/node-0 > > and I have run > ec2-authorize hdfs-cluster -p 54310 > (but I am not sure this is the way to open the port) > > I'm using the cloudera AMI. > Full trace is below and any pointers are greatly appreciated! > > Cheers > > Tim > > > 09/06/20 06:54:09 ERROR gui.ReadOnlyStoreManagementServlet: Error > while performing operation. > java.net.ConnectException: Call to > ip-10-244-191-175.ec2.internal/10.244.191.175:54310 failed on > connection exception: java.net.ConnectException: Connection refused > at org.apache.hadoop.ipc.Client.wrapException(Client.java:743) > at org.apache.hadoop.ipc.Client.call(Client.java:719) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216) > at org.apache.hadoop.dfs.$Proxy6.getProtocolVersion(Unknown Source) > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:348) > at > org.apache.hadoop.dfs.DFSClient.createRPCNamenode(DFSClient.java:103) > at org.apache.hadoop.dfs.DFSClient.<init>(DFSClient.java:172) > at > org.apache.hadoop.dfs.DistributedFileSystem.initialize(DistributedFileSystem.java:67) > at > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1328) > at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:56) > at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1343) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:213) > at org.apache.hadoop.fs.Path.getFileSystem(Path.java:175) > at > voldemort.store.readonly.fetcher.HdfsFetcher.fetch(HdfsFetcher.java:82) > at > voldemort.server.http.gui.ReadOnlyStoreManagementServlet.doFetch(ReadOnlyStoreManagementServlet.java:162) > at > voldemort.server.http.gui.ReadOnlyStoreManagementServlet.doPost(ReadOnlyStoreManagementServlet.java:125) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) > at > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502) > at > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:389) > at > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) > at > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) > at org.mortbay.jetty.Server.handle(Server.java:326) > at > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534) > at > org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:879) > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:747) > at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218) > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) > at > org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409) > at > org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:520) > -- Harish Mallipeddi http://blog.poundbang.in
