According to https://forums.aws.amazon.com/thread.jspa?threadID=80436 there is a bug in how pig works with S3 that Amazon has fixed. Anyone know anything about this? I need to be pigging with EMR.
2012-02-16 21:33:16,394 [main] ERROR org.apache.pig.tools.grunt.Grunt - java.lang.IllegalArgumentException: This file system object (hdfs:// 10.215.190.97:9000) does not support access to the request path 's3://elasticmapreduce/samples/pig-apache/input' You possibly called FileSystem.get(conf) when you should have called FileSystem.get(uri, conf) to obtain a file system supporting your path. at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:372) at org.apache.hadoop.hdfs.DistributedFileSystem.checkPath(DistributedFileSystem.java:106) at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:162) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:530) at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:709) at org.apache.pig.backend.hadoop.datastorage.HPath.exists(HPath.java:106) at org.apache.pig.tools.grunt.GruntParser.processCD(GruntParser.java:629) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:275) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164) at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69) at org.apache.pig.Main.run(Main.java:495) at org.apache.pig.Main.main(Main.java:111) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) Details also at logfile: /tmp/pig_1329427404831.log -- Russell Jurney twitter.com/rjurney [email protected] datasyndrome.com
