Which version of sqoop are you using. Sqoop 1.4.4 addressed use of other filesystems with the fix mentioned in SQOOP-1033
Thanks Venkat On Tue, Feb 4, 2014 at 8:14 AM, Jarek Jarcec Cecho <[email protected]>wrote: > Yes Imran, > I would try to define the fs.defaultFS for the S3 in core-site.xml and see > if it will help Sqoop to accept the S3 path. > > Jarcec > > On Tue, Feb 04, 2014 at 08:08:17AM -0800, Imran Akbar wrote: > > thanks Jarek, > > How would I do that? Do I need to set fs.defaultFS in core-site.xml, > or > > is it something else? Is there a document somewhere which describes > this? > > > > yours, > > imran > > > > > > On Mon, Feb 3, 2014 at 9:31 PM, Jarek Jarcec Cecho <[email protected] > >wrote: > > > > > Would you mind trying to set the S3 filesystem as the default one for > > > Sqoop? > > > > > > Jarcec > > > > > > On Mon, Feb 03, 2014 at 10:25:50AM -0800, Imran Akbar wrote: > > > > Hi, > > > > I've been able to sqoop from MySQL into HDFS, but I was > wondering if > > > it > > > > was possible to send the data directly to S3 instead. I've read some > > > posts > > > > on this forum and others that indicate that it's not possible to do > this > > > - > > > > could someone confirm? > > > > > > > > I tried to get it to work by setting: > > > > --warehouse-dir s3n://MYS3APIKEY:MYS3SECRETKEY@bucketname/folder/ > > > > or > > > > --target-dir s3n://MYS3APIKEY:MYS3SECRETKEY@bucketname/folder/ > > > > > > > > options but I get the error: > > > > ERROR tool.ImportTool: Imported Failed: This file system object > (hdfs:// > > > > 10.168.22.133:9000) does not support access to the request path > > > > 's3n://****:****@iakbar.emr/new-hive-output/_logs' You possibly > called > > > > FileSystem.get(conf) when you should have called FileSystem.get(uri, > > > conf) > > > > to obtain a file system supporting your path > > > > > > > > If it's not possible to do this, should I just import to HDFS and > then > > > > output to S3? Is there an easy way to do this without having to > specify > > > > the schema of the whole table again? > > > > > > > > thanks, > > > > imran > > > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
