To push the file to HDFS (put it in the 'a_hdfsDirectory' directory)
Configuration config = new Configuration();
FileSystem hdfs = FileSystem.get(config);
Path srcPath = new Path(a_directory + "/" + outputName);
Path dstPath = new Path(a_hdfsDirectory + "/" + outputName);
hdfs.copyFromLocalFile(sr
Thanks Chris for your reply!
Well, I could not understand much of what has been discussed on that forum.
I am unaware of Cascading.
My problem is simple - I want a directory to present in the local working
directory of tasks so that I can access it from my map task in the following
manner :
Fil
Hi,
I've found it much easier to write the file to HDFS use the API, then pass
the 'path' to the file in HDFS as a property. You'll need to remember to
clean up the file after you're done with it.
Example details are in this thread:
http://groups.google.com/group/cascading-user/browse_thread/thre
Yes, my HDFS paths are of the form /home/user-name/
And I have used these in DistributedCache's addCacheFiles method
successfully.
Thanks,
Akhil
Amareshwari Sriramadasu wrote:
>
> Is your hdfs path /home/akhil1988/Config.zip? Usually hdfs path is of the
> form /user/akhil1988/Config.zip.
> J
Is your hdfs path /home/akhil1988/Config.zip? Usually hdfs path is of the form
/user/akhil1988/Config.zip.
Just wondering if you are giving wrong path in the uri!
Thanks
Amareshwari
akhil1988 wrote:
Thanks Amareshwari for your reply!
The file Config.zip is lying in the HDFS, if it would not h
Thanks Amareshwari for your reply!
The file Config.zip is lying in the HDFS, if it would not have been then the
error would be reported by the jobtracker itself while executing the
statement:
DistributedCache.addCacheArchive(new URI("/home/akhil1988/Config.zip"),
conf);
But I get error in the ma
Hi Akhil,
DistributedCache.addCacheArchive takes path on hdfs. From your code, it looks
like you are passing local path.
Also, if you want to create symlink, you should pass URI as hdfs://#, besides calling
DistributedCache.createSymlink(conf);
Thanks
Amareshwari
akhil1988 wrote:
Please a
Please ask any questions if I am not clear above about the problem I am
facing.
Thanks,
Akhil
akhil1988 wrote:
>
> Hi All!
>
> I want a directory to be present in the local working directory of the
> task for which I am using the following statements:
>
> DistributedCache.addCacheArchive(new
