I'm slightly confused how it works in code that is executed in the job submitter context (so outside of mappers/reducers code). You might want to check Hadoop documentation on that.
On Thu, Oct 11, 2012 at 3:41 PM, Bai Shen <[email protected]> wrote: > I'll give that a try. I know when I was doing new File("dir/blah.txt") it > was looking in the directory on the client machine that I was running the > job from. > > On Thu, Oct 11, 2012 at 9:30 AM, Ferdy Galema <[email protected] > >wrote: > > > new File("./blah.txt") should do it. It's a relative path. > > > > On Thu, Oct 11, 2012 at 2:52 PM, Bai Shen <[email protected]> > wrote: > > > > > I'm trying the first method now. However, some of my code requires a > > file > > > object that is a directory. So I can't use the first method with it. > > > > > > If I put something in the /classes, what path do I reference it with? > > Do I > > > just do new File(/classes/blah.txt")? > > > > > > Thanks. > > > > > > On Thu, Oct 11, 2012 at 7:40 AM, Ferdy Galema < > [email protected] > > > >wrote: > > > > > > > There are some options. The best way imho is to use > > > getResourceAsStream(). > > > > Put the file in the same package as the calling java code. Then do > > > > something like > > > > getClass().getClassLoader().getResourceAsStream("myfile"); > > > > > > > > If you really want to directly access the file then you can put it in > > > > /classes in the job file. This directory is expanded into the current > > > > working directory of a running task. > > > > > > > > Last but not least you are able to use shared filesystem, for example > > the > > > > HDFS or the mapreduce DistributedCache. This is useful if the files > are > > > > big, or change a lot. > > > > > > > > On Thu, Oct 11, 2012 at 1:20 PM, Bai Shen <[email protected]> > > > wrote: > > > > > > > > > I need to reference a file from my plugin. However, when I try to > > call > > > > it > > > > > using File(blah.txt), it looks for the file at the location where I > > run > > > > > nutch from, not in the job file. > > > > > > > > > > What is the proper way to refer to the files in the job file? > > > > > > > > > > Thanks. > > > > > > > > > > > > > > >
