Bhupesh: If you use FileSystem.newInstance(), does that return the correct object type? This sidesteps CACHE. - A
On Thu, Oct 15, 2009 at 3:07 PM, Bhupesh Bansal <[email protected]>wrote: > This code is not map/reduce code and run only on single machine and > Also each node prints the right value for "fs.default.name" so it is > Reading the right configuration file too .. The issue looks like use of > CACHE in filesystem and someplace my code is setting up a wrong value > If that is possible. > > Best > Bhupesh > > > > > On 10/15/09 1:46 PM, "Ashutosh Chauhan" <[email protected]> > wrote: > > > Each node reads its own conf files (mapred-site.xml, hdfs-site.xml etc.) > > Make sure your configs are consistent on all nodes across entire cluster > and > > are pointing to correct fs. > > > > Hope it helps, > > Ashutosh > > > > On Thu, Oct 15, 2009 at 16:36, Bhupesh Bansal <[email protected]> > wrote: > > > >> Hey Folks, > >> > >> I am seeing a very weird problem in FileSystem.get(Configuration). > >> > >> I want to get a FileSystem given the configuration, so I am using > >> > >> Configuration conf = new Configuration(); > >> > >> _fs = FileSystem.get(conf); > >> > >> > >> The problem is I am getting LocalFileSystem on some machines and > >> Distributed > >> on others. I am printing conf.get("fs.default.name") at all places and > >> It returns the right HDFS value 'hdfs://dummy:9000' > >> > >> My expectation is looking at fs.default.name if it is hdfs:// it should > >> give > >> me a DistributedFileSystem always. > >> > >> Best > >> Bhupesh > >> > >> > >> > >> > >> > >> > >
