Yes, i expected the reader to handle it as well and couldn't find some setting to force it to handle it. The choice remained just not to write the file in the first place.
> Thanks for pointing me to Nutch-1029. This will fix it. It seems > completely Hadoop related though, because Nutch tries to open map files > using merely Hadoop code. CrawlDbReader delegates it to > MapFileOutputFormat.getReaders(...). I'd except at least Hadoop itself > to handle floating _SUCCESS files correctly. > > On 09/06/2011 05:03 PM, Markus Jelsma wrote: > > Strange, i've not seen permission issues with the _SUCCESS file! Anyway, > > deleting or not creating the file fixes it. I've committed some fixes for > > readdb -stats and readlinkdb but not for updatedb. > > > > Its related to: > > https://issues.apache.org/jira/browse/NUTCH-1029 > > Can you open an issue? > > > >> Please see following exception. It looks like it is caused by the > >> _SUCCESS file created by Hadoop when trying to open map files in a > >> permission checked HDFS on CDH3u1. After deleting the _SUCCESS it works. > >> What is a better solution for this problem? > >> > >> Exception in thread "main" > >> org.apache.hadoop.security.AccessControlException: > >> org.apache.hadoop.security.AccessControlException: Permission denied: > >> user=ferdy, access=EXECUTE, > >> inode="/user/ferdy/root10/crawldb/current/_SUCCESS":ferdy:ferdy:-rw-r--r > >> -- > >> > >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > >> > >> Method) > >> > >> at > >> > >> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorA > >> cce ssorImpl.java:39) at > >> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCons > >> tru ctorAccessorImpl.java:27) at > >> java.lang.reflect.Constructor.newInstance(Constructor.java:513) at > >> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteExcepti > >> on. java:95) at > >> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteExcept > >> ion .java:57) at > >> org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:699) at > >> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFi > >> leS ystem.java:542) at > >> org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:776) at > >> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424) > >> > >> at > >> > >> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419) > >> > >> at > >> > >> org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:30 > >> 2) > >> > >> at org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:284) > >> at org.apache.hadoop.io.MapFile$Reader.<init>(MapFile.java:273) > >> at org.apache.hadoop.io.MapFile$Reader.<init>(MapFile.java:260) > >> at org.apache.hadoop.io.MapFile$Reader.<init>(MapFile.java:253) > >> at > >> > >> org.apache.hadoop.mapred.MapFileOutputFormat.getReaders(MapFileOutputFor > >> mat .java:93) at > >> org.apache.nutch.crawl.CrawlDbReader.openReaders(CrawlDbReader.java:81) > >> > >> at > >> org.apache.nutch.crawl.CrawlDbReader.get(CrawlDbReader.java:379) > >> at > >> > >> org.apache.nutch.crawl.CrawlDbReader.readUrl(CrawlDbReader.java:386) at > >> org.apache.nutch.crawl.CrawlDbReader.main(CrawlDbReader.java:511) at > >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav > >> a:3 9) at > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor > >> Imp l.java:25) at java.lang.reflect.Method.invoke(Method.java:597) > >> > >> at org.apache.hadoop.util.RunJar.main(RunJar.java:186) > >> > >> Caused by: org.apache.hadoop.ipc.RemoteException: > >> org.apache.hadoop.security.AccessControlException: Permission denied: > >> user=ferdy, access=EXECUTE, > >> inode="/user/ferdy/root10/crawldb/current/_SUCCESS":ferdy:ferdy:-rw-r--r > >> -- > >> > >> at > >> > >> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermi > >> ssi onChecker.java:203)
