ok, found it: we are using Cloudera CDHu3u, they change the ulimit for child jobs. but I still don't know how to change their default settings yet
On Wed, Jun 13, 2012 at 2:15 PM, Yang <teddyyyy...@gmail.com> wrote: > I got the OutOfMemoryError when I tried to open an Lucene index. > > it's very weird since this is only seen when I run this inside an Apache > PIG LATIN script on a particular hadoop cluster of ours, > and it runs fine on an old cluster; also it runs fine if I run the java > code outside of PIG. > > > as I understand, OutOfMemoryError is not out of heap, so does this mean > that mmap() failed? > I checked the ulimit of the user account running this, it seems to be all > unlimited > > > any clues? > > Thanks > yang > > Caused by: java.io.IOException: Map failed > at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748) > at > org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:265) > at > org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216) > at > org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:86) > at > org.apache.lucene.index.SegmentReader.get(SegmentReader.java:115) > at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:93) > at > org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:113) > at > org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:29) > at > org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:81) > at > org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:754) > at > org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:75) > at org.apache.lucene.index.IndexReader.open(IndexReader.java:462) > at org.apache.lucene.index.IndexReader.open(IndexReader.java:308) > at > cgm.places.matcher.ListingSearcher.createIndex(ListingSearcher.java:45) > at > cgm.places.matcher.ListingSearcher.<init>(ListingSearcher.java:38) > at > cgm.places.matcher.SearchListingUdf.<init>(SearchListingUdf.java:34) > ... 83 more > Caused by: java.lang.OutOfMemoryError: Map failed > at sun.nio.ch.FileChannelImpl.map0(Native Method) > at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:745) > ... 98 more > >
