Ah - I tried it on a new, 64-bit VM and the tests passed. Tommy
On Fri, Nov 15, 2013 at 11:56 PM, Timothy Chen <[email protected]> wrote: > That's what I thought took > > Of course it's possible there is a bug in your reader allocating too large > or too many times memory. > > Tim > > Sent from my iPhone > > > On Nov 15, 2013, at 9:37 PM, Tommy Guy <[email protected]> wrote: > > > > Odd, aren't we setting this in pom.xml: > > > > <plugin> > > <artifactId>maven-surefire-plugin</artifactId> > > <version>2.15</version> > > <configuration> > > <argLine>-XX:MaxDirectMemorySize=4096M </argLine> > > </configuration> > > </plugin> > > > > Thanks, Tim. > > > > Tommy > > > > > > > >> On Fri, Nov 15, 2013 at 10:55 PM, Timothy Chen <[email protected]> > wrote: > >> > >> I forgot what the flag is but you also want to set your direct memory to > >> allocate at a larger size too. > >> > >> Tim > >> > >> Sent from my iPhone > >> > >>> On Nov 15, 2013, at 8:44 PM, Tommy Guy <[email protected]> > wrote: > >>> > >>> Hello everyone, > >>> > >>> I'm working on a CSV parser for Drill, and I'm running into trouble > with > >>> the tests. I've set -Xmx=2048m and error persists. I've tried this on > >>> Ubuntu 11.10 and 13.10 (13.10 running in Vagrant) > >>> > >>> Anyone seen this before? > >>> > >>> Tommy > >>> > >>> Here's an example stack trace > >>> from > >> > org.apache.drill.exec.physical.impl.TestSimpleFunctions.testByteSubstring > >>> > >>> java.lang.UnsupportedOperationException: null > >>> at > io.netty.buffer.PoolArenaL$DirectArena.newByteBuf(PoolArenaL.java:400) > >>> at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:94) > >>> at > >> > io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:236) > >>> at > >> > io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:132) > >>> at > >> > io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:123) > >>> at > >> > org.apache.drill.exec.memory.DirectBufferAllocator.buffer(DirectBufferAllocator.java:35) > >>> at > org.apache.drill.exec.vector.IntVector.allocateNew(IntVector.java:100) > >>> at > >> > org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:29) > >>> at > >> > org.apache.drill.exec.store.mock.MockRecordReader.getVector(MockRecordReader.java:63) > >>> at > >> > org.apache.drill.exec.store.mock.MockRecordReader.setup(MockRecordReader.java:78) > >>> at > >> org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:65) > >>> at > >> > org.apache.drill.exec.store.mock.MockScanBatchCreator.getBatch(MockScanBatchCreator.java:44) > >>> at > >> > org.apache.drill.exec.physical.impl.ImplCreator.visitSubScan(ImplCreator.java:105) > >>> at > >> > org.apache.drill.exec.physical.impl.ImplCreator.visitSubScan(ImplCreator.java:60) > >>> at > >> > org.apache.drill.exec.store.mock.MockSubScanPOP.accept(MockSubScanPOP.java:94) > >>> at > >> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195) > >>> at > >> > org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:202) > >>> at > >> > org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:60) > >>> at > >> > org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:40) > >>> at > >> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195) > >>> at > >> > org.apache.drill.exec.physical.impl.ImplCreator.visitProject(ImplCreator.java:92) > >>> at > >> > org.apache.drill.exec.physical.impl.ImplCreator.visitProject(ImplCreator.java:60) > >>> at > org.apache.drill.exec.physical.config.Project.accept(Project.java:51) > >>> at > >> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195) > >>> at > >> > org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:202) > >>> at > >> > org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:60) > >>> at > >> > org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:40) > >>> at > >> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195) > >>> at > >> > org.apache.drill.exec.physical.impl.ImplCreator.visitScreen(ImplCreator.java:145) > >>> at > >> > org.apache.drill.exec.physical.impl.ImplCreator.visitScreen(ImplCreator.java:60) > >>> at org.apache.drill.exec.physical.config.Screen.accept(Screen.java:102) > >>> at > >> > org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:212) > >>> at > >> > org.apache.drill.exec.physical.impl.TestSimpleFunctions.testByteSubstring(TestSimpleFunctions.java:200) > >>> > >>> I'm also seeing the error in Lillth: > >>> Exception in pipeline. Closing channel between local /127.0.1.1:31010 > and > >>> remote /127.0.0.1:43946 > >>> > >>> Lillith shows: > >>> java.lang.OutOfMemoryError > >>> > >>> at > >> > org.apache.drill.exec.rpc.RpcExceptionHandler.exceptionCaught(RpcExceptionHandler.java:33) > >>> at > >> > io.netty.channel.DefaultChannelHandlerContext.invokeExceptionCaught(DefaultChannelHandlerContext.java:272) > >>> at > >> > io.netty.channel.DefaultChannelHandlerContext.fireExceptionCaught(DefaultChannelHandlerContext.java:250) > >>> at > >> > io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:60) > >>> at > >> > io.netty.channel.DefaultChannelHandlerContext.invokeExceptionCaught(DefaultChannelHandlerContext.java:272) > >>> at > >> > io.netty.channel.DefaultChannelHandlerContext.fireExceptionCaught(DefaultChannelHandlerContext.java:250) > >>> at > >> > io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:60) > >>> at > >> > io.netty.channel.DefaultChannelHandlerContext.invokeExceptionCaught(DefaultChannelHandlerContext.java:272) > >> >
