That's what I thought took Of course it's possible there is a bug in your reader allocating too large or too many times memory.
Tim Sent from my iPhone > On Nov 15, 2013, at 9:37 PM, Tommy Guy <[email protected]> wrote: > > Odd, aren't we setting this in pom.xml: > > <plugin> > <artifactId>maven-surefire-plugin</artifactId> > <version>2.15</version> > <configuration> > <argLine>-XX:MaxDirectMemorySize=4096M </argLine> > </configuration> > </plugin> > > Thanks, Tim. > > Tommy > > > >> On Fri, Nov 15, 2013 at 10:55 PM, Timothy Chen <[email protected]> wrote: >> >> I forgot what the flag is but you also want to set your direct memory to >> allocate at a larger size too. >> >> Tim >> >> Sent from my iPhone >> >>> On Nov 15, 2013, at 8:44 PM, Tommy Guy <[email protected]> wrote: >>> >>> Hello everyone, >>> >>> I'm working on a CSV parser for Drill, and I'm running into trouble with >>> the tests. I've set -Xmx=2048m and error persists. I've tried this on >>> Ubuntu 11.10 and 13.10 (13.10 running in Vagrant) >>> >>> Anyone seen this before? >>> >>> Tommy >>> >>> Here's an example stack trace >>> from >> org.apache.drill.exec.physical.impl.TestSimpleFunctions.testByteSubstring >>> >>> java.lang.UnsupportedOperationException: null >>> at io.netty.buffer.PoolArenaL$DirectArena.newByteBuf(PoolArenaL.java:400) >>> at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:94) >>> at >> io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:236) >>> at >> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:132) >>> at >> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:123) >>> at >> org.apache.drill.exec.memory.DirectBufferAllocator.buffer(DirectBufferAllocator.java:35) >>> at org.apache.drill.exec.vector.IntVector.allocateNew(IntVector.java:100) >>> at >> org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:29) >>> at >> org.apache.drill.exec.store.mock.MockRecordReader.getVector(MockRecordReader.java:63) >>> at >> org.apache.drill.exec.store.mock.MockRecordReader.setup(MockRecordReader.java:78) >>> at >> org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:65) >>> at >> org.apache.drill.exec.store.mock.MockScanBatchCreator.getBatch(MockScanBatchCreator.java:44) >>> at >> org.apache.drill.exec.physical.impl.ImplCreator.visitSubScan(ImplCreator.java:105) >>> at >> org.apache.drill.exec.physical.impl.ImplCreator.visitSubScan(ImplCreator.java:60) >>> at >> org.apache.drill.exec.store.mock.MockSubScanPOP.accept(MockSubScanPOP.java:94) >>> at >> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195) >>> at >> org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:202) >>> at >> org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:60) >>> at >> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:40) >>> at >> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195) >>> at >> org.apache.drill.exec.physical.impl.ImplCreator.visitProject(ImplCreator.java:92) >>> at >> org.apache.drill.exec.physical.impl.ImplCreator.visitProject(ImplCreator.java:60) >>> at org.apache.drill.exec.physical.config.Project.accept(Project.java:51) >>> at >> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195) >>> at >> org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:202) >>> at >> org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:60) >>> at >> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:40) >>> at >> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195) >>> at >> org.apache.drill.exec.physical.impl.ImplCreator.visitScreen(ImplCreator.java:145) >>> at >> org.apache.drill.exec.physical.impl.ImplCreator.visitScreen(ImplCreator.java:60) >>> at org.apache.drill.exec.physical.config.Screen.accept(Screen.java:102) >>> at >> org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:212) >>> at >> org.apache.drill.exec.physical.impl.TestSimpleFunctions.testByteSubstring(TestSimpleFunctions.java:200) >>> >>> I'm also seeing the error in Lillth: >>> Exception in pipeline. Closing channel between local /127.0.1.1:31010and >>> remote /127.0.0.1:43946 >>> >>> Lillith shows: >>> java.lang.OutOfMemoryError >>> >>> at >> org.apache.drill.exec.rpc.RpcExceptionHandler.exceptionCaught(RpcExceptionHandler.java:33) >>> at >> io.netty.channel.DefaultChannelHandlerContext.invokeExceptionCaught(DefaultChannelHandlerContext.java:272) >>> at >> io.netty.channel.DefaultChannelHandlerContext.fireExceptionCaught(DefaultChannelHandlerContext.java:250) >>> at >> io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:60) >>> at >> io.netty.channel.DefaultChannelHandlerContext.invokeExceptionCaught(DefaultChannelHandlerContext.java:272) >>> at >> io.netty.channel.DefaultChannelHandlerContext.fireExceptionCaught(DefaultChannelHandlerContext.java:250) >>> at >> io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:60) >>> at >> io.netty.channel.DefaultChannelHandlerContext.invokeExceptionCaught(DefaultChannelHandlerContext.java:272) >>
