I am still seeing the TestValueVector unit tests fail on my linux machine (2/3 of my builds)
On Fri, Aug 14, 2015 at 2:40 PM, Abdel Hakim Deneche <[email protected]> wrote: > You can safely ignore the TestImpersonationMetadata error, it's most > likely caused by my change > > On Fri, Aug 14, 2015 at 12:36 PM, Abdel Hakim Deneche < > [email protected]> wrote: > >> In addition to test*VectorReallocation errors I am seeing frequently on >> my linux VM, today I saw the following error multiple times on my VM and >> once on my Macbook: >> >> Tests in error: >>> TestImpersonationMetadata>BaseTestQuery.closeClient:239 ยป IllegalState >>> Failure... >>> java.lang.IllegalStateException: Failure while closing accountor. >>> Expected private and shared pools to be set to initial values. However, >>> one or more were not. Stats are >>> zone init allocated delta >>> private 0 0 0 >>> shared 3221225472 3214166779 7058693. >>> at >>> org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200) >>> ~[classes/:na] >>> at org.apache.drill.exec.memory.Accountor.close(Accountor.java:390) >>> ~[classes/:na] >>> at >>> org.apache.drill.exec.memory.TopLevelAllocator.close(TopLevelAllocator.java:185) >>> ~[classes/:na] >>> at >>> org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:75) >>> ~[classes/:na] >>> at com.google.common.io.Closeables.close(Closeables.java:77) >>> ~[guava-14.0.1.jar:na] >>> at com.google.common.io.Closeables.closeQuietly(Closeables.java:108) >>> ~[guava-14.0.1.jar:na] >>> at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:294) >>> ~[classes/:na] >>> at >>> org.apache.drill.exec.server.Drillbit$ShutdownThread.run(Drillbit.java:332) >>> ~[classes/:na] >> >> >> On Wed, Aug 5, 2015 at 2:52 PM, Chris Westin <[email protected]> >> wrote: >> >>> Given that the difference is just >>> >>> > java.lang.Exception: Unexpected exception, >>> > expected<org.apache.drill.exec.exception.OversizedAllocationException> >>> but >>> > was<org.apache.drill.exec.memory.OutOfMemoryRuntimeException> >>> >>> The question of "what constitutes an oversized allocation?" comes to >>> mind. >>> Is this test fragile relative to being run in different environments? >>> I haven't seen the test so how is the determination that something is >>> oversized made? It seems like that criterion sometimes fails, and we get >>> an >>> OOM because whatever the request is is still very large. >>> >>> >>> On Wed, Aug 5, 2015 at 2:26 PM, Hanifi Gunes <[email protected]> >>> wrote: >>> >>> > I don't seem to be able to re-prod this. Let me look at this and >>> update you >>> > all. >>> > >>> > On Thu, Aug 6, 2015 at 12:03 AM, Abdel Hakim Deneche < >>> > [email protected]> >>> > wrote: >>> > >>> > > I didn't make any change, I'm running 2 forks (the default). I got >>> those >>> > > errors 3 times now, 2 on a linux VM and 1 on a linux physical node >>> > > >>> > > On Wed, Aug 5, 2015 at 1:03 PM, Hanifi Gunes <[email protected]> >>> > wrote: >>> > > >>> > > > Did you tighten your memory settings? How many forks are you >>> running >>> > > with? >>> > > > I bet you are truly running out of memory while executing this >>> > particular >>> > > > test case. >>> > > > >>> > > > -H+ >>> > > > >>> > > > On Wed, Aug 5, 2015 at 8:56 PM, Sudheesh Katkam < >>> [email protected]> >>> > > > wrote: >>> > > > >>> > > > > b2bbd99 committed on July 6th introduced the test. >>> > > > > >>> > > > > > On Aug 5, 2015, at 10:21 AM, Jinfeng Ni <[email protected] >>> > >>> > > wrote: >>> > > > > > >>> > > > > > In that case, we probably need do binary search to figure out >>> > which >>> > > > > recent >>> > > > > > patch is causing this problem. >>> > > > > > >>> > > > > > On Wed, Aug 5, 2015 at 10:03 AM, Abdel Hakim Deneche < >>> > > > > [email protected]> >>> > > > > > wrote: >>> > > > > > >>> > > > > >> Just got those errors on master too >>> > > > > >> >>> > > > > >> On Wed, Aug 5, 2015 at 9:07 AM, Abdel Hakim Deneche < >>> > > > > [email protected] >>> > > > > >>> >>> > > > > >> wrote: >>> > > > > >> >>> > > > > >>> I'm seeing those errors intermittently when building my >>> private >>> > > > > branch, I >>> > > > > >>> don't believe I made any change that would have caused them. >>> > Anyone >>> > > > > seen >>> > > > > >>> them too ? >>> > > > > >>> >>> > > > > >>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> testBitVectorReallocation(org.apache.drill.exec.record.vector.TestValueVector) >>> > > > > >>>> Time elapsed: 2.043 sec <<< ERROR! >>> > > > > >>>> java.lang.Exception: Unexpected exception, >>> > > > > >>>> >>> > > > >>> expected<org.apache.drill.exec.exception.OversizedAllocationException> >>> > > > > >> but >>> > > > > >>>> >>> was<org.apache.drill.exec.memory.OutOfMemoryRuntimeException> >>> > > > > >>>> at java.nio.Bits.reserveMemory(Bits.java:658) >>> > > > > >>>> at >>> java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) >>> > > > > >>>> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.UnpooledUnsafeDirectByteBuf.allocateDirect(UnpooledUnsafeDirectByteBuf.java:108) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.UnpooledUnsafeDirectByteBuf.<init>(UnpooledUnsafeDirectByteBuf.java:69) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:50) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:130) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.PooledByteBufAllocatorL.directBuffer(PooledByteBufAllocatorL.java:171) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> org.apache.drill.exec.memory.TopLevelAllocator.buffer(TopLevelAllocator.java:100) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> org.apache.drill.exec.memory.TopLevelAllocator.buffer(TopLevelAllocator.java:116) >>> > > > > >>>> at >>> > > > org.apache.drill.exec.vector.BitVector.reAlloc(BitVector.java:139) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> org.apache.drill.exec.record.vector.TestValueVector.testBitVectorReallocation(TestValueVector.java:125) >>> > > > > >>> >>> > > > > >>> >>> > > > > >>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> testFixedVectorReallocation(org.apache.drill.exec.record.vector.TestValueVector) >>> > > > > >>>> Time elapsed: 0.436 sec <<< ERROR! >>> > > > > >>>> java.lang.Exception: Unexpected exception, >>> > > > > >>>> >>> > > > >>> expected<org.apache.drill.exec.exception.OversizedAllocationException> >>> > > > > >> but >>> > > > > >>>> >>> was<org.apache.drill.exec.memory.OutOfMemoryRuntimeException> >>> > > > > >>>> at java.nio.Bits.reserveMemory(Bits.java:658) >>> > > > > >>>> at >>> java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) >>> > > > > >>>> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.UnpooledUnsafeDirectByteBuf.allocateDirect(UnpooledUnsafeDirectByteBuf.java:108) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.UnpooledUnsafeDirectByteBuf.<init>(UnpooledUnsafeDirectByteBuf.java:69) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:50) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:130) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.PooledByteBufAllocatorL.directBuffer(PooledByteBufAllocatorL.java:171) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> org.apache.drill.exec.memory.TopLevelAllocator.buffer(TopLevelAllocator.java:100) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> org.apache.drill.exec.memory.TopLevelAllocator.buffer(TopLevelAllocator.java:116) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> org.apache.drill.exec.vector.UInt4Vector.allocateBytes(UInt4Vector.java:187) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> org.apache.drill.exec.vector.UInt4Vector.allocateNew(UInt4Vector.java:177) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> org.apache.drill.exec.record.vector.TestValueVector.testFixedVectorReallocation(TestValueVector.java:85) >>> > > > > >>> >>> > > > > >>> >>> > > > > >>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> testVariableVectorReallocation(org.apache.drill.exec.record.vector.TestValueVector) >>> > > > > >>>> Time elapsed: 0.788 sec <<< ERROR! >>> > > > > >>>> java.lang.Exception: Unexpected exception, >>> > > > > >>>> >>> > > > >>> expected<org.apache.drill.exec.exception.OversizedAllocationException> >>> > > > > >> but >>> > > > > >>>> >>> was<org.apache.drill.exec.memory.OutOfMemoryRuntimeException> >>> > > > > >>>> at java.nio.Bits.reserveMemory(Bits.java:658) >>> > > > > >>>> at >>> java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) >>> > > > > >>>> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.UnpooledUnsafeDirectByteBuf.allocateDirect(UnpooledUnsafeDirectByteBuf.java:108) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.UnpooledUnsafeDirectByteBuf.<init>(UnpooledUnsafeDirectByteBuf.java:69) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:50) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:130) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> io.netty.buffer.PooledByteBufAllocatorL.directBuffer(PooledByteBufAllocatorL.java:171) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> org.apache.drill.exec.memory.TopLevelAllocator.buffer(TopLevelAllocator.java:100) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> org.apache.drill.exec.memory.TopLevelAllocator.buffer(TopLevelAllocator.java:116) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> org.apache.drill.exec.vector.VarCharVector.allocateNew(VarCharVector.java:372) >>> > > > > >>>> at >>> > > > > >>>> >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> org.apache.drill.exec.record.vector.TestValueVector.testVariableVectorReallocation(TestValueVector.java:142) >>> > > > > >>> >>> > > > > >>> >>> > > > > >>> Thanks >>> > > > > >>> -- >>> > > > > >>> >>> > > > > >>> Abdelhakim Deneche >>> > > > > >>> >>> > > > > >>> Software Engineer >>> > > > > >>> >>> > > > > >>> <http://www.mapr.com/> >>> > > > > >>> >>> > > > > >>> >>> > > > > >>> Now Available - Free Hadoop On-Demand Training >>> > > > > >>> < >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available >>> > > > > >>> >>> > > > > >>> >>> > > > > >> >>> > > > > >> >>> > > > > >> >>> > > > > >> -- >>> > > > > >> >>> > > > > >> Abdelhakim Deneche >>> > > > > >> >>> > > > > >> Software Engineer >>> > > > > >> >>> > > > > >> <http://www.mapr.com/> >>> > > > > >> >>> > > > > >> >>> > > > > >> Now Available - Free Hadoop On-Demand Training >>> > > > > >> < >>> > > > > >> >>> > > > > >>> > > > >>> > > >>> > >>> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available >>> > > > > >>> >>> > > > > >> >>> > > > > >>> > > > > >>> > > > >>> > > >>> > > >>> > > >>> > > -- >>> > > >>> > > Abdelhakim Deneche >>> > > >>> > > Software Engineer >>> > > >>> > > <http://www.mapr.com/> >>> > > >>> > > >>> > > Now Available - Free Hadoop On-Demand Training >>> > > < >>> > > >>> > >>> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available >>> > > > >>> > > >>> > >>> >> >> >> >> -- >> >> Abdelhakim Deneche >> >> Software Engineer >> >> <http://www.mapr.com/> >> >> >> Now Available - Free Hadoop On-Demand Training >> <http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available> >> > > > > -- > > Abdelhakim Deneche > > Software Engineer > > <http://www.mapr.com/> > > > Now Available - Free Hadoop On-Demand Training > <http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available> > -- Abdelhakim Deneche Software Engineer <http://www.mapr.com/> Now Available - Free Hadoop On-Demand Training <http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>
