paul-rogers edited a comment on issue #1830: DRILL-7333: Set container record count where missing URL: https://github.com/apache/drill/pull/1830#issuecomment-515725625 Several tests fail due a race condition in Drillbit shutdown: ``` [ERROR] TestUnionAll.testUnionAllEmptySides:348->BaseTestQuery.testRunAndReturn:340 » Rpc [ERROR] TestUnionDistinct.testUnionDistinctEmptySides:347->BaseTestQuery.testRunAndReturn:340 » Rpc [ERROR] TestFastComplexSchema.test5:96->BaseTestQuery.testRunAndReturn:340 » Rpc org.a... [ERROR] TestFlatten.testFlattenAfterJoin1:526->BaseTestQuery.testRunAndReturn:340 » Rpc [ERROR] TestFlatten.testFlattenAfterJoin2:537->BaseTestQuery.testRunAndReturn:340 » Rpc [ERROR] TestFlatten.testFlattenAfterSort:514->BaseTestQuery.testRunAndReturn:340 » Rpc ``` A typical error stack: ``` [INFO] Running org.apache.drill.exec.store.parquet.ParquetInternalsTest 16:45:18.456 [BitClient-1] ERROR o.a.d.exec.rpc.RpcExceptionHandler - Exception in RPC communication. Connection: /172.16.10.3:35130 <--> paul-linux/172.16.10.3:31075 (data client). Closing connection. java.lang.IllegalStateException: Attempting operation on allocator when allocator is closed. Allocator(rpc:bit-data) 0/0/8256/9223372036854775807 (res/actual/peak/limit) child allocators: 0 ledgers: 0 reservations: 0 at org.apache.drill.exec.memory.BaseAllocator.assertOpen(BaseAllocator.java:130) ~[drill-memory-base-1.17.0-SNAPSHOT.jar:1.17.0-SNAPSHOT] at org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:209) ~[drill-memory-base-1.17.0-SNAPSHOT.jar:1.17.0-SNAPSHOT] at org.apache.drill.exec.memory.DrillByteBufAllocator.buffer(DrillByteBufAllocator.java:49) ~[drill-memory-base-1.17.0-SNAPSHOT.jar:1.17.0-SNAPSHOT] at org.apache.drill.exec.memory.DrillByteBufAllocator.ioBuffer(DrillByteBufAllocator.java:64) ~[drill-memory-base-1.17.0-SNAPSHOT.jar:1.17.0-SNAPSHOT] at io.netty.channel.AdaptiveRecvByteBufAllocator$HandleImpl.allocate(AdaptiveRecvByteBufAllocator.java:104) ~[netty-transport-4.0.48.Final.jar:4.0.48.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:117) ~[netty-transport-4.0.48.Final.jar:4.0.48.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) [netty-transport-4.0.48.Final.jar:4.0.48.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) [netty-transport-4.0.48.Final.jar:4.0.48.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497) [netty-transport-4.0.48.Final.jar:4.0.48.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) [netty-transport-4.0.48.Final.jar:4.0.48.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) [netty-common-4.0.48.Final.jar:4.0.48.Final] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212] ``` This error is similar to the one that causes a UDF test to randomly fail. I've tracked it down to a race condition on Drillbit shutdown in unit tests. Somehow, some worker threads are still running as the Drillbit itself tries to shut down. I'll try to track it down, but since the error is pre-existing, I'll not hold up this PR for that fix.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
