Hi, I'm facing this issue when i try to run the test in
org.apache.drill.exec.store.parquet.ParquetRecordReaderTest [tanujit@legolas java-exec]$ mvn -Dtest=ParquetRecordReaderTest#testMultipleRowGroupsAndReads test [INFO] Scanning for projects... [WARNING] [WARNING] Some problems were encountered while building the effective model for org.apache.drill.exec:java-exec:jar:1.0-SNAPSHOT [WARNING] Detected profiles.xml alongside org.apache.drill.exec:java-exec:1.0-SNAPSHOT, this file is no longer supported and was ignored, please use the settings.xml instead [WARNING] [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. [WARNING] [WARNING] For this reason, future Maven versions might no longer support building such malformed projects. [WARNING] [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building java-exec 1.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ Downloading: http://repository.apache.org/snapshots/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml Downloading: http://conjars.org/repo/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml Downloading: https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml Downloading: http://repository.apache.org/snapshots/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml Downloading: https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml Downloading: http://conjars.org/repo/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml Downloading: http://repository.apache.org/snapshots/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml Downloading: http://conjars.org/repo/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml Downloading: https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml [INFO] [INFO] --- maven-resources-plugin:2.6:copy-resources (copy-resources) @ java-exec --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 370 resources [INFO] [INFO] --- maven-enforcer-plugin:1.2:enforce (no_commons_logging) @ java-exec --- [INFO] [INFO] --- maven-antrun-plugin:1.6:run (generate-sources) @ java-exec --- [WARNING] Parameter tasks is deprecated, use target instead [INFO] Executing tasks main: [INFO] Executed tasks [INFO] Registering compile source root /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/generated-sources [INFO] [INFO] --- fmpp-maven-plugin:1.0:generate (generate-sources) @ java-exec --- Downloading: https://oss.sonatype.org/content/repositories/snapshots/org/freemarker/freemarker/maven-metadata.xml Downloading: https://oss.sonatype.org/content/repositories/snapshots/oro/oro/maven-metadata.xml Downloading: https://oss.sonatype.org/content/repositories/snapshots/org/beanshell/bsh/maven-metadata.xml Downloading: https://oss.sonatype.org/content/repositories/snapshots/xml-resolver/xml-resolver/maven-metadata.xml - Executing: ValueHolders.java log4j:WARN No appenders could be found for logger (freemarker.cache). log4j:WARN Please initialize the log4j system properly. - Executing: VariableLengthVectors.java - Executing: FixedValueVectors.java - Executing: TypeHelper.java - Executing: NullableValueVectors.java - Executing: RepeatedValueVectors.java [INFO] Done [INFO] [INFO] --- maven-remote-resources-plugin:1.1:process (default) @ java-exec --- [INFO] Setting property: classpath.resource.loader.class => 'org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader'. [INFO] Setting property: velocimacro.messages.on => 'false'. [INFO] Setting property: resource.loader => 'classpath'. [INFO] Setting property: resource.manager.logwhenfound => 'false'. [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ java-exec --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 1 resource [INFO] Copying 3 resources [INFO] [INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @ java-exec --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 460 source files to /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/classes [WARNING] /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetStorageEngine.java:[20,21] com.sun.corba.se.impl.interceptors.CodecFactoryImpl is internal proprietary API and may be removed in a future release [WARNING] /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/bit/BitConnection.java: Some input files use or override a deprecated API. [WARNING] /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/bit/BitConnection.java: Recompile with -Xlint:deprecation for details. [WARNING] /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java: Some input files use unchecked or unsafe operations. [WARNING] /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java: Recompile with -Xlint:unchecked for details. [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ java-exec --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 37 resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-compiler-plugin:3.0:testCompile (default-testCompile) @ java-exec --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 127 source files to /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/test-classes [WARNING] /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[25,16] sun.misc.Unsafe is internal proprietary API and may be removed in a future release [WARNING] /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[53,17] sun.misc.Unsafe is internal proprietary API and may be removed in a future release [WARNING] /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[94,12] sun.misc.Unsafe is internal proprietary API and may be removed in a future release [WARNING] /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[95,14] sun.misc.Unsafe is internal proprietary API and may be removed in a future release [WARNING] /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[98,26] sun.misc.Unsafe is internal proprietary API and may be removed in a future release [WARNING] /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[100,27] sun.misc.Unsafe is internal proprietary API and may be removed in a future release [WARNING] /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/TestParquetPhysicalPlan.java: Some input files use or override a deprecated API. [WARNING] /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/TestParquetPhysicalPlan.java: Recompile with -Xlint:deprecation for details. [WARNING] /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFileGenerator.java: Some input files use unchecked or unsafe operations. [WARNING] /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFileGenerator.java: Recompile with -Xlint:unchecked for details. [INFO] [INFO] --- maven-surefire-plugin:2.15:test (default-test) @ java-exec --- [INFO] Surefire report directory: /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/surefire-reports ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.drill.exec.store.parquet.ParquetRecordReaderTest 09:23:52.569 [main] DEBUG org.apache.hadoop.security.Groups - Creating new Groups object 09:23:52.662 [main] DEBUG org.apache.hadoop.security.Groups - Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping; cacheTimeout=300000 09:23:52.841 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop login 09:23:52.842 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop login commit 09:23:52.863 [main] DEBUG o.a.h.security.UserGroupInformation - using local user:UnixPrincipal: tanujit 09:23:52.867 [main] DEBUG o.a.h.security.UserGroupInformation - UGI loginUser:tanujit 09:23:52.921 [main] DEBUG org.apache.hadoop.fs.FileSystem - Creating filesystem for file:/// 09:23:53.011 [main] DEBUG o.a.hadoop.util.NativeCodeLoader - Trying to load the custom-built native-hadoop library... 09:23:53.033 [main] DEBUG o.a.hadoop.util.NativeCodeLoader - Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path 09:23:53.034 [main] DEBUG o.a.hadoop.util.NativeCodeLoader - java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 09:23:53.034 [main] WARN o.a.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. 09:23:55.443 [main] DEBUG org.reflections.Reflections - going to scan these urls: jar:file:/home/tanujit/.m2/repository/org/apache/drill/common/1.0-SNAPSHOT/common-1.0-SNAPSHOT.jar!/ file:/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/test-classes/ file:/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/classes/ jar:file:/home/tanujit/.m2/repository/org/apache/drill/common/1.0-SNAPSHOT/common-1.0-SNAPSHOT-tests.jar!/ 09:23:56.000 [main] INFO org.reflections.Reflections - Reflections took 556 ms to scan 4 urls, producing 669 keys and 1634 values 09:23:56.028 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J as the default logging framework 09:23:56.037 [main] DEBUG i.n.c.MultithreadEventLoopGroup - -Dio.netty.eventLoopThreads: 8 09:23:56.074 [main] DEBUG i.n.util.internal.PlatformDependent - UID: 1000 09:23:56.075 [main] DEBUG i.n.util.internal.PlatformDependent - Java version: 7 09:23:56.076 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noUnsafe: false 09:23:56.079 [main] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.ByteBuffer.cleaner: available 09:23:56.079 [main] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Buffer.address: available 09:23:56.080 [main] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: true 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent - sun.misc.Unsafe: available 09:23:56.082 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noJavassist: false 09:23:56.145 [main] DEBUG i.n.util.internal.PlatformDependent - Javassist: available 09:23:56.145 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false 09:23:56.164 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.noKeySetOptimization: false 09:23:56.164 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.selectorAutoRebuildThreshold: 512 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL - -Dio.netty.allocator.numHeapArenas: 4 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL - -Dio.netty.allocator.numDirectArenas: 4 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL - -Dio.netty.allocator.pageSize: 8192 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL - -Dio.netty.allocator.maxOrder: 11 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL - -Dio.netty.allocator.chunkSize: 16777216 running bit 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback interface: lo 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback address: /0:0:0:0:0:0:0:1%1 (primary) 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback address: / 127.0.0.1 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - /proc/sys/net/core/somaxconn: 128 09:23:56.288 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id: 0x036be036] REGISTERED 09:23:56.290 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id: 0x036be036] BIND(0.0.0.0/0.0.0.0:31010) 09:23:56.294 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id: 0x036be036, /0:0:0:0:0:0:0:0:31010] ACTIVE 09:23:56.310 [BitServer-1] INFO i.n.handler.logging.LoggingHandler - [id: 0xb8299b35] REGISTERED 09:23:56.311 [BitServer-1] INFO i.n.handler.logging.LoggingHandler - [id: 0xb8299b35] BIND(0.0.0.0/0.0.0.0:31011) 09:23:56.311 [BitServer-1] INFO i.n.handler.logging.LoggingHandler - [id: 0xb8299b35, /0:0:0:0:0:0:0:0:31011] ACTIVE running client connecting - legolas -- 31010 09:23:56.545 [Client-1] DEBUG i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated: io.netty.util.internal.__matchers__.io.netty.buffer.ByteBufMatcher 09:23:56.549 [Client-1] DEBUG i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated: io.netty.util.internal.__matchers__.org.apache.drill.exec.rpc.OutboundRpcMessageMatcher 09:23:56.550 [Client-1] DEBUG i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated: io.netty.util.internal.__matchers__.org.apache.drill.exec.rpc.InboundRpcMessageMatcher 09:23:56.554 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id: 0x036be036, /0:0:0:0:0:0:0:0:31010] RECEIVED: [id: 0xf046bccd, / 192.168.0.103:32952 => /192.168.0.103:31010] 09:23:56.568 [Client-1] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.noResourceLeakDetection: false running the query Screen [endpoint=address: "legolas" user_port: 31010 bit_port: 31011 , getChild()=org.apache.drill.exec.store.parquet.ParquetGroupScan@59c81460] 09:24:00.420 [WorkManager-2] WARN io.netty.util.ResourceLeakDetector - LEAK: ByteBuf was GC'd before being released correctly. The following stack trace shows where the leaked object was created, rather than where you failed to release it. io.netty.util.ResourceLeakException: io.netty.buffer.PooledUnsafeDirectByteBufL@659974ba at io.netty.util.ResourceLeakDetector$DefaultResourceLeak.<init>(ResourceLeakDetector.java:174) ~[netty-common-4.0.7.Final.jar:na] at io.netty.util.ResourceLeakDetector.open(ResourceLeakDetector.java:116) ~[netty-common-4.0.7.Final.jar:na] at io.netty.buffer.PooledByteBufL.<init>(PooledByteBufL.java:45) ~[netty-bufferl-4.0.7.Final.jar:na] at io.netty.buffer.PooledUnsafeDirectByteBufL.<init>(PooledUnsafeDirectByteBufL.java:53) ~[netty-bufferl-4.0.7.Final.jar:na] at io.netty.buffer.PooledUnsafeDirectByteBufL.<init>(PooledUnsafeDirectByteBufL.java:32) ~[netty-bufferl-4.0.7.Final.jar:na] at io.netty.buffer.PooledUnsafeDirectByteBufL$1.newObject(PooledUnsafeDirectByteBufL.java:39) ~[netty-bufferl-4.0.7.Final.jar:na] at io.netty.buffer.PooledUnsafeDirectByteBufL$1.newObject(PooledUnsafeDirectByteBufL.java:36) ~[netty-bufferl-4.0.7.Final.jar:na] at io.netty.util.Recycler.get(Recycler.java:40) ~[netty-common-4.0.7.Final.jar:na] at io.netty.buffer.PooledUnsafeDirectByteBufL.newInstance(PooledUnsafeDirectByteBufL.java:44) ~[netty-bufferl-4.0.7.Final.jar:na] at io.netty.buffer.PoolArenaL$DirectArena.newByteBuf(PoolArenaL.java:397) ~[netty-bufferl-4.0.7.Final.jar:na] at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:93) ~[netty-bufferl-4.0.7.Final.jar:na] at io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:235) ~[netty-bufferl-4.0.7.Final.jar:na] at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:132) ~[netty-buffer-4.0.7.Final.jar:na] at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:123) ~[netty-buffer-4.0.7.Final.jar:na] at org.apache.drill.exec.memory.DirectBufferAllocator.buffer(DirectBufferAllocator.java:35) ~[classes/:na] at org.apache.drill.exec.vector.BitVector.allocateNew(BitVector.java:54) ~[classes/:na] at org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:12) ~[classes/:na] at org.apache.drill.exec.store.VectorHolder.allocateNew(VectorHolder.java:87) ~[classes/:na] at org.apache.drill.exec.store.VectorHolder.reset(VectorHolder.java:74) ~[classes/:na] at org.apache.drill.exec.store.parquet.ParquetRecordReader.resetBatch(ParquetRecordReader.java:276) ~[classes/:na] at org.apache.drill.exec.store.parquet.ParquetRecordReader.next(ParquetRecordReader.java:318) ~[classes/:na] at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:99) ~[classes/:na] at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:77) ~[classes/:na] at org.apache.drill.exec.work.FragmentRunner.run(FragmentRunner.java:79) ~[classes/:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_25] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_25] at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25] Took 10003 ms to run query 09:24:07.614 [BitServer-1] INFO i.n.handler.logging.LoggingHandler - [id: 0xb8299b35, /0:0:0:0:0:0:0:0:31011] UNREGISTERED 09:24:07.614 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id: 0x036be036, /0:0:0:0:0:0:0:0:31010] UNREGISTERED Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.329 sec - in org.apache.drill.exec.store.parquet.ParquetRecordReaderTest 09:24:07.621 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Starting clear of FileSystem cache with 1 elements. 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Removing filesystem for file:/// 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Removing filesystem for file:/// 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Done clearing cache Results : Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 55.501s [INFO] Finished at: Tue Aug 27 09:24:07 IST 2013 [INFO] Final Memory: 36M/429M [INFO] ------------------------------------------------------------------------ -- Regards, Tanujit
