Changed the default in conf from /dev to /tmp as you suggest and that helped!
[root@sgt-pepper conf]# $CRAIL_HOME/bin/crail datanode 20/02/11 08:43:36 INFO crail: crail.version 3101 20/02/11 08:43:36 INFO crail: crail.directorydepth 16 20/02/11 08:43:36 INFO crail: crail.tokenexpiration 10 20/02/11 08:43:36 INFO crail: crail.blocksize 1048576 20/02/11 08:43:36 INFO crail: crail.cachelimit 1073741824 20/02/11 08:43:36 INFO crail: crail.cachepath /tmp/hugepages/cache 20/02/11 08:43:36 INFO crail: crail.user crail 20/02/11 08:43:36 INFO crail: crail.shadowreplication 1 20/02/11 08:43:36 INFO crail: crail.debug false 20/02/11 08:43:36 INFO crail: crail.statistics true 20/02/11 08:43:36 INFO crail: crail.rpctimeout 1000 20/02/11 08:43:36 INFO crail: crail.datatimeout 1000 20/02/11 08:43:36 INFO crail: crail.buffersize 1048576 20/02/11 08:43:36 INFO crail: crail.slicesize 524288 20/02/11 08:43:36 INFO crail: crail.singleton true 20/02/11 08:43:36 INFO crail: crail.regionsize 1073741824 20/02/11 08:43:36 INFO crail: crail.directoryrecord 512 20/02/11 08:43:36 INFO crail: crail.directoryrandomize true 20/02/11 08:43:36 INFO crail: crail.cacheimpl org.apache.crail.memory.MappedBufferCache 20/02/11 08:43:36 INFO crail: crail.locationmap 20/02/11 08:43:36 INFO crail: crail.namenode.address crail://sgt-pepper:9060 20/02/11 08:43:36 INFO crail: crail.namenode.blockselection roundrobin 20/02/11 08:43:36 INFO crail: crail.namenode.fileblocks 16 20/02/11 08:43:36 INFO crail: crail.namenode.rpctype org.apache.crail.namenode.rpc.tcp.TcpNameNode 20/02/11 08:43:36 INFO crail: crail.namenode.log 20/02/11 08:43:36 INFO crail: crail.storage.types org.apache.crail.storage.tcp.TcpStorageTier 20/02/11 08:43:36 INFO crail: crail.storage.classes 1 20/02/11 08:43:36 INFO crail: crail.storage.rootclass 0 20/02/11 08:43:36 INFO crail: crail.storage.keepalive 2 20/02/11 08:43:36 INFO narpc: new NaRPC server group v1.5.0, queueDepth 16, messageSize 2097152, nodealy false, cores 1 20/02/11 08:43:36 INFO crail: crail.storage.tcp.interface eth0 20/02/11 08:43:36 INFO crail: crail.storage.tcp.port 50020 20/02/11 08:43:36 INFO crail: crail.storage.tcp.storagelimit 1073741824 20/02/11 08:43:36 INFO crail: crail.storage.tcp.allocationsize 1073741824 20/02/11 08:43:36 INFO crail: crail.storage.tcp.datapath /tmp/hugepages/data 20/02/11 08:43:36 INFO crail: crail.storage.tcp.queuedepth 16 20/02/11 08:43:36 INFO crail: crail.storage.tcp.cores 1 20/02/11 08:43:36 INFO crail: running TCP storage server, address / 10.114.222.23:50020 20/02/11 08:43:36 INFO narpc: new NaRPC client group v1.5.0, queueDepth 32, messageSize 512, nodealy true 20/02/11 08:43:36 INFO crail: crail.namenode.tcp.queueDepth 32 20/02/11 08:43:36 INFO crail: crail.namenode.tcp.messageSize 512 20/02/11 08:43:36 INFO crail: crail.namenode.tcp.cores 1 20/02/11 08:43:36 INFO crail: connected to namenode(s) sgt-pepper/ 10.114.222.23:9060 20/02/11 08:43:36 INFO crail: datanode statistics, freeBlocks 1024 20/02/11 08:43:37 INFO crail: datanode statistics, freeBlocks 1024 20/02/11 08:43:39 INFO crail: datanode statistics, freeBlocks 1024 20/02/11 08:43:41 INFO crail: datanode statistics, freeBlocks 1024 20/02/11 08:43:43 INFO crail: datanode statistics, freeBlocks 1024 ... Now to see if I can use the CLI... Lou. On Tue, Feb 11, 2020 at 9:40 AM Lou DeGenaro <lou.degen...@gmail.com> wrote: > ...and if it makes any difference: > > [root@sgt-pepper conf]# which java > /usr/bin/java > [root@sgt-pepper conf]# java -version > openjdk version "1.8.0_242" > OpenJDK Runtime Environment (build 1.8.0_242-b08) > OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode) > > [root@sgt-pepper ~]# vmstat -s > 8003404 K total memory > 906212 K used memory > 1658432 K active memory > 2281696 K inactive memory > 3692808 K free memory > 123776 K buffer memory > 3280608 K swap cache > 0 K total swap > 0 K used swap > 0 K free swap > 59564 non-nice user cpu ticks > 359 nice user cpu ticks > 22392 system cpu ticks > 1165850 idle cpu ticks > 9712 IO-wait cpu ticks > 0 IRQ cpu ticks > 1288 softirq cpu ticks > 7522 stolen cpu ticks > 313554 pages paged in > 4620824 pages paged out > 0 pages swapped in > 0 pages swapped out > 12475579 interrupts > 20632577 CPU context switches > 1581425491 boot time > 86760 forks > > Lou. > > On Tue, Feb 11, 2020 at 9:19 AM Lou DeGenaro <lou.degen...@gmail.com> > wrote: > >> I've now removed docker from the picture. I simply download the bin >> distribution onto my 8GB VM and copy the templates in conf w/o changes. >> ====== >> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail >> namenode >> 20/02/11 08:09:08 INFO crail: initalizing namenode >> 20/02/11 08:09:08 INFO crail: crail.version 3101 >> 20/02/11 08:09:08 INFO crail: crail.directorydepth 16 >> 20/02/11 08:09:08 INFO crail: crail.tokenexpiration 10 >> 20/02/11 08:09:08 INFO crail: crail.blocksize 1048576 >> 20/02/11 08:09:08 INFO crail: crail.cachelimit 1073741824 >> 20/02/11 08:09:08 INFO crail: crail.cachepath /dev/hugepages/cache >> 20/02/11 08:09:08 INFO crail: crail.user crail >> 20/02/11 08:09:08 INFO crail: crail.shadowreplication 1 >> 20/02/11 08:09:08 INFO crail: crail.debug false >> 20/02/11 08:09:08 INFO crail: crail.statistics true >> 20/02/11 08:09:08 INFO crail: crail.rpctimeout 1000 >> 20/02/11 08:09:08 INFO crail: crail.datatimeout 1000 >> 20/02/11 08:09:08 INFO crail: crail.buffersize 1048576 >> 20/02/11 08:09:08 INFO crail: crail.slicesize 524288 >> 20/02/11 08:09:08 INFO crail: crail.singleton true >> 20/02/11 08:09:08 INFO crail: crail.regionsize 1073741824 >> 20/02/11 08:09:08 INFO crail: crail.directoryrecord 512 >> 20/02/11 08:09:08 INFO crail: crail.directoryrandomize true >> 20/02/11 08:09:08 INFO crail: crail.cacheimpl >> org.apache.crail.memory.MappedBufferCache >> 20/02/11 08:09:08 INFO crail: crail.locationmap >> 20/02/11 08:09:08 INFO crail: crail.namenode.address >> crail://localhost:9060?id=0&size=1 >> 20/02/11 08:09:08 INFO crail: crail.namenode.blockselection roundrobin >> 20/02/11 08:09:08 INFO crail: crail.namenode.fileblocks 16 >> 20/02/11 08:09:08 INFO crail: crail.namenode.rpctype >> org.apache.crail.namenode.rpc.tcp.TcpNameNode >> 20/02/11 08:09:08 INFO crail: crail.namenode.log >> 20/02/11 08:09:08 INFO crail: crail.storage.types >> org.apache.crail.storage.tcp.TcpStorageTier >> 20/02/11 08:09:08 INFO crail: crail.storage.classes 1 >> 20/02/11 08:09:08 INFO crail: crail.storage.rootclass 0 >> 20/02/11 08:09:08 INFO crail: crail.storage.keepalive 2 >> 20/02/11 08:09:08 INFO crail: round robin block selection >> 20/02/11 08:09:08 INFO narpc: new NaRPC server group v1.5.0, queueDepth >> 32, messageSize 512, nodealy true, cores 1 >> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.queueDepth 32 >> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.messageSize 512 >> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.cores 1 >> 20/02/11 08:11:28 INFO crail: new connection from /127.0.0.1:40578 >> 20/02/11 08:11:28 INFO narpc: adding new channel to selector, from / >> 127.0.0.1:40578 >> ===== >> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail >> datanode >> 20/02/11 08:11:28 INFO crail: crail.version 3101 >> 20/02/11 08:11:28 INFO crail: crail.directorydepth 16 >> 20/02/11 08:11:28 INFO crail: crail.tokenexpiration 10 >> 20/02/11 08:11:28 INFO crail: crail.blocksize 1048576 >> 20/02/11 08:11:28 INFO crail: crail.cachelimit 1073741824 >> 20/02/11 08:11:28 INFO crail: crail.cachepath /dev/hugepages/cache >> 20/02/11 08:11:28 INFO crail: crail.user crail >> 20/02/11 08:11:28 INFO crail: crail.shadowreplication 1 >> 20/02/11 08:11:28 INFO crail: crail.debug false >> 20/02/11 08:11:28 INFO crail: crail.statistics true >> 20/02/11 08:11:28 INFO crail: crail.rpctimeout 1000 >> 20/02/11 08:11:28 INFO crail: crail.datatimeout 1000 >> 20/02/11 08:11:28 INFO crail: crail.buffersize 1048576 >> 20/02/11 08:11:28 INFO crail: crail.slicesize 524288 >> 20/02/11 08:11:28 INFO crail: crail.singleton true >> 20/02/11 08:11:28 INFO crail: crail.regionsize 1073741824 >> 20/02/11 08:11:28 INFO crail: crail.directoryrecord 512 >> 20/02/11 08:11:28 INFO crail: crail.directoryrandomize true >> 20/02/11 08:11:28 INFO crail: crail.cacheimpl >> org.apache.crail.memory.MappedBufferCache >> 20/02/11 08:11:28 INFO crail: crail.locationmap >> 20/02/11 08:11:28 INFO crail: crail.namenode.address >> crail://localhost:9060 >> 20/02/11 08:11:28 INFO crail: crail.namenode.blockselection roundrobin >> 20/02/11 08:11:28 INFO crail: crail.namenode.fileblocks 16 >> 20/02/11 08:11:28 INFO crail: crail.namenode.rpctype >> org.apache.crail.namenode.rpc.tcp.TcpNameNode >> 20/02/11 08:11:28 INFO crail: crail.namenode.log >> 20/02/11 08:11:28 INFO crail: crail.storage.types >> org.apache.crail.storage.tcp.TcpStorageTier >> 20/02/11 08:11:28 INFO crail: crail.storage.classes 1 >> 20/02/11 08:11:28 INFO crail: crail.storage.rootclass 0 >> 20/02/11 08:11:28 INFO crail: crail.storage.keepalive 2 >> 20/02/11 08:11:28 INFO narpc: new NaRPC server group v1.5.0, queueDepth >> 16, messageSize 2097152, nodealy false, cores 1 >> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.interface eth0 >> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.port 50020 >> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.storagelimit 1073741824 >> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.allocationsize 1073741824 >> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.datapath >> /dev/hugepages/data >> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.queuedepth 16 >> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.cores 1 >> 20/02/11 08:11:28 INFO crail: running TCP storage server, address / >> 10.114.222.23:50020 >> 20/02/11 08:11:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth >> 32, messageSize 512, nodealy true >> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.queueDepth 32 >> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.messageSize 512 >> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.cores 1 >> 20/02/11 08:11:28 INFO crail: connected to namenode(s) localhost/ >> 127.0.0.1:9060 >> Exception in thread "main" java.io.IOException: Map failed >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938) >> at >> org.apache.crail.storage.tcp.TcpStorageServer.allocateResource(TcpStorageServer.java:83) >> at org.apache.crail.storage.StorageServer.main(StorageServer.java:152) >> Caused by: java.lang.OutOfMemoryError: Map failed >> at sun.nio.ch.FileChannelImpl.map0(Native Method) >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935) >> ... 2 more >> ===== >> Sorry for being a noob. Thx for the help. >> >> Lou. >> >> On Mon, Feb 10, 2020 at 6:31 PM David Crespi < >> david.cre...@storedgesystems.com> wrote: >> >>> Was does your core-site and crail-site look like. >>> At first thought, it may not be resolving “sgt-pepper”. >>> Why not try the actual IP address to eliminate. >>> >>> Regards, >>> David >>> >>> (C) 714-476-2692 >>> >>> From: Lou DeGenaro<mailto:lou.degen...@gmail.com> >>> Sent: Monday, February 10, 2020 1:12 PM >>> To: dev@crail.apache.org<mailto:dev@crail.apache.org> >>> Subject: Re: iobench >>> >>> I seem to have gone backwards? I started over with new VM. I started 2 >>> docker containers, one each for name and data: >>> >>> [root@sgt-pepper ~]# docker run -it --network host -e >>> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 namenode >>> 20/02/10 20:32:44 INFO crail: initalizing namenode >>> 20/02/10 20:32:44 INFO crail: crail.version 3101 >>> 20/02/10 20:32:44 INFO crail: crail.directorydepth 16 >>> 20/02/10 20:32:44 INFO crail: crail.tokenexpiration 10 >>> 20/02/10 20:32:44 INFO crail: crail.blocksize 1048576 >>> 20/02/10 20:32:44 INFO crail: crail.cachelimit 0 >>> 20/02/10 20:32:44 INFO crail: crail.cachepath /dev/hugepages/cache >>> 20/02/10 20:32:44 INFO crail: crail.user crail >>> 20/02/10 20:32:44 INFO crail: crail.shadowreplication 1 >>> 20/02/10 20:32:44 INFO crail: crail.debug false >>> 20/02/10 20:32:44 INFO crail: crail.statistics true >>> 20/02/10 20:32:44 INFO crail: crail.rpctimeout 1000 >>> 20/02/10 20:32:44 INFO crail: crail.datatimeout 1000 >>> 20/02/10 20:32:44 INFO crail: crail.buffersize 1048576 >>> 20/02/10 20:32:44 INFO crail: crail.slicesize 524288 >>> 20/02/10 20:32:44 INFO crail: crail.singleton true >>> 20/02/10 20:32:44 INFO crail: crail.regionsize 1073741824 >>> 20/02/10 20:32:44 INFO crail: crail.directoryrecord 512 >>> 20/02/10 20:32:44 INFO crail: crail.directoryrandomize true >>> 20/02/10 20:32:44 INFO crail: crail.cacheimpl >>> org.apache.crail.memory.MappedBufferCache >>> 20/02/10 20:32:44 INFO crail: crail.locationmap >>> 20/02/10 20:32:44 INFO crail: crail.namenode.address >>> crail://sgt-pepper:9060?id=0&size=1 >>> 20/02/10 20:32:44 INFO crail: crail.namenode.blockselection roundrobin >>> 20/02/10 20:32:44 INFO crail: crail.namenode.fileblocks 16 >>> 20/02/10 20:32:44 INFO crail: crail.namenode.rpctype >>> org.apache.crail.namenode.rpc.tcp.TcpNameNode >>> 20/02/10 20:32:44 INFO crail: crail.namenode.log >>> 20/02/10 20:32:44 INFO crail: crail.storage.types >>> org.apache.crail.storage.tcp.TcpStorageTier >>> 20/02/10 20:32:44 INFO crail: crail.storage.classes 1 >>> 20/02/10 20:32:44 INFO crail: crail.storage.rootclass 0 >>> 20/02/10 20:32:44 INFO crail: crail.storage.keepalive 2 >>> 20/02/10 20:32:44 INFO crail: round robin block selection >>> 20/02/10 20:32:45 INFO narpc: new NaRPC server group v1.5.0, queueDepth >>> 32, >>> messageSize 512, nodealy true, cores 1 >>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.queueDepth 32 >>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.messageSize 512 >>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.cores 1 >>> 20/02/10 20:35:36 INFO crail: new connection from /10.114.222.82:37328 >>> 20/02/10 20:35:36 INFO narpc: adding new channel to selector, from / >>> 10.114.222.82:37328 >>> 20/02/10 20:35:36 INFO crail: adding datanode /10.114.222.82:50020 of >>> type >>> 0 to storage class 0 >>> >>> [root@sgt-pepper ~]# docker run -it --network host -e >>> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 datanode >>> 20/02/10 20:35:36 INFO crail: crail.version 3101 >>> 20/02/10 20:35:36 INFO crail: crail.directorydepth 16 >>> 20/02/10 20:35:36 INFO crail: crail.tokenexpiration 10 >>> 20/02/10 20:35:36 INFO crail: crail.blocksize 1048576 >>> 20/02/10 20:35:36 INFO crail: crail.cachelimit 0 >>> 20/02/10 20:35:36 INFO crail: crail.cachepath /dev/hugepages/cache >>> 20/02/10 20:35:36 INFO crail: crail.user crail >>> 20/02/10 20:35:36 INFO crail: crail.shadowreplication 1 >>> 20/02/10 20:35:36 INFO crail: crail.debug false >>> 20/02/10 20:35:36 INFO crail: crail.statistics true >>> 20/02/10 20:35:36 INFO crail: crail.rpctimeout 1000 >>> 20/02/10 20:35:36 INFO crail: crail.datatimeout 1000 >>> 20/02/10 20:35:36 INFO crail: crail.buffersize 1048576 >>> 20/02/10 20:35:36 INFO crail: crail.slicesize 524288 >>> 20/02/10 20:35:36 INFO crail: crail.singleton true >>> 20/02/10 20:35:36 INFO crail: crail.regionsize 1073741824 >>> 20/02/10 20:35:36 INFO crail: crail.directoryrecord 512 >>> 20/02/10 20:35:36 INFO crail: crail.directoryrandomize true >>> 20/02/10 20:35:36 INFO crail: crail.cacheimpl >>> org.apache.crail.memory.MappedBufferCache >>> 20/02/10 20:35:36 INFO crail: crail.locationmap >>> 20/02/10 20:35:36 INFO crail: crail.namenode.address >>> crail://sgt-pepper:9060 >>> 20/02/10 20:35:36 INFO crail: crail.namenode.blockselection roundrobin >>> 20/02/10 20:35:36 INFO crail: crail.namenode.fileblocks 16 >>> 20/02/10 20:35:36 INFO crail: crail.namenode.rpctype >>> org.apache.crail.namenode.rpc.tcp.TcpNameNode >>> 20/02/10 20:35:36 INFO crail: crail.namenode.log >>> 20/02/10 20:35:36 INFO crail: crail.storage.types >>> org.apache.crail.storage.tcp.TcpStorageTier >>> 20/02/10 20:35:36 INFO crail: crail.storage.classes 1 >>> 20/02/10 20:35:36 INFO crail: crail.storage.rootclass 0 >>> 20/02/10 20:35:36 INFO crail: crail.storage.keepalive 2 >>> 20/02/10 20:35:36 INFO narpc: new NaRPC server group v1.5.0, queueDepth >>> 16, >>> messageSize 2097152, nodealy false, cores 1 >>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.interface eth0 >>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.port 50020 >>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.storagelimit 1073741824 >>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.allocationsize 1073741824 >>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.datapath >>> /dev/hugepages/data >>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.queuedepth 16 >>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.cores 1 >>> 20/02/10 20:35:36 INFO crail: running TCP storage server, address / >>> 10.114.222.82:50020 >>> 20/02/10 20:35:36 INFO narpc: new NaRPC client group v1.5.0, queueDepth >>> 32, >>> messageSize 512, nodealy true >>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.queueDepth 32 >>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.messageSize 512 >>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.cores 1 >>> 20/02/10 20:35:36 INFO crail: connected to namenode(s) sgt-pepper/ >>> 10.114.222.82:9060 >>> 20/02/10 20:35:36 INFO crail: datanode statistics, freeBlocks 1024 >>> ... >>> >>> This seems to be OK so far, since we see *adding datanode >>> /10.114.222.82:50020 <http://10.114.222.82:50020> of type 0 to storage >>> class 0* in the namenode log. >>> >>> Then I download the Non-Official Binary Release from here: >>> https://crail.apache.org/download/ and I extract and create the 4 conf >>> files and the revised log4j properties file: >>> [root@sgt-pepper conf]# ls -atl >>> total 40 >>> drwxr-xr-x. 2 kube kube 4096 Feb 10 14:14 . >>> -rw-r--r--. 1 kube kube 569 Feb 10 14:14 log4j.properties >>> drwxr-xr-x. 8 root root 4096 Feb 10 13:52 .. >>> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml >>> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml.template >>> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh >>> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh.template >>> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf >>> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf.template >>> -rw-r--r--. 1 kube kube 568 Feb 28 2018 log4j.properties.orig >>> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves >>> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves.template >>> [root@sgt-pepper conf]# diff log4j.properties log4j.properties.orig >>> 2c2 >>> < log4j.rootCategory=DEBUG, console >>> --- >>> > log4j.rootCategory=INFO, console >>> >>> I setup CRAIL_HOME: >>> >>> [root@sgt-pepper apache-crail-1.2-incubating]# export CRAIL_HOME=$PWD >>> [root@sgt-pepper apache-crail-1.2-incubating]# env | grep CRAIL >>> CRAIL_HOME=/usr/local/apache-crail-1.2-incubating >>> >>> ===== >>> >>> Now I try using the CLI: >>> >>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs >>> -ls >>> / >>> 20/02/10 14:54:51 DEBUG Shell: Failed to detect a valid hadoop home >>> directory >>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set. >>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326) >>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351) >>> at >>> >>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440) >>> at >>> >>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486) >>> at >>> >>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170) >>> at >>> >>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153) >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64) >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340) >>> 20/02/10 14:54:51 DEBUG Shell: setsid exited with exit code 0 >>> 20/02/10 14:54:51 DEBUG Configuration: parsing URL >>> >>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml >>> 20/02/10 14:54:51 DEBUG Configuration: parsing input stream >>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8 >>> 20/02/10 14:54:52 DEBUG Configuration: parsing URL >>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml >>> 20/02/10 14:54:52 DEBUG Configuration: parsing input stream >>> java.io.BufferedInputStream@71c7db30 >>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field >>> org.apache.hadoop.metrics2.lib.MutableRate >>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess >>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate >>> of >>> successful kerberos logins and latency (milliseconds)]) >>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field >>> org.apache.hadoop.metrics2.lib.MutableRate >>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure >>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate >>> of >>> failed kerberos logins and latency (milliseconds)]) >>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field >>> org.apache.hadoop.metrics2.lib.MutableRate >>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with >>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, >>> value=[GetGroups]) >>> 20/02/10 14:54:52 DEBUG MetricsSystemImpl: UgiMetrics, User and group >>> related metrics >>> 20/02/10 14:54:52 DEBUG KerberosName: Kerberos krb5 configuration not >>> found, setting default realm to empty >>> 20/02/10 14:54:52 DEBUG Groups: Creating new Groups object >>> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Trying to load the custom-built >>> native-hadoop library... >>> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Failed to load native-hadoop >>> with >>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path >>> 20/02/10 14:54:52 DEBUG NativeCodeLoader: >>> >>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib >>> 20/02/10 14:54:52 WARN NativeCodeLoader: Unable to load native-hadoop >>> library for your platform... using builtin-java classes where applicable >>> 20/02/10 14:54:52 DEBUG PerformanceAdvisory: Falling back to shell based >>> 20/02/10 14:54:52 DEBUG JniBasedUnixGroupsMappingWithFallback: Group >>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping >>> 20/02/10 14:54:52 DEBUG Groups: Group mapping >>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; >>> cacheTimeout=300000; warningDeltaMs=5000 >>> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login >>> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login commit >>> 20/02/10 14:54:52 DEBUG UserGroupInformation: using local >>> user:UnixPrincipal: root >>> 20/02/10 14:54:52 DEBUG UserGroupInformation: Using user: "UnixPrincipal: >>> root" with name root >>> 20/02/10 14:54:52 DEBUG UserGroupInformation: User entry: "root" >>> 20/02/10 14:54:52 DEBUG UserGroupInformation: UGI loginUser:root >>> (auth:SIMPLE) >>> 20/02/10 14:54:52 INFO crail: CrailHadoopFileSystem construction >>> 20/02/10 14:54:52 INFO crail: creating singleton crail file system >>> 20/02/10 14:54:52 INFO crail: crail.version 3101 >>> 20/02/10 14:54:52 INFO crail: crail.directorydepth 16 >>> 20/02/10 14:54:52 INFO crail: crail.tokenexpiration 10 >>> 20/02/10 14:54:52 INFO crail: crail.blocksize 1048576 >>> 20/02/10 14:54:52 INFO crail: crail.cachelimit 1073741824 >>> 20/02/10 14:54:52 INFO crail: crail.cachepath /dev/hugepages/cache >>> 20/02/10 14:54:52 INFO crail: crail.user crail >>> 20/02/10 14:54:52 INFO crail: crail.shadowreplication 1 >>> 20/02/10 14:54:52 INFO crail: crail.debug false >>> 20/02/10 14:54:52 INFO crail: crail.statistics true >>> 20/02/10 14:54:52 INFO crail: crail.rpctimeout 1000 >>> 20/02/10 14:54:52 INFO crail: crail.datatimeout 1000 >>> 20/02/10 14:54:52 INFO crail: crail.buffersize 1048576 >>> 20/02/10 14:54:52 INFO crail: crail.slicesize 524288 >>> 20/02/10 14:54:52 INFO crail: crail.singleton true >>> 20/02/10 14:54:52 INFO crail: crail.regionsize 1073741824 >>> 20/02/10 14:54:52 INFO crail: crail.directoryrecord 512 >>> 20/02/10 14:54:52 INFO crail: crail.directoryrandomize true >>> 20/02/10 14:54:52 INFO crail: crail.cacheimpl >>> org.apache.crail.memory.MappedBufferCache >>> 20/02/10 14:54:52 INFO crail: crail.locationmap >>> 20/02/10 14:54:52 INFO crail: crail.namenode.address >>> crail://localhost:9060 >>> 20/02/10 14:54:52 INFO crail: crail.namenode.blockselection roundrobin >>> 20/02/10 14:54:52 INFO crail: crail.namenode.fileblocks 16 >>> 20/02/10 14:54:52 INFO crail: crail.namenode.rpctype >>> org.apache.crail.namenode.rpc.tcp.TcpNameNode >>> 20/02/10 14:54:52 INFO crail: crail.namenode.log >>> 20/02/10 14:54:52 INFO crail: crail.storage.types >>> org.apache.crail.storage.tcp.TcpStorageTier >>> 20/02/10 14:54:52 INFO crail: crail.storage.classes 1 >>> 20/02/10 14:54:52 INFO crail: crail.storage.rootclass 0 >>> 20/02/10 14:54:52 INFO crail: crail.storage.keepalive 2 >>> 20/02/10 14:54:52 INFO crail: buffer cache, allocationCount 1, >>> bufferCount >>> 1024 >>> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth >>> 16, >>> messageSize 2097152, nodealy false >>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.interface eth0 >>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.port 50020 >>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.storagelimit 1073741824 >>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.allocationsize 1073741824 >>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.datapath >>> /dev/hugepages/data >>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.queuedepth 16 >>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.cores 1 >>> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth >>> 32, >>> messageSize 512, nodealy true >>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.queueDepth 32 >>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.messageSize 512 >>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.cores 1 >>> ls: java.io.IOException: java.net.ConnectException: Connection refused >>> >>> ===== >>> >>> Since I get connection refused, I try changing "localhost" to >>> "sgt-pepper" >>> in the conf files: >>> >>> [root@sgt-pepper conf]# grep sgt-pepper * >>> core-site.xml: <value>crail://sgt-pepper:9060</value> >>> crail-site.conf:crail.namenode.address crail://sgt-pepper:9060 >>> >>> Then I try using the CLI again: >>> >>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs >>> -ls >>> / >>> 20/02/10 15:00:27 DEBUG Shell: Failed to detect a valid hadoop home >>> directory >>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set. >>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326) >>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351) >>> at >>> >>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440) >>> at >>> >>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486) >>> at >>> >>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170) >>> at >>> >>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153) >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64) >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340) >>> 20/02/10 15:00:27 DEBUG Shell: setsid exited with exit code 0 >>> 20/02/10 15:00:27 DEBUG Configuration: parsing URL >>> >>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml >>> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream >>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8 >>> 20/02/10 15:00:27 DEBUG Configuration: parsing URL >>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml >>> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream >>> java.io.BufferedInputStream@71c7db30 >>> 20/02/10 15:00:27 DEBUG MutableMetricsFactory: field >>> org.apache.hadoop.metrics2.lib.MutableRate >>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess >>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate >>> of >>> successful kerberos logins and latency (milliseconds)]) >>> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field >>> org.apache.hadoop.metrics2.lib.MutableRate >>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure >>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate >>> of >>> failed kerberos logins and latency (milliseconds)]) >>> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field >>> org.apache.hadoop.metrics2.lib.MutableRate >>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with >>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, >>> value=[GetGroups]) >>> 20/02/10 15:00:28 DEBUG MetricsSystemImpl: UgiMetrics, User and group >>> related metrics >>> 20/02/10 15:00:28 DEBUG KerberosName: Kerberos krb5 configuration not >>> found, setting default realm to empty >>> 20/02/10 15:00:28 DEBUG Groups: Creating new Groups object >>> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Trying to load the custom-built >>> native-hadoop library... >>> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Failed to load native-hadoop >>> with >>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path >>> 20/02/10 15:00:28 DEBUG NativeCodeLoader: >>> >>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib >>> 20/02/10 15:00:28 WARN NativeCodeLoader: Unable to load native-hadoop >>> library for your platform... using builtin-java classes where applicable >>> 20/02/10 15:00:28 DEBUG PerformanceAdvisory: Falling back to shell based >>> 20/02/10 15:00:28 DEBUG JniBasedUnixGroupsMappingWithFallback: Group >>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping >>> 20/02/10 15:00:28 DEBUG Groups: Group mapping >>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; >>> cacheTimeout=300000; warningDeltaMs=5000 >>> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login >>> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login commit >>> 20/02/10 15:00:28 DEBUG UserGroupInformation: using local >>> user:UnixPrincipal: root >>> 20/02/10 15:00:28 DEBUG UserGroupInformation: Using user: "UnixPrincipal: >>> root" with name root >>> 20/02/10 15:00:28 DEBUG UserGroupInformation: User entry: "root" >>> 20/02/10 15:00:28 DEBUG UserGroupInformation: UGI loginUser:root >>> (auth:SIMPLE) >>> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem construction >>> 20/02/10 15:00:28 INFO crail: creating singleton crail file system >>> 20/02/10 15:00:28 INFO crail: crail.version 3101 >>> 20/02/10 15:00:28 INFO crail: crail.directorydepth 16 >>> 20/02/10 15:00:28 INFO crail: crail.tokenexpiration 10 >>> 20/02/10 15:00:28 INFO crail: crail.blocksize 1048576 >>> 20/02/10 15:00:28 INFO crail: crail.cachelimit 1073741824 >>> 20/02/10 15:00:28 INFO crail: crail.cachepath /dev/hugepages/cache >>> 20/02/10 15:00:28 INFO crail: crail.user crail >>> 20/02/10 15:00:28 INFO crail: crail.shadowreplication 1 >>> 20/02/10 15:00:28 INFO crail: crail.debug false >>> 20/02/10 15:00:28 INFO crail: crail.statistics true >>> 20/02/10 15:00:28 INFO crail: crail.rpctimeout 1000 >>> 20/02/10 15:00:28 INFO crail: crail.datatimeout 1000 >>> 20/02/10 15:00:28 INFO crail: crail.buffersize 1048576 >>> 20/02/10 15:00:28 INFO crail: crail.slicesize 524288 >>> 20/02/10 15:00:28 INFO crail: crail.singleton true >>> 20/02/10 15:00:28 INFO crail: crail.regionsize 1073741824 >>> 20/02/10 15:00:28 INFO crail: crail.directoryrecord 512 >>> 20/02/10 15:00:28 INFO crail: crail.directoryrandomize true >>> 20/02/10 15:00:28 INFO crail: crail.cacheimpl >>> org.apache.crail.memory.MappedBufferCache >>> 20/02/10 15:00:28 INFO crail: crail.locationmap >>> 20/02/10 15:00:28 INFO crail: crail.namenode.address >>> crail://sgt-pepper:9060 >>> 20/02/10 15:00:28 INFO crail: crail.namenode.blockselection roundrobin >>> 20/02/10 15:00:28 INFO crail: crail.namenode.fileblocks 16 >>> 20/02/10 15:00:28 INFO crail: crail.namenode.rpctype >>> org.apache.crail.namenode.rpc.tcp.TcpNameNode >>> 20/02/10 15:00:28 INFO crail: crail.namenode.log >>> 20/02/10 15:00:28 INFO crail: crail.storage.types >>> org.apache.crail.storage.tcp.TcpStorageTier >>> 20/02/10 15:00:28 INFO crail: crail.storage.classes 1 >>> 20/02/10 15:00:28 INFO crail: crail.storage.rootclass 0 >>> 20/02/10 15:00:28 INFO crail: crail.storage.keepalive 2 >>> 20/02/10 15:00:28 INFO crail: buffer cache, allocationCount 1, >>> bufferCount >>> 1024 >>> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth >>> 16, >>> messageSize 2097152, nodealy false >>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.interface eth0 >>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.port 50020 >>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.storagelimit 1073741824 >>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.allocationsize 1073741824 >>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.datapath >>> /dev/hugepages/data >>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.queuedepth 16 >>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.cores 1 >>> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth >>> 32, >>> messageSize 512, nodealy true >>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.queueDepth 32 >>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.messageSize 512 >>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.cores 1 >>> 20/02/10 15:00:28 INFO crail: connected to namenode(s) sgt-pepper/ >>> 10.114.222.82:9060 >>> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem fs initialization >>> done.. >>> ls: / >>> 20/02/10 15:00:28 INFO crail: Closing CrailHadoopFileSystem >>> 20/02/10 15:00:28 INFO crail: Closing CrailFS singleton >>> 20/02/10 15:00:28 INFO crail: mapped client cache closed >>> >>> Seem better, but not really. >>> >>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs >>> -mkdir /foobar >>> 20/02/10 15:06:14 DEBUG Shell: Failed to detect a valid hadoop home >>> directory >>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set. >>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326) >>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351) >>> at >>> >>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440) >>> at >>> >>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486) >>> at >>> >>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170) >>> at >>> >>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153) >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64) >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340) >>> 20/02/10 15:06:14 DEBUG Shell: setsid exited with exit code 0 >>> 20/02/10 15:06:14 DEBUG Configuration: parsing URL >>> >>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml >>> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream >>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8 >>> 20/02/10 15:06:14 DEBUG Configuration: parsing URL >>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml >>> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream >>> java.io.BufferedInputStream@71c7db30 >>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field >>> org.apache.hadoop.metrics2.lib.MutableRate >>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess >>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate >>> of >>> successful kerberos logins and latency (milliseconds)]) >>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field >>> org.apache.hadoop.metrics2.lib.MutableRate >>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure >>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate >>> of >>> failed kerberos logins and latency (milliseconds)]) >>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field >>> org.apache.hadoop.metrics2.lib.MutableRate >>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with >>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, >>> value=[GetGroups]) >>> 20/02/10 15:06:14 DEBUG MetricsSystemImpl: UgiMetrics, User and group >>> related metrics >>> 20/02/10 15:06:14 DEBUG KerberosName: Kerberos krb5 configuration not >>> found, setting default realm to empty >>> 20/02/10 15:06:14 DEBUG Groups: Creating new Groups object >>> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Trying to load the custom-built >>> native-hadoop library... >>> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Failed to load native-hadoop >>> with >>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path >>> 20/02/10 15:06:14 DEBUG NativeCodeLoader: >>> >>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib >>> 20/02/10 15:06:14 WARN NativeCodeLoader: Unable to load native-hadoop >>> library for your platform... using builtin-java classes where applicable >>> 20/02/10 15:06:14 DEBUG PerformanceAdvisory: Falling back to shell based >>> 20/02/10 15:06:14 DEBUG JniBasedUnixGroupsMappingWithFallback: Group >>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping >>> 20/02/10 15:06:14 DEBUG Groups: Group mapping >>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; >>> cacheTimeout=300000; warningDeltaMs=5000 >>> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login >>> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login commit >>> 20/02/10 15:06:14 DEBUG UserGroupInformation: using local >>> user:UnixPrincipal: root >>> 20/02/10 15:06:14 DEBUG UserGroupInformation: Using user: "UnixPrincipal: >>> root" with name root >>> 20/02/10 15:06:14 DEBUG UserGroupInformation: User entry: "root" >>> 20/02/10 15:06:14 DEBUG UserGroupInformation: UGI loginUser:root >>> (auth:SIMPLE) >>> 20/02/10 15:06:14 INFO crail: CrailHadoopFileSystem construction >>> 20/02/10 15:06:14 INFO crail: creating singleton crail file system >>> 20/02/10 15:06:14 INFO crail: crail.version 3101 >>> 20/02/10 15:06:14 INFO crail: crail.directorydepth 16 >>> 20/02/10 15:06:14 INFO crail: crail.tokenexpiration 10 >>> 20/02/10 15:06:14 INFO crail: crail.blocksize 1048576 >>> 20/02/10 15:06:14 INFO crail: crail.cachelimit 1073741824 >>> 20/02/10 15:06:14 INFO crail: crail.cachepath /dev/hugepages/cache >>> 20/02/10 15:06:14 INFO crail: crail.user crail >>> 20/02/10 15:06:14 INFO crail: crail.shadowreplication 1 >>> 20/02/10 15:06:14 INFO crail: crail.debug false >>> 20/02/10 15:06:14 INFO crail: crail.statistics true >>> 20/02/10 15:06:14 INFO crail: crail.rpctimeout 1000 >>> 20/02/10 15:06:14 INFO crail: crail.datatimeout 1000 >>> 20/02/10 15:06:14 INFO crail: crail.buffersize 1048576 >>> 20/02/10 15:06:14 INFO crail: crail.slicesize 524288 >>> 20/02/10 15:06:14 INFO crail: crail.singleton true >>> 20/02/10 15:06:14 INFO crail: crail.regionsize 1073741824 >>> 20/02/10 15:06:14 INFO crail: crail.directoryrecord 512 >>> 20/02/10 15:06:14 INFO crail: crail.directoryrandomize true >>> 20/02/10 15:06:14 INFO crail: crail.cacheimpl >>> org.apache.crail.memory.MappedBufferCache >>> 20/02/10 15:06:14 INFO crail: crail.locationmap >>> 20/02/10 15:06:14 INFO crail: crail.namenode.address >>> crail://sgt-pepper:9060 >>> 20/02/10 15:06:14 INFO crail: crail.namenode.blockselection roundrobin >>> 20/02/10 15:06:14 INFO crail: crail.namenode.fileblocks 16 >>> 20/02/10 15:06:14 INFO crail: crail.namenode.rpctype >>> org.apache.crail.namenode.rpc.tcp.TcpNameNode >>> 20/02/10 15:06:14 INFO crail: crail.namenode.log >>> 20/02/10 15:06:14 INFO crail: crail.storage.types >>> org.apache.crail.storage.tcp.TcpStorageTier >>> 20/02/10 15:06:14 INFO crail: crail.storage.classes 1 >>> 20/02/10 15:06:14 INFO crail: crail.storage.rootclass 0 >>> 20/02/10 15:06:14 INFO crail: crail.storage.keepalive 2 >>> 20/02/10 15:06:14 INFO crail: buffer cache, allocationCount 1, >>> bufferCount >>> 1024 >>> 20/02/10 15:06:14 INFO narpc: new NaRPC client group v1.5.0, queueDepth >>> 16, >>> messageSize 2097152, nodealy false >>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.interface eth0 >>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.port 50020 >>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.storagelimit 1073741824 >>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.allocationsize 1073741824 >>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.datapath >>> /dev/hugepages/data >>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.queuedepth 16 >>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.cores 1 >>> 20/02/10 15:06:15 INFO narpc: new NaRPC client group v1.5.0, queueDepth >>> 32, >>> messageSize 512, nodealy true >>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.queueDepth 32 >>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.messageSize 512 >>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.cores 1 >>> 20/02/10 15:06:15 INFO crail: connected to namenode(s) sgt-pepper/ >>> 10.114.222.82:9060 >>> 20/02/10 15:06:15 INFO crail: CrailHadoopFileSystem fs initialization >>> done.. >>> mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map >>> failed >>> 20/02/10 15:06:15 INFO crail: Closing CrailHadoopFileSystem >>> 20/02/10 15:06:15 INFO crail: Closing CrailFS singleton >>> 20/02/10 15:06:15 INFO crail: mapped client cache closed >>> >>> I'm thinkinh that *mkdir: java.util.concurrent.ExecutionException: >>> java.io.IOException: Map failed *is not good. >>> >>> Sorry for the long append. Probably something really dumb, but I'm >>> sratching my head... >>> >>> Lou. >>> >>> >>> On Mon, Feb 10, 2020 at 12:13 PM David Crespi < >>> david.cre...@storedgesystems.com> wrote: >>> >>> > Looks like both cmds are working, but it’s not really finding the >>> > datastore. >>> > You’re getting info into the namenode, but not the datanode (which >>> means >>> > you’re really not writing to the datanode). The >>> > First warning you can ignore (WARN NativeCodeLoader: Unable to load >>> > native-hadoop library for your platform) >>> > >>> > you may want to turn on debug to see more of what’s going on. >>> > Edit /crail/conf/log4j.properties and change from INFO to DEBUG. >>> > >>> > You should probably also look at the individual docker logs. >>> > docker logs YourNamenodeName & docker logs YourDatanodeName >>> > >>> > may give you some more hints of what’s happening. >>> > >>> > >>> > Regards, >>> > David >>> > >>> > >>> > From: Lou DeGenaro<mailto:lou.degen...@gmail.com> >>> > Sent: Monday, February 10, 2020 8:12 AM >>> > To: dev@crail.apache.org<mailto:dev@crail.apache.org> >>> > Subject: Re: iobench >>> > >>> > OK, I take it back. Having trouble with crail fs. >>> > >>> > ===== >>> > >>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar >>> > 20/02/10 10:07:16 WARN NativeCodeLoader: Unable to load native-hadoop >>> > library for your platform... using builtin-java classes where >>> applicable >>> > ... >>> > 20/02/10 10:07:16 INFO crail: connected to namenode(s) abbey-road/ >>> > 10.114.222.23:9060 >>> > 20/02/10 10:07:16 INFO crail: CrailHadoopFileSystem fs initialization >>> > done.. >>> > mkdir: java.util.concurrent.ExecutionException: java.io.IOException: >>> Map >>> > failed >>> > 20/02/10 10:07:16 INFO crail: Closing CrailHadoopFileSystem >>> > 20/02/10 10:07:16 INFO crail: Closing CrailFS singleton >>> > 20/02/10 10:07:16 INFO crail: mapped client cache closed >>> > >>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -rmdir /foobar >>> > 20/02/10 10:08:55 WARN NativeCodeLoader: Unable to load native-hadoop >>> > library for your platform... using builtin-java classes where >>> applicable >>> > ...20/02/10 10:08:55 INFO crail: CrailHadoopFileSystem fs >>> initialization >>> > done.. >>> > rmdir: /foobar >>> > 20/02/10 10:08:55 INFO crail: Closing CrailHadoopFileSystem >>> > 20/02/10 10:08:55 INFO crail: Closing CrailFS singleton >>> > 20/02/10 10:08:55 INFO crail: mapped client cache closed >>> > >>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar >>> > 20/02/10 10:09:43 WARN NativeCodeLoader: Unable to load native-hadoop >>> > library for your platform... using builtin-java classes where >>> applicable >>> > ... >>> > 20/02/10 10:09:43 INFO crail: CrailHadoopFileSystem fs initialization >>> > done.. >>> > mkdir: `/foobar': File exists >>> > 20/02/10 10:09:43 INFO crail: Closing CrailHadoopFileSystem >>> > 20/02/10 10:09:43 INFO crail: Closing CrailFS singleton >>> > 20/02/10 10:09:43 INFO crail: mapped client cache closed >>> > >>> > ===== >>> > >>> > Seems that mkdir works, but rmdir does not? Also, IOException was >>> reported >>> > on mkdir, which seems worrisome. >>> > >>> > Lou. >>> > >>> > On Mon, Feb 10, 2020 at 8:16 AM Lou DeGenaro <lou.degen...@gmail.com> >>> > wrote: >>> > >>> > > David, >>> > > >>> > > Thanks. I'm able to use *crail fs *successfully to >>> create/list/delete. >>> > > Will try to increase available storage space next... >>> > > >>> > > Lou. >>> > > >>> > > On Fri, Feb 7, 2020 at 6:37 PM David Crespi < >>> > > david.cre...@storedgesystems.com> wrote: >>> > > >>> > >> Oh, and it may have already written something on the datastore, so >>> you’d >>> > >> need >>> > >> to check it and remove it with the fs command anyway. >>> > >> >>> > >> crail fs -ls -R / >>> > >> >>> > >> would show you what you have there. >>> > >> >>> > >> Regards, >>> > >> David >>> > >> >>> > >> (C) 714-476-2692 >>> > >> >>> > >> From: Lou DeGenaro<mailto:lou.degen...@gmail.com> >>> > >> Sent: Friday, February 7, 2020 12:15 PM >>> > >> To: dev@crail.apache.org<mailto:dev@crail.apache.org> >>> > >> Subject: iobench >>> > >> >>> > >> Still a noob. Got namenode and datanode running as docker images >>> on my >>> > >> VM. I'm looking for the simplest example of writing something then >>> > >> reading >>> > >> something. >>> > >> >>> > >> [root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write -f >>> > >> /filename >>> > >> -s 1024 -k 1 >>> > >> 20/02/07 14:10:22 INFO crail: creating singleton crail file system >>> > >> 20/02/07 14:10:23 INFO crail: crail.version 3101 >>> > >> 20/02/07 14:10:23 INFO crail: crail.directorydepth 16 >>> > >> 20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10 >>> > >> 20/02/07 14:10:23 INFO crail: crail.blocksize 1048576 >>> > >> 20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824 >>> > >> 20/02/07 14:10:23 INFO crail: crail.cachepath /dev/hugepages/cache >>> > >> 20/02/07 14:10:23 INFO crail: crail.user crail >>> > >> 20/02/07 14:10:23 INFO crail: crail.shadowreplication 1 >>> > >> 20/02/07 14:10:23 INFO crail: crail.debug false >>> > >> 20/02/07 14:10:23 INFO crail: crail.statistics true >>> > >> 20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000 >>> > >> 20/02/07 14:10:23 INFO crail: crail.datatimeout 1000 >>> > >> 20/02/07 14:10:23 INFO crail: crail.buffersize 1048576 >>> > >> 20/02/07 14:10:23 INFO crail: crail.slicesize 524288 >>> > >> 20/02/07 14:10:23 INFO crail: crail.singleton true >>> > >> 20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824 >>> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrecord 512 >>> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrandomize true >>> > >> 20/02/07 14:10:23 INFO crail: crail.cacheimpl >>> > >> org.apache.crail.memory.MappedBufferCache >>> > >> 20/02/07 14:10:23 INFO crail: crail.locationmap >>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.address >>> > >> crail://abbey-road:9060 >>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.blockselection >>> roundrobin >>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16 >>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.rpctype >>> > >> org.apache.crail.namenode.rpc.tcp.TcpNameNode >>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.log >>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.types >>> > >> org.apache.crail.storage.tcp.TcpStorageTier >>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.classes 1 >>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0 >>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2 >>> > >> 20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1, >>> > bufferCount >>> > >> 1024 >>> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, >>> queueDepth >>> > >> 16, >>> > >> messageSize 2097152, nodealy false >>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0 >>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020 >>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit >>> 1073741824 >>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize >>> > 1073741824 >>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath >>> > >> /dev/hugepages/data >>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16 >>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1 >>> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, >>> queueDepth >>> > >> 32, >>> > >> messageSize 512, nodealy true >>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32 >>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512 >>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1 >>> > >> 20/02/07 14:10:23 INFO crail: connected to namenode(s) abbey-road/ >>> > >> 10.114.222.23:9060 >>> > >> write, filename /filename, size 1024, loop 1, storageClass 0, >>> > >> locationClass >>> > >> 0, buffered true >>> > >> Exception in thread "main" java.io.IOException: Map failed >>> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938) >>> > >> at >>> > >> >>> > >> >>> > >>> org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94) >>> > >> at >>> > org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95) >>> > >> at >>> > >> >>> > >>> org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482) >>> > >> at >>> org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87) >>> > >> at >>> org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070) >>> > >> Caused by: java.lang.OutOfMemoryError: Map failed >>> > >> at sun.nio.ch.FileChannelImpl.map0(Native Method) >>> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935) >>> > >> ... 5 more >>> > >> >>> > >> >>> > >>> > >>> >>>
