Congrats and no worries… I doubt you’ll have a problem with docker. Happy Crailing 😉
Regards, David From: Lou DeGenaro<mailto:lou.degen...@gmail.com> Sent: Tuesday, February 11, 2020 7:03 AM To: dev@crail.apache.org<mailto:dev@crail.apache.org> Subject: Re: iobench Success! [root@sgt-pepper ~]# $CRAIL_HOME/bin/crail fs -ls /foo ... Found 1 items -rw-rw-rw- 1 crail crail 1079 2020-02-11 08:58 /foo/favicon.png Thanks for the help and suggestions. Now to try all over again with docker... Lou. On Tue, Feb 11, 2020 at 9:47 AM Lou DeGenaro <lou.degen...@gmail.com> wrote: > Changed the default in conf from /dev to /tmp as you suggest and that > helped! > > [root@sgt-pepper conf]# $CRAIL_HOME/bin/crail datanode > 20/02/11 08:43:36 INFO crail: crail.version 3101 > 20/02/11 08:43:36 INFO crail: crail.directorydepth 16 > 20/02/11 08:43:36 INFO crail: crail.tokenexpiration 10 > 20/02/11 08:43:36 INFO crail: crail.blocksize 1048576 > 20/02/11 08:43:36 INFO crail: crail.cachelimit 1073741824 > 20/02/11 08:43:36 INFO crail: crail.cachepath /tmp/hugepages/cache > 20/02/11 08:43:36 INFO crail: crail.user crail > 20/02/11 08:43:36 INFO crail: crail.shadowreplication 1 > 20/02/11 08:43:36 INFO crail: crail.debug false > 20/02/11 08:43:36 INFO crail: crail.statistics true > 20/02/11 08:43:36 INFO crail: crail.rpctimeout 1000 > 20/02/11 08:43:36 INFO crail: crail.datatimeout 1000 > 20/02/11 08:43:36 INFO crail: crail.buffersize 1048576 > 20/02/11 08:43:36 INFO crail: crail.slicesize 524288 > 20/02/11 08:43:36 INFO crail: crail.singleton true > 20/02/11 08:43:36 INFO crail: crail.regionsize 1073741824 > 20/02/11 08:43:36 INFO crail: crail.directoryrecord 512 > 20/02/11 08:43:36 INFO crail: crail.directoryrandomize true > 20/02/11 08:43:36 INFO crail: crail.cacheimpl > org.apache.crail.memory.MappedBufferCache > 20/02/11 08:43:36 INFO crail: crail.locationmap > 20/02/11 08:43:36 INFO crail: crail.namenode.address > crail://sgt-pepper:9060 > 20/02/11 08:43:36 INFO crail: crail.namenode.blockselection roundrobin > 20/02/11 08:43:36 INFO crail: crail.namenode.fileblocks 16 > 20/02/11 08:43:36 INFO crail: crail.namenode.rpctype > org.apache.crail.namenode.rpc.tcp.TcpNameNode > 20/02/11 08:43:36 INFO crail: crail.namenode.log > 20/02/11 08:43:36 INFO crail: crail.storage.types > org.apache.crail.storage.tcp.TcpStorageTier > 20/02/11 08:43:36 INFO crail: crail.storage.classes 1 > 20/02/11 08:43:36 INFO crail: crail.storage.rootclass 0 > 20/02/11 08:43:36 INFO crail: crail.storage.keepalive 2 > 20/02/11 08:43:36 INFO narpc: new NaRPC server group v1.5.0, queueDepth > 16, messageSize 2097152, nodealy false, cores 1 > 20/02/11 08:43:36 INFO crail: crail.storage.tcp.interface eth0 > 20/02/11 08:43:36 INFO crail: crail.storage.tcp.port 50020 > 20/02/11 08:43:36 INFO crail: crail.storage.tcp.storagelimit 1073741824 > 20/02/11 08:43:36 INFO crail: crail.storage.tcp.allocationsize 1073741824 > 20/02/11 08:43:36 INFO crail: crail.storage.tcp.datapath > /tmp/hugepages/data > 20/02/11 08:43:36 INFO crail: crail.storage.tcp.queuedepth 16 > 20/02/11 08:43:36 INFO crail: crail.storage.tcp.cores 1 > 20/02/11 08:43:36 INFO crail: running TCP storage server, address / > 10.114.222.23:50020 > 20/02/11 08:43:36 INFO narpc: new NaRPC client group v1.5.0, queueDepth > 32, messageSize 512, nodealy true > 20/02/11 08:43:36 INFO crail: crail.namenode.tcp.queueDepth 32 > 20/02/11 08:43:36 INFO crail: crail.namenode.tcp.messageSize 512 > 20/02/11 08:43:36 INFO crail: crail.namenode.tcp.cores 1 > 20/02/11 08:43:36 INFO crail: connected to namenode(s) sgt-pepper/ > 10.114.222.23:9060 > 20/02/11 08:43:36 INFO crail: datanode statistics, freeBlocks 1024 > 20/02/11 08:43:37 INFO crail: datanode statistics, freeBlocks 1024 > 20/02/11 08:43:39 INFO crail: datanode statistics, freeBlocks 1024 > 20/02/11 08:43:41 INFO crail: datanode statistics, freeBlocks 1024 > 20/02/11 08:43:43 INFO crail: datanode statistics, freeBlocks 1024 > ... > > Now to see if I can use the CLI... > > Lou. > > > > On Tue, Feb 11, 2020 at 9:40 AM Lou DeGenaro <lou.degen...@gmail.com> > wrote: > >> ...and if it makes any difference: >> >> [root@sgt-pepper conf]# which java >> /usr/bin/java >> [root@sgt-pepper conf]# java -version >> openjdk version "1.8.0_242" >> OpenJDK Runtime Environment (build 1.8.0_242-b08) >> OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode) >> >> [root@sgt-pepper ~]# vmstat -s >> 8003404 K total memory >> 906212 K used memory >> 1658432 K active memory >> 2281696 K inactive memory >> 3692808 K free memory >> 123776 K buffer memory >> 3280608 K swap cache >> 0 K total swap >> 0 K used swap >> 0 K free swap >> 59564 non-nice user cpu ticks >> 359 nice user cpu ticks >> 22392 system cpu ticks >> 1165850 idle cpu ticks >> 9712 IO-wait cpu ticks >> 0 IRQ cpu ticks >> 1288 softirq cpu ticks >> 7522 stolen cpu ticks >> 313554 pages paged in >> 4620824 pages paged out >> 0 pages swapped in >> 0 pages swapped out >> 12475579 interrupts >> 20632577 CPU context switches >> 1581425491 boot time >> 86760 forks >> >> Lou. >> >> On Tue, Feb 11, 2020 at 9:19 AM Lou DeGenaro <lou.degen...@gmail.com> >> wrote: >> >>> I've now removed docker from the picture. I simply download the bin >>> distribution onto my 8GB VM and copy the templates in conf w/o changes. >>> ====== >>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail >>> namenode >>> 20/02/11 08:09:08 INFO crail: initalizing namenode >>> 20/02/11 08:09:08 INFO crail: crail.version 3101 >>> 20/02/11 08:09:08 INFO crail: crail.directorydepth 16 >>> 20/02/11 08:09:08 INFO crail: crail.tokenexpiration 10 >>> 20/02/11 08:09:08 INFO crail: crail.blocksize 1048576 >>> 20/02/11 08:09:08 INFO crail: crail.cachelimit 1073741824 >>> 20/02/11 08:09:08 INFO crail: crail.cachepath /dev/hugepages/cache >>> 20/02/11 08:09:08 INFO crail: crail.user crail >>> 20/02/11 08:09:08 INFO crail: crail.shadowreplication 1 >>> 20/02/11 08:09:08 INFO crail: crail.debug false >>> 20/02/11 08:09:08 INFO crail: crail.statistics true >>> 20/02/11 08:09:08 INFO crail: crail.rpctimeout 1000 >>> 20/02/11 08:09:08 INFO crail: crail.datatimeout 1000 >>> 20/02/11 08:09:08 INFO crail: crail.buffersize 1048576 >>> 20/02/11 08:09:08 INFO crail: crail.slicesize 524288 >>> 20/02/11 08:09:08 INFO crail: crail.singleton true >>> 20/02/11 08:09:08 INFO crail: crail.regionsize 1073741824 >>> 20/02/11 08:09:08 INFO crail: crail.directoryrecord 512 >>> 20/02/11 08:09:08 INFO crail: crail.directoryrandomize true >>> 20/02/11 08:09:08 INFO crail: crail.cacheimpl >>> org.apache.crail.memory.MappedBufferCache >>> 20/02/11 08:09:08 INFO crail: crail.locationmap >>> 20/02/11 08:09:08 INFO crail: crail.namenode.address >>> crail://localhost:9060?id=0&size=1 >>> 20/02/11 08:09:08 INFO crail: crail.namenode.blockselection roundrobin >>> 20/02/11 08:09:08 INFO crail: crail.namenode.fileblocks 16 >>> 20/02/11 08:09:08 INFO crail: crail.namenode.rpctype >>> org.apache.crail.namenode.rpc.tcp.TcpNameNode >>> 20/02/11 08:09:08 INFO crail: crail.namenode.log >>> 20/02/11 08:09:08 INFO crail: crail.storage.types >>> org.apache.crail.storage.tcp.TcpStorageTier >>> 20/02/11 08:09:08 INFO crail: crail.storage.classes 1 >>> 20/02/11 08:09:08 INFO crail: crail.storage.rootclass 0 >>> 20/02/11 08:09:08 INFO crail: crail.storage.keepalive 2 >>> 20/02/11 08:09:08 INFO crail: round robin block selection >>> 20/02/11 08:09:08 INFO narpc: new NaRPC server group v1.5.0, queueDepth >>> 32, messageSize 512, nodealy true, cores 1 >>> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.queueDepth 32 >>> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.messageSize 512 >>> 20/02/11 08:09:08 INFO crail: crail.namenode.tcp.cores 1 >>> 20/02/11 08:11:28 INFO crail: new connection from /127.0.0.1:40578 >>> 20/02/11 08:11:28 INFO narpc: adding new channel to selector, from / >>> 127.0.0.1:40578 >>> ===== >>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail >>> datanode >>> 20/02/11 08:11:28 INFO crail: crail.version 3101 >>> 20/02/11 08:11:28 INFO crail: crail.directorydepth 16 >>> 20/02/11 08:11:28 INFO crail: crail.tokenexpiration 10 >>> 20/02/11 08:11:28 INFO crail: crail.blocksize 1048576 >>> 20/02/11 08:11:28 INFO crail: crail.cachelimit 1073741824 >>> 20/02/11 08:11:28 INFO crail: crail.cachepath /dev/hugepages/cache >>> 20/02/11 08:11:28 INFO crail: crail.user crail >>> 20/02/11 08:11:28 INFO crail: crail.shadowreplication 1 >>> 20/02/11 08:11:28 INFO crail: crail.debug false >>> 20/02/11 08:11:28 INFO crail: crail.statistics true >>> 20/02/11 08:11:28 INFO crail: crail.rpctimeout 1000 >>> 20/02/11 08:11:28 INFO crail: crail.datatimeout 1000 >>> 20/02/11 08:11:28 INFO crail: crail.buffersize 1048576 >>> 20/02/11 08:11:28 INFO crail: crail.slicesize 524288 >>> 20/02/11 08:11:28 INFO crail: crail.singleton true >>> 20/02/11 08:11:28 INFO crail: crail.regionsize 1073741824 >>> 20/02/11 08:11:28 INFO crail: crail.directoryrecord 512 >>> 20/02/11 08:11:28 INFO crail: crail.directoryrandomize true >>> 20/02/11 08:11:28 INFO crail: crail.cacheimpl >>> org.apache.crail.memory.MappedBufferCache >>> 20/02/11 08:11:28 INFO crail: crail.locationmap >>> 20/02/11 08:11:28 INFO crail: crail.namenode.address >>> crail://localhost:9060 >>> 20/02/11 08:11:28 INFO crail: crail.namenode.blockselection roundrobin >>> 20/02/11 08:11:28 INFO crail: crail.namenode.fileblocks 16 >>> 20/02/11 08:11:28 INFO crail: crail.namenode.rpctype >>> org.apache.crail.namenode.rpc.tcp.TcpNameNode >>> 20/02/11 08:11:28 INFO crail: crail.namenode.log >>> 20/02/11 08:11:28 INFO crail: crail.storage.types >>> org.apache.crail.storage.tcp.TcpStorageTier >>> 20/02/11 08:11:28 INFO crail: crail.storage.classes 1 >>> 20/02/11 08:11:28 INFO crail: crail.storage.rootclass 0 >>> 20/02/11 08:11:28 INFO crail: crail.storage.keepalive 2 >>> 20/02/11 08:11:28 INFO narpc: new NaRPC server group v1.5.0, queueDepth >>> 16, messageSize 2097152, nodealy false, cores 1 >>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.interface eth0 >>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.port 50020 >>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.storagelimit 1073741824 >>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.allocationsize 1073741824 >>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.datapath >>> /dev/hugepages/data >>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.queuedepth 16 >>> 20/02/11 08:11:28 INFO crail: crail.storage.tcp.cores 1 >>> 20/02/11 08:11:28 INFO crail: running TCP storage server, address / >>> 10.114.222.23:50020 >>> 20/02/11 08:11:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth >>> 32, messageSize 512, nodealy true >>> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.queueDepth 32 >>> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.messageSize 512 >>> 20/02/11 08:11:28 INFO crail: crail.namenode.tcp.cores 1 >>> 20/02/11 08:11:28 INFO crail: connected to namenode(s) localhost/ >>> 127.0.0.1:9060 >>> Exception in thread "main" java.io.IOException: Map failed >>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938) >>> at >>> org.apache.crail.storage.tcp.TcpStorageServer.allocateResource(TcpStorageServer.java:83) >>> at org.apache.crail.storage.StorageServer.main(StorageServer.java:152) >>> Caused by: java.lang.OutOfMemoryError: Map failed >>> at sun.nio.ch.FileChannelImpl.map0(Native Method) >>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935) >>> ... 2 more >>> ===== >>> Sorry for being a noob. Thx for the help. >>> >>> Lou. >>> >>> On Mon, Feb 10, 2020 at 6:31 PM David Crespi < >>> david.cre...@storedgesystems.com> wrote: >>> >>>> Was does your core-site and crail-site look like. >>>> At first thought, it may not be resolving “sgt-pepper”. >>>> Why not try the actual IP address to eliminate. >>>> >>>> Regards, >>>> David >>>> >>>> (C) 714-476-2692 >>>> >>>> From: Lou DeGenaro<mailto:lou.degen...@gmail.com> >>>> Sent: Monday, February 10, 2020 1:12 PM >>>> To: dev@crail.apache.org<mailto:dev@crail.apache.org> >>>> Subject: Re: iobench >>>> >>>> I seem to have gone backwards? I started over with new VM. I started 2 >>>> docker containers, one each for name and data: >>>> >>>> [root@sgt-pepper ~]# docker run -it --network host -e >>>> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 namenode >>>> 20/02/10 20:32:44 INFO crail: initalizing namenode >>>> 20/02/10 20:32:44 INFO crail: crail.version 3101 >>>> 20/02/10 20:32:44 INFO crail: crail.directorydepth 16 >>>> 20/02/10 20:32:44 INFO crail: crail.tokenexpiration 10 >>>> 20/02/10 20:32:44 INFO crail: crail.blocksize 1048576 >>>> 20/02/10 20:32:44 INFO crail: crail.cachelimit 0 >>>> 20/02/10 20:32:44 INFO crail: crail.cachepath /dev/hugepages/cache >>>> 20/02/10 20:32:44 INFO crail: crail.user crail >>>> 20/02/10 20:32:44 INFO crail: crail.shadowreplication 1 >>>> 20/02/10 20:32:44 INFO crail: crail.debug false >>>> 20/02/10 20:32:44 INFO crail: crail.statistics true >>>> 20/02/10 20:32:44 INFO crail: crail.rpctimeout 1000 >>>> 20/02/10 20:32:44 INFO crail: crail.datatimeout 1000 >>>> 20/02/10 20:32:44 INFO crail: crail.buffersize 1048576 >>>> 20/02/10 20:32:44 INFO crail: crail.slicesize 524288 >>>> 20/02/10 20:32:44 INFO crail: crail.singleton true >>>> 20/02/10 20:32:44 INFO crail: crail.regionsize 1073741824 >>>> 20/02/10 20:32:44 INFO crail: crail.directoryrecord 512 >>>> 20/02/10 20:32:44 INFO crail: crail.directoryrandomize true >>>> 20/02/10 20:32:44 INFO crail: crail.cacheimpl >>>> org.apache.crail.memory.MappedBufferCache >>>> 20/02/10 20:32:44 INFO crail: crail.locationmap >>>> 20/02/10 20:32:44 INFO crail: crail.namenode.address >>>> crail://sgt-pepper:9060?id=0&size=1 >>>> 20/02/10 20:32:44 INFO crail: crail.namenode.blockselection roundrobin >>>> 20/02/10 20:32:44 INFO crail: crail.namenode.fileblocks 16 >>>> 20/02/10 20:32:44 INFO crail: crail.namenode.rpctype >>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode >>>> 20/02/10 20:32:44 INFO crail: crail.namenode.log >>>> 20/02/10 20:32:44 INFO crail: crail.storage.types >>>> org.apache.crail.storage.tcp.TcpStorageTier >>>> 20/02/10 20:32:44 INFO crail: crail.storage.classes 1 >>>> 20/02/10 20:32:44 INFO crail: crail.storage.rootclass 0 >>>> 20/02/10 20:32:44 INFO crail: crail.storage.keepalive 2 >>>> 20/02/10 20:32:44 INFO crail: round robin block selection >>>> 20/02/10 20:32:45 INFO narpc: new NaRPC server group v1.5.0, queueDepth >>>> 32, >>>> messageSize 512, nodealy true, cores 1 >>>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.queueDepth 32 >>>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.messageSize 512 >>>> 20/02/10 20:32:45 INFO crail: crail.namenode.tcp.cores 1 >>>> 20/02/10 20:35:36 INFO crail: new connection from /10.114.222.82:37328 >>>> 20/02/10 20:35:36 INFO narpc: adding new channel to selector, from / >>>> 10.114.222.82:37328 >>>> 20/02/10 20:35:36 INFO crail: adding datanode /10.114.222.82:50020 of >>>> type >>>> 0 to storage class 0 >>>> >>>> [root@sgt-pepper ~]# docker run -it --network host -e >>>> NAMENODE_HOST=sgt-pepper apache/incubator-crail:1.2 datanode >>>> 20/02/10 20:35:36 INFO crail: crail.version 3101 >>>> 20/02/10 20:35:36 INFO crail: crail.directorydepth 16 >>>> 20/02/10 20:35:36 INFO crail: crail.tokenexpiration 10 >>>> 20/02/10 20:35:36 INFO crail: crail.blocksize 1048576 >>>> 20/02/10 20:35:36 INFO crail: crail.cachelimit 0 >>>> 20/02/10 20:35:36 INFO crail: crail.cachepath /dev/hugepages/cache >>>> 20/02/10 20:35:36 INFO crail: crail.user crail >>>> 20/02/10 20:35:36 INFO crail: crail.shadowreplication 1 >>>> 20/02/10 20:35:36 INFO crail: crail.debug false >>>> 20/02/10 20:35:36 INFO crail: crail.statistics true >>>> 20/02/10 20:35:36 INFO crail: crail.rpctimeout 1000 >>>> 20/02/10 20:35:36 INFO crail: crail.datatimeout 1000 >>>> 20/02/10 20:35:36 INFO crail: crail.buffersize 1048576 >>>> 20/02/10 20:35:36 INFO crail: crail.slicesize 524288 >>>> 20/02/10 20:35:36 INFO crail: crail.singleton true >>>> 20/02/10 20:35:36 INFO crail: crail.regionsize 1073741824 >>>> 20/02/10 20:35:36 INFO crail: crail.directoryrecord 512 >>>> 20/02/10 20:35:36 INFO crail: crail.directoryrandomize true >>>> 20/02/10 20:35:36 INFO crail: crail.cacheimpl >>>> org.apache.crail.memory.MappedBufferCache >>>> 20/02/10 20:35:36 INFO crail: crail.locationmap >>>> 20/02/10 20:35:36 INFO crail: crail.namenode.address >>>> crail://sgt-pepper:9060 >>>> 20/02/10 20:35:36 INFO crail: crail.namenode.blockselection roundrobin >>>> 20/02/10 20:35:36 INFO crail: crail.namenode.fileblocks 16 >>>> 20/02/10 20:35:36 INFO crail: crail.namenode.rpctype >>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode >>>> 20/02/10 20:35:36 INFO crail: crail.namenode.log >>>> 20/02/10 20:35:36 INFO crail: crail.storage.types >>>> org.apache.crail.storage.tcp.TcpStorageTier >>>> 20/02/10 20:35:36 INFO crail: crail.storage.classes 1 >>>> 20/02/10 20:35:36 INFO crail: crail.storage.rootclass 0 >>>> 20/02/10 20:35:36 INFO crail: crail.storage.keepalive 2 >>>> 20/02/10 20:35:36 INFO narpc: new NaRPC server group v1.5.0, queueDepth >>>> 16, >>>> messageSize 2097152, nodealy false, cores 1 >>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.interface eth0 >>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.port 50020 >>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.storagelimit 1073741824 >>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.allocationsize >>>> 1073741824 >>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.datapath >>>> /dev/hugepages/data >>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.queuedepth 16 >>>> 20/02/10 20:35:36 INFO crail: crail.storage.tcp.cores 1 >>>> 20/02/10 20:35:36 INFO crail: running TCP storage server, address / >>>> 10.114.222.82:50020 >>>> 20/02/10 20:35:36 INFO narpc: new NaRPC client group v1.5.0, queueDepth >>>> 32, >>>> messageSize 512, nodealy true >>>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.queueDepth 32 >>>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.messageSize 512 >>>> 20/02/10 20:35:36 INFO crail: crail.namenode.tcp.cores 1 >>>> 20/02/10 20:35:36 INFO crail: connected to namenode(s) sgt-pepper/ >>>> 10.114.222.82:9060 >>>> 20/02/10 20:35:36 INFO crail: datanode statistics, freeBlocks 1024 >>>> ... >>>> >>>> This seems to be OK so far, since we see *adding datanode >>>> /10.114.222.82:50020 <http://10.114.222.82:50020> of type 0 to storage >>>> class 0* in the namenode log. >>>> >>>> Then I download the Non-Official Binary Release from here: >>>> https://crail.apache.org/download/ and I extract and create the 4 conf >>>> files and the revised log4j properties file: >>>> [root@sgt-pepper conf]# ls -atl >>>> total 40 >>>> drwxr-xr-x. 2 kube kube 4096 Feb 10 14:14 . >>>> -rw-r--r--. 1 kube kube 569 Feb 10 14:14 log4j.properties >>>> drwxr-xr-x. 8 root root 4096 Feb 10 13:52 .. >>>> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml >>>> -rw-r--r--. 1 kube kube 1211 Sep 25 2018 core-site.xml.template >>>> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh >>>> -rw-r--r--. 1 kube kube 125 Sep 25 2018 crail-env.sh.template >>>> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf >>>> -rw-r--r--. 1 kube kube 296 Sep 25 2018 crail-site.conf.template >>>> -rw-r--r--. 1 kube kube 568 Feb 28 2018 log4j.properties.orig >>>> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves >>>> -rw-r--r--. 1 kube kube 0 Feb 28 2018 slaves.template >>>> [root@sgt-pepper conf]# diff log4j.properties log4j.properties.orig >>>> 2c2 >>>> < log4j.rootCategory=DEBUG, console >>>> --- >>>> > log4j.rootCategory=INFO, console >>>> >>>> I setup CRAIL_HOME: >>>> >>>> [root@sgt-pepper apache-crail-1.2-incubating]# export CRAIL_HOME=$PWD >>>> [root@sgt-pepper apache-crail-1.2-incubating]# env | grep CRAIL >>>> CRAIL_HOME=/usr/local/apache-crail-1.2-incubating >>>> >>>> ===== >>>> >>>> Now I try using the CLI: >>>> >>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail >>>> fs -ls >>>> / >>>> 20/02/10 14:54:51 DEBUG Shell: Failed to detect a valid hadoop home >>>> directory >>>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set. >>>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326) >>>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351) >>>> at >>>> >>>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440) >>>> at >>>> >>>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486) >>>> at >>>> >>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170) >>>> at >>>> >>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153) >>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64) >>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >>>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340) >>>> 20/02/10 14:54:51 DEBUG Shell: setsid exited with exit code 0 >>>> 20/02/10 14:54:51 DEBUG Configuration: parsing URL >>>> >>>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml >>>> 20/02/10 14:54:51 DEBUG Configuration: parsing input stream >>>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8 >>>> 20/02/10 14:54:52 DEBUG Configuration: parsing URL >>>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml >>>> 20/02/10 14:54:52 DEBUG Configuration: parsing input stream >>>> java.io.BufferedInputStream@71c7db30 >>>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field >>>> org.apache.hadoop.metrics2.lib.MutableRate >>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess >>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate >>>> of >>>> successful kerberos logins and latency (milliseconds)]) >>>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field >>>> org.apache.hadoop.metrics2.lib.MutableRate >>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure >>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate >>>> of >>>> failed kerberos logins and latency (milliseconds)]) >>>> 20/02/10 14:54:52 DEBUG MutableMetricsFactory: field >>>> org.apache.hadoop.metrics2.lib.MutableRate >>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups >>>> with >>>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, >>>> value=[GetGroups]) >>>> 20/02/10 14:54:52 DEBUG MetricsSystemImpl: UgiMetrics, User and group >>>> related metrics >>>> 20/02/10 14:54:52 DEBUG KerberosName: Kerberos krb5 configuration not >>>> found, setting default realm to empty >>>> 20/02/10 14:54:52 DEBUG Groups: Creating new Groups object >>>> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Trying to load the >>>> custom-built >>>> native-hadoop library... >>>> 20/02/10 14:54:52 DEBUG NativeCodeLoader: Failed to load native-hadoop >>>> with >>>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path >>>> 20/02/10 14:54:52 DEBUG NativeCodeLoader: >>>> >>>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib >>>> 20/02/10 14:54:52 WARN NativeCodeLoader: Unable to load native-hadoop >>>> library for your platform... using builtin-java classes where applicable >>>> 20/02/10 14:54:52 DEBUG PerformanceAdvisory: Falling back to shell based >>>> 20/02/10 14:54:52 DEBUG JniBasedUnixGroupsMappingWithFallback: Group >>>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping >>>> 20/02/10 14:54:52 DEBUG Groups: Group mapping >>>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; >>>> cacheTimeout=300000; warningDeltaMs=5000 >>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login >>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: hadoop login commit >>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: using local >>>> user:UnixPrincipal: root >>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: Using user: >>>> "UnixPrincipal: >>>> root" with name root >>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: User entry: "root" >>>> 20/02/10 14:54:52 DEBUG UserGroupInformation: UGI loginUser:root >>>> (auth:SIMPLE) >>>> 20/02/10 14:54:52 INFO crail: CrailHadoopFileSystem construction >>>> 20/02/10 14:54:52 INFO crail: creating singleton crail file system >>>> 20/02/10 14:54:52 INFO crail: crail.version 3101 >>>> 20/02/10 14:54:52 INFO crail: crail.directorydepth 16 >>>> 20/02/10 14:54:52 INFO crail: crail.tokenexpiration 10 >>>> 20/02/10 14:54:52 INFO crail: crail.blocksize 1048576 >>>> 20/02/10 14:54:52 INFO crail: crail.cachelimit 1073741824 >>>> 20/02/10 14:54:52 INFO crail: crail.cachepath /dev/hugepages/cache >>>> 20/02/10 14:54:52 INFO crail: crail.user crail >>>> 20/02/10 14:54:52 INFO crail: crail.shadowreplication 1 >>>> 20/02/10 14:54:52 INFO crail: crail.debug false >>>> 20/02/10 14:54:52 INFO crail: crail.statistics true >>>> 20/02/10 14:54:52 INFO crail: crail.rpctimeout 1000 >>>> 20/02/10 14:54:52 INFO crail: crail.datatimeout 1000 >>>> 20/02/10 14:54:52 INFO crail: crail.buffersize 1048576 >>>> 20/02/10 14:54:52 INFO crail: crail.slicesize 524288 >>>> 20/02/10 14:54:52 INFO crail: crail.singleton true >>>> 20/02/10 14:54:52 INFO crail: crail.regionsize 1073741824 >>>> 20/02/10 14:54:52 INFO crail: crail.directoryrecord 512 >>>> 20/02/10 14:54:52 INFO crail: crail.directoryrandomize true >>>> 20/02/10 14:54:52 INFO crail: crail.cacheimpl >>>> org.apache.crail.memory.MappedBufferCache >>>> 20/02/10 14:54:52 INFO crail: crail.locationmap >>>> 20/02/10 14:54:52 INFO crail: crail.namenode.address >>>> crail://localhost:9060 >>>> 20/02/10 14:54:52 INFO crail: crail.namenode.blockselection roundrobin >>>> 20/02/10 14:54:52 INFO crail: crail.namenode.fileblocks 16 >>>> 20/02/10 14:54:52 INFO crail: crail.namenode.rpctype >>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode >>>> 20/02/10 14:54:52 INFO crail: crail.namenode.log >>>> 20/02/10 14:54:52 INFO crail: crail.storage.types >>>> org.apache.crail.storage.tcp.TcpStorageTier >>>> 20/02/10 14:54:52 INFO crail: crail.storage.classes 1 >>>> 20/02/10 14:54:52 INFO crail: crail.storage.rootclass 0 >>>> 20/02/10 14:54:52 INFO crail: crail.storage.keepalive 2 >>>> 20/02/10 14:54:52 INFO crail: buffer cache, allocationCount 1, >>>> bufferCount >>>> 1024 >>>> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth >>>> 16, >>>> messageSize 2097152, nodealy false >>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.interface eth0 >>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.port 50020 >>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.storagelimit 1073741824 >>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.allocationsize >>>> 1073741824 >>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.datapath >>>> /dev/hugepages/data >>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.queuedepth 16 >>>> 20/02/10 14:54:52 INFO crail: crail.storage.tcp.cores 1 >>>> 20/02/10 14:54:52 INFO narpc: new NaRPC client group v1.5.0, queueDepth >>>> 32, >>>> messageSize 512, nodealy true >>>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.queueDepth 32 >>>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.messageSize 512 >>>> 20/02/10 14:54:52 INFO crail: crail.namenode.tcp.cores 1 >>>> ls: java.io.IOException: java.net.ConnectException: Connection refused >>>> >>>> ===== >>>> >>>> Since I get connection refused, I try changing "localhost" to >>>> "sgt-pepper" >>>> in the conf files: >>>> >>>> [root@sgt-pepper conf]# grep sgt-pepper * >>>> core-site.xml: <value>crail://sgt-pepper:9060</value> >>>> crail-site.conf:crail.namenode.address >>>> crail://sgt-pepper:9060 >>>> >>>> Then I try using the CLI again: >>>> >>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail >>>> fs -ls >>>> / >>>> 20/02/10 15:00:27 DEBUG Shell: Failed to detect a valid hadoop home >>>> directory >>>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set. >>>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326) >>>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351) >>>> at >>>> >>>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440) >>>> at >>>> >>>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486) >>>> at >>>> >>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170) >>>> at >>>> >>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153) >>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64) >>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >>>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340) >>>> 20/02/10 15:00:27 DEBUG Shell: setsid exited with exit code 0 >>>> 20/02/10 15:00:27 DEBUG Configuration: parsing URL >>>> >>>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml >>>> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream >>>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8 >>>> 20/02/10 15:00:27 DEBUG Configuration: parsing URL >>>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml >>>> 20/02/10 15:00:27 DEBUG Configuration: parsing input stream >>>> java.io.BufferedInputStream@71c7db30 >>>> 20/02/10 15:00:27 DEBUG MutableMetricsFactory: field >>>> org.apache.hadoop.metrics2.lib.MutableRate >>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess >>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate >>>> of >>>> successful kerberos logins and latency (milliseconds)]) >>>> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field >>>> org.apache.hadoop.metrics2.lib.MutableRate >>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure >>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate >>>> of >>>> failed kerberos logins and latency (milliseconds)]) >>>> 20/02/10 15:00:28 DEBUG MutableMetricsFactory: field >>>> org.apache.hadoop.metrics2.lib.MutableRate >>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups >>>> with >>>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, >>>> value=[GetGroups]) >>>> 20/02/10 15:00:28 DEBUG MetricsSystemImpl: UgiMetrics, User and group >>>> related metrics >>>> 20/02/10 15:00:28 DEBUG KerberosName: Kerberos krb5 configuration not >>>> found, setting default realm to empty >>>> 20/02/10 15:00:28 DEBUG Groups: Creating new Groups object >>>> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Trying to load the >>>> custom-built >>>> native-hadoop library... >>>> 20/02/10 15:00:28 DEBUG NativeCodeLoader: Failed to load native-hadoop >>>> with >>>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path >>>> 20/02/10 15:00:28 DEBUG NativeCodeLoader: >>>> >>>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib >>>> 20/02/10 15:00:28 WARN NativeCodeLoader: Unable to load native-hadoop >>>> library for your platform... using builtin-java classes where applicable >>>> 20/02/10 15:00:28 DEBUG PerformanceAdvisory: Falling back to shell based >>>> 20/02/10 15:00:28 DEBUG JniBasedUnixGroupsMappingWithFallback: Group >>>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping >>>> 20/02/10 15:00:28 DEBUG Groups: Group mapping >>>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; >>>> cacheTimeout=300000; warningDeltaMs=5000 >>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login >>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: hadoop login commit >>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: using local >>>> user:UnixPrincipal: root >>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: Using user: >>>> "UnixPrincipal: >>>> root" with name root >>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: User entry: "root" >>>> 20/02/10 15:00:28 DEBUG UserGroupInformation: UGI loginUser:root >>>> (auth:SIMPLE) >>>> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem construction >>>> 20/02/10 15:00:28 INFO crail: creating singleton crail file system >>>> 20/02/10 15:00:28 INFO crail: crail.version 3101 >>>> 20/02/10 15:00:28 INFO crail: crail.directorydepth 16 >>>> 20/02/10 15:00:28 INFO crail: crail.tokenexpiration 10 >>>> 20/02/10 15:00:28 INFO crail: crail.blocksize 1048576 >>>> 20/02/10 15:00:28 INFO crail: crail.cachelimit 1073741824 >>>> 20/02/10 15:00:28 INFO crail: crail.cachepath /dev/hugepages/cache >>>> 20/02/10 15:00:28 INFO crail: crail.user crail >>>> 20/02/10 15:00:28 INFO crail: crail.shadowreplication 1 >>>> 20/02/10 15:00:28 INFO crail: crail.debug false >>>> 20/02/10 15:00:28 INFO crail: crail.statistics true >>>> 20/02/10 15:00:28 INFO crail: crail.rpctimeout 1000 >>>> 20/02/10 15:00:28 INFO crail: crail.datatimeout 1000 >>>> 20/02/10 15:00:28 INFO crail: crail.buffersize 1048576 >>>> 20/02/10 15:00:28 INFO crail: crail.slicesize 524288 >>>> 20/02/10 15:00:28 INFO crail: crail.singleton true >>>> 20/02/10 15:00:28 INFO crail: crail.regionsize 1073741824 >>>> 20/02/10 15:00:28 INFO crail: crail.directoryrecord 512 >>>> 20/02/10 15:00:28 INFO crail: crail.directoryrandomize true >>>> 20/02/10 15:00:28 INFO crail: crail.cacheimpl >>>> org.apache.crail.memory.MappedBufferCache >>>> 20/02/10 15:00:28 INFO crail: crail.locationmap >>>> 20/02/10 15:00:28 INFO crail: crail.namenode.address >>>> crail://sgt-pepper:9060 >>>> 20/02/10 15:00:28 INFO crail: crail.namenode.blockselection roundrobin >>>> 20/02/10 15:00:28 INFO crail: crail.namenode.fileblocks 16 >>>> 20/02/10 15:00:28 INFO crail: crail.namenode.rpctype >>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode >>>> 20/02/10 15:00:28 INFO crail: crail.namenode.log >>>> 20/02/10 15:00:28 INFO crail: crail.storage.types >>>> org.apache.crail.storage.tcp.TcpStorageTier >>>> 20/02/10 15:00:28 INFO crail: crail.storage.classes 1 >>>> 20/02/10 15:00:28 INFO crail: crail.storage.rootclass 0 >>>> 20/02/10 15:00:28 INFO crail: crail.storage.keepalive 2 >>>> 20/02/10 15:00:28 INFO crail: buffer cache, allocationCount 1, >>>> bufferCount >>>> 1024 >>>> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth >>>> 16, >>>> messageSize 2097152, nodealy false >>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.interface eth0 >>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.port 50020 >>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.storagelimit 1073741824 >>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.allocationsize >>>> 1073741824 >>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.datapath >>>> /dev/hugepages/data >>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.queuedepth 16 >>>> 20/02/10 15:00:28 INFO crail: crail.storage.tcp.cores 1 >>>> 20/02/10 15:00:28 INFO narpc: new NaRPC client group v1.5.0, queueDepth >>>> 32, >>>> messageSize 512, nodealy true >>>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.queueDepth 32 >>>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.messageSize 512 >>>> 20/02/10 15:00:28 INFO crail: crail.namenode.tcp.cores 1 >>>> 20/02/10 15:00:28 INFO crail: connected to namenode(s) sgt-pepper/ >>>> 10.114.222.82:9060 >>>> 20/02/10 15:00:28 INFO crail: CrailHadoopFileSystem fs initialization >>>> done.. >>>> ls: / >>>> 20/02/10 15:00:28 INFO crail: Closing CrailHadoopFileSystem >>>> 20/02/10 15:00:28 INFO crail: Closing CrailFS singleton >>>> 20/02/10 15:00:28 INFO crail: mapped client cache closed >>>> >>>> Seem better, but not really. >>>> >>>> [root@sgt-pepper apache-crail-1.2-incubating]# $CRAIL_HOME/bin/crail fs >>>> -mkdir /foobar >>>> 20/02/10 15:06:14 DEBUG Shell: Failed to detect a valid hadoop home >>>> directory >>>> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set. >>>> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:326) >>>> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:351) >>>> at >>>> >>>> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:440) >>>> at >>>> >>>> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:486) >>>> at >>>> >>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170) >>>> at >>>> >>>> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153) >>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64) >>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >>>> at org.apache.hadoop.fs.FsShell.main(FsShell.java:340) >>>> 20/02/10 15:06:14 DEBUG Shell: setsid exited with exit code 0 >>>> 20/02/10 15:06:14 DEBUG Configuration: parsing URL >>>> >>>> jar:file:/usr/local/apache-crail-1.2-incubating/jars/hadoop-common-2.7.3.jar!/core-default.xml >>>> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream >>>> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@52d455b8 >>>> 20/02/10 15:06:14 DEBUG Configuration: parsing URL >>>> file:/usr/local/apache-crail-1.2-incubating/conf/core-site.xml >>>> 20/02/10 15:06:14 DEBUG Configuration: parsing input stream >>>> java.io.BufferedInputStream@71c7db30 >>>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field >>>> org.apache.hadoop.metrics2.lib.MutableRate >>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess >>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate >>>> of >>>> successful kerberos logins and latency (milliseconds)]) >>>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field >>>> org.apache.hadoop.metrics2.lib.MutableRate >>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure >>>> with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate >>>> of >>>> failed kerberos logins and latency (milliseconds)]) >>>> 20/02/10 15:06:14 DEBUG MutableMetricsFactory: field >>>> org.apache.hadoop.metrics2.lib.MutableRate >>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups >>>> with >>>> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, >>>> sampleName=Ops, always=false, type=DEFAULT, valueName=Time, >>>> value=[GetGroups]) >>>> 20/02/10 15:06:14 DEBUG MetricsSystemImpl: UgiMetrics, User and group >>>> related metrics >>>> 20/02/10 15:06:14 DEBUG KerberosName: Kerberos krb5 configuration not >>>> found, setting default realm to empty >>>> 20/02/10 15:06:14 DEBUG Groups: Creating new Groups object >>>> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Trying to load the >>>> custom-built >>>> native-hadoop library... >>>> 20/02/10 15:06:14 DEBUG NativeCodeLoader: Failed to load native-hadoop >>>> with >>>> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path >>>> 20/02/10 15:06:14 DEBUG NativeCodeLoader: >>>> >>>> java.library.path=/usr/local/apache-crail-1.2-incubating/bin/../lib::/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib >>>> 20/02/10 15:06:14 WARN NativeCodeLoader: Unable to load native-hadoop >>>> library for your platform... using builtin-java classes where applicable >>>> 20/02/10 15:06:14 DEBUG PerformanceAdvisory: Falling back to shell based >>>> 20/02/10 15:06:14 DEBUG JniBasedUnixGroupsMappingWithFallback: Group >>>> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping >>>> 20/02/10 15:06:14 DEBUG Groups: Group mapping >>>> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; >>>> cacheTimeout=300000; warningDeltaMs=5000 >>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login >>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: hadoop login commit >>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: using local >>>> user:UnixPrincipal: root >>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: Using user: >>>> "UnixPrincipal: >>>> root" with name root >>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: User entry: "root" >>>> 20/02/10 15:06:14 DEBUG UserGroupInformation: UGI loginUser:root >>>> (auth:SIMPLE) >>>> 20/02/10 15:06:14 INFO crail: CrailHadoopFileSystem construction >>>> 20/02/10 15:06:14 INFO crail: creating singleton crail file system >>>> 20/02/10 15:06:14 INFO crail: crail.version 3101 >>>> 20/02/10 15:06:14 INFO crail: crail.directorydepth 16 >>>> 20/02/10 15:06:14 INFO crail: crail.tokenexpiration 10 >>>> 20/02/10 15:06:14 INFO crail: crail.blocksize 1048576 >>>> 20/02/10 15:06:14 INFO crail: crail.cachelimit 1073741824 >>>> 20/02/10 15:06:14 INFO crail: crail.cachepath /dev/hugepages/cache >>>> 20/02/10 15:06:14 INFO crail: crail.user crail >>>> 20/02/10 15:06:14 INFO crail: crail.shadowreplication 1 >>>> 20/02/10 15:06:14 INFO crail: crail.debug false >>>> 20/02/10 15:06:14 INFO crail: crail.statistics true >>>> 20/02/10 15:06:14 INFO crail: crail.rpctimeout 1000 >>>> 20/02/10 15:06:14 INFO crail: crail.datatimeout 1000 >>>> 20/02/10 15:06:14 INFO crail: crail.buffersize 1048576 >>>> 20/02/10 15:06:14 INFO crail: crail.slicesize 524288 >>>> 20/02/10 15:06:14 INFO crail: crail.singleton true >>>> 20/02/10 15:06:14 INFO crail: crail.regionsize 1073741824 >>>> 20/02/10 15:06:14 INFO crail: crail.directoryrecord 512 >>>> 20/02/10 15:06:14 INFO crail: crail.directoryrandomize true >>>> 20/02/10 15:06:14 INFO crail: crail.cacheimpl >>>> org.apache.crail.memory.MappedBufferCache >>>> 20/02/10 15:06:14 INFO crail: crail.locationmap >>>> 20/02/10 15:06:14 INFO crail: crail.namenode.address >>>> crail://sgt-pepper:9060 >>>> 20/02/10 15:06:14 INFO crail: crail.namenode.blockselection roundrobin >>>> 20/02/10 15:06:14 INFO crail: crail.namenode.fileblocks 16 >>>> 20/02/10 15:06:14 INFO crail: crail.namenode.rpctype >>>> org.apache.crail.namenode.rpc.tcp.TcpNameNode >>>> 20/02/10 15:06:14 INFO crail: crail.namenode.log >>>> 20/02/10 15:06:14 INFO crail: crail.storage.types >>>> org.apache.crail.storage.tcp.TcpStorageTier >>>> 20/02/10 15:06:14 INFO crail: crail.storage.classes 1 >>>> 20/02/10 15:06:14 INFO crail: crail.storage.rootclass 0 >>>> 20/02/10 15:06:14 INFO crail: crail.storage.keepalive 2 >>>> 20/02/10 15:06:14 INFO crail: buffer cache, allocationCount 1, >>>> bufferCount >>>> 1024 >>>> 20/02/10 15:06:14 INFO narpc: new NaRPC client group v1.5.0, queueDepth >>>> 16, >>>> messageSize 2097152, nodealy false >>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.interface eth0 >>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.port 50020 >>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.storagelimit 1073741824 >>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.allocationsize >>>> 1073741824 >>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.datapath >>>> /dev/hugepages/data >>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.queuedepth 16 >>>> 20/02/10 15:06:14 INFO crail: crail.storage.tcp.cores 1 >>>> 20/02/10 15:06:15 INFO narpc: new NaRPC client group v1.5.0, queueDepth >>>> 32, >>>> messageSize 512, nodealy true >>>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.queueDepth 32 >>>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.messageSize 512 >>>> 20/02/10 15:06:15 INFO crail: crail.namenode.tcp.cores 1 >>>> 20/02/10 15:06:15 INFO crail: connected to namenode(s) sgt-pepper/ >>>> 10.114.222.82:9060 >>>> 20/02/10 15:06:15 INFO crail: CrailHadoopFileSystem fs initialization >>>> done.. >>>> mkdir: java.util.concurrent.ExecutionException: java.io.IOException: Map >>>> failed >>>> 20/02/10 15:06:15 INFO crail: Closing CrailHadoopFileSystem >>>> 20/02/10 15:06:15 INFO crail: Closing CrailFS singleton >>>> 20/02/10 15:06:15 INFO crail: mapped client cache closed >>>> >>>> I'm thinkinh that *mkdir: java.util.concurrent.ExecutionException: >>>> java.io.IOException: Map failed *is not good. >>>> >>>> Sorry for the long append. Probably something really dumb, but I'm >>>> sratching my head... >>>> >>>> Lou. >>>> >>>> >>>> On Mon, Feb 10, 2020 at 12:13 PM David Crespi < >>>> david.cre...@storedgesystems.com> wrote: >>>> >>>> > Looks like both cmds are working, but it’s not really finding the >>>> > datastore. >>>> > You’re getting info into the namenode, but not the datanode (which >>>> means >>>> > you’re really not writing to the datanode). The >>>> > First warning you can ignore (WARN NativeCodeLoader: Unable to load >>>> > native-hadoop library for your platform) >>>> > >>>> > you may want to turn on debug to see more of what’s going on. >>>> > Edit /crail/conf/log4j.properties and change from INFO to DEBUG. >>>> > >>>> > You should probably also look at the individual docker logs. >>>> > docker logs YourNamenodeName & docker logs YourDatanodeName >>>> > >>>> > may give you some more hints of what’s happening. >>>> > >>>> > >>>> > Regards, >>>> > David >>>> > >>>> > >>>> > From: Lou DeGenaro<mailto:lou.degen...@gmail.com> >>>> > Sent: Monday, February 10, 2020 8:12 AM >>>> > To: dev@crail.apache.org<mailto:dev@crail.apache.org> >>>> > Subject: Re: iobench >>>> > >>>> > OK, I take it back. Having trouble with crail fs. >>>> > >>>> > ===== >>>> > >>>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar >>>> > 20/02/10 10:07:16 WARN NativeCodeLoader: Unable to load native-hadoop >>>> > library for your platform... using builtin-java classes where >>>> applicable >>>> > ... >>>> > 20/02/10 10:07:16 INFO crail: connected to namenode(s) abbey-road/ >>>> > 10.114.222.23:9060 >>>> > 20/02/10 10:07:16 INFO crail: CrailHadoopFileSystem fs initialization >>>> > done.. >>>> > mkdir: java.util.concurrent.ExecutionException: java.io.IOException: >>>> Map >>>> > failed >>>> > 20/02/10 10:07:16 INFO crail: Closing CrailHadoopFileSystem >>>> > 20/02/10 10:07:16 INFO crail: Closing CrailFS singleton >>>> > 20/02/10 10:07:16 INFO crail: mapped client cache closed >>>> > >>>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -rmdir /foobar >>>> > 20/02/10 10:08:55 WARN NativeCodeLoader: Unable to load native-hadoop >>>> > library for your platform... using builtin-java classes where >>>> applicable >>>> > ...20/02/10 10:08:55 INFO crail: CrailHadoopFileSystem fs >>>> initialization >>>> > done.. >>>> > rmdir: /foobar >>>> > 20/02/10 10:08:55 INFO crail: Closing CrailHadoopFileSystem >>>> > 20/02/10 10:08:55 INFO crail: Closing CrailFS singleton >>>> > 20/02/10 10:08:55 INFO crail: mapped client cache closed >>>> > >>>> > [root@abbey-road ~]# $CRAIL_HOME/bin/crail fs -mkdir /foobar >>>> > 20/02/10 10:09:43 WARN NativeCodeLoader: Unable to load native-hadoop >>>> > library for your platform... using builtin-java classes where >>>> applicable >>>> > ... >>>> > 20/02/10 10:09:43 INFO crail: CrailHadoopFileSystem fs initialization >>>> > done.. >>>> > mkdir: `/foobar': File exists >>>> > 20/02/10 10:09:43 INFO crail: Closing CrailHadoopFileSystem >>>> > 20/02/10 10:09:43 INFO crail: Closing CrailFS singleton >>>> > 20/02/10 10:09:43 INFO crail: mapped client cache closed >>>> > >>>> > ===== >>>> > >>>> > Seems that mkdir works, but rmdir does not? Also, IOException was >>>> reported >>>> > on mkdir, which seems worrisome. >>>> > >>>> > Lou. >>>> > >>>> > On Mon, Feb 10, 2020 at 8:16 AM Lou DeGenaro <lou.degen...@gmail.com> >>>> > wrote: >>>> > >>>> > > David, >>>> > > >>>> > > Thanks. I'm able to use *crail fs *successfully to >>>> create/list/delete. >>>> > > Will try to increase available storage space next... >>>> > > >>>> > > Lou. >>>> > > >>>> > > On Fri, Feb 7, 2020 at 6:37 PM David Crespi < >>>> > > david.cre...@storedgesystems.com> wrote: >>>> > > >>>> > >> Oh, and it may have already written something on the datastore, so >>>> you’d >>>> > >> need >>>> > >> to check it and remove it with the fs command anyway. >>>> > >> >>>> > >> crail fs -ls -R / >>>> > >> >>>> > >> would show you what you have there. >>>> > >> >>>> > >> Regards, >>>> > >> David >>>> > >> >>>> > >> (C) 714-476-2692 >>>> > >> >>>> > >> From: Lou DeGenaro<mailto:lou.degen...@gmail.com> >>>> > >> Sent: Friday, February 7, 2020 12:15 PM >>>> > >> To: dev@crail.apache.org<mailto:dev@crail.apache.org> >>>> > >> Subject: iobench >>>> > >> >>>> > >> Still a noob. Got namenode and datanode running as docker images >>>> on my >>>> > >> VM. I'm looking for the simplest example of writing something then >>>> > >> reading >>>> > >> something. >>>> > >> >>>> > >> [root@abbey-road conf]# $CRAIL_HOME/bin/crail iobench -t write -f >>>> > >> /filename >>>> > >> -s 1024 -k 1 >>>> > >> 20/02/07 14:10:22 INFO crail: creating singleton crail file system >>>> > >> 20/02/07 14:10:23 INFO crail: crail.version 3101 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.directorydepth 16 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.tokenexpiration 10 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.blocksize 1048576 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.cachelimit 1073741824 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.cachepath /dev/hugepages/cache >>>> > >> 20/02/07 14:10:23 INFO crail: crail.user crail >>>> > >> 20/02/07 14:10:23 INFO crail: crail.shadowreplication 1 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.debug false >>>> > >> 20/02/07 14:10:23 INFO crail: crail.statistics true >>>> > >> 20/02/07 14:10:23 INFO crail: crail.rpctimeout 1000 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.datatimeout 1000 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.buffersize 1048576 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.slicesize 524288 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.singleton true >>>> > >> 20/02/07 14:10:23 INFO crail: crail.regionsize 1073741824 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrecord 512 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.directoryrandomize true >>>> > >> 20/02/07 14:10:23 INFO crail: crail.cacheimpl >>>> > >> org.apache.crail.memory.MappedBufferCache >>>> > >> 20/02/07 14:10:23 INFO crail: crail.locationmap >>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.address >>>> > >> crail://abbey-road:9060 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.blockselection >>>> roundrobin >>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.fileblocks 16 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.rpctype >>>> > >> org.apache.crail.namenode.rpc.tcp.TcpNameNode >>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.log >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.types >>>> > >> org.apache.crail.storage.tcp.TcpStorageTier >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.classes 1 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.rootclass 0 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.keepalive 2 >>>> > >> 20/02/07 14:10:23 INFO crail: buffer cache, allocationCount 1, >>>> > bufferCount >>>> > >> 1024 >>>> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, >>>> queueDepth >>>> > >> 16, >>>> > >> messageSize 2097152, nodealy false >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.interface eth0 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.port 50020 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.storagelimit >>>> 1073741824 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.allocationsize >>>> > 1073741824 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.datapath >>>> > >> /dev/hugepages/data >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.queuedepth 16 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.storage.tcp.cores 1 >>>> > >> 20/02/07 14:10:23 INFO narpc: new NaRPC client group v1.5.0, >>>> queueDepth >>>> > >> 32, >>>> > >> messageSize 512, nodealy true >>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.queueDepth 32 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.messageSize 512 >>>> > >> 20/02/07 14:10:23 INFO crail: crail.namenode.tcp.cores 1 >>>> > >> 20/02/07 14:10:23 INFO crail: connected to namenode(s) abbey-road/ >>>> > >> 10.114.222.23:9060 >>>> > >> write, filename /filename, size 1024, loop 1, storageClass 0, >>>> > >> locationClass >>>> > >> 0, buffered true >>>> > >> Exception in thread "main" java.io.IOException: Map failed >>>> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:938) >>>> > >> at >>>> > >> >>>> > >> >>>> > >>>> org.apache.crail.memory.MappedBufferCache.allocateRegion(MappedBufferCache.java:94) >>>> > >> at >>>> > >>>> org.apache.crail.memory.BufferCache.allocateBuffer(BufferCache.java:95) >>>> > >> at >>>> > >> >>>> > >>>> org.apache.crail.core.CoreDataStore.allocateBuffer(CoreDataStore.java:482) >>>> > >> at >>>> org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:87) >>>> > >> at >>>> org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1070) >>>> > >> Caused by: java.lang.OutOfMemoryError: Map failed >>>> > >> at sun.nio.ch.FileChannelImpl.map0(Native Method) >>>> > >> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:935) >>>> > >> ... 5 more >>>> > >> >>>> > >> >>>> > >>>> > >>>> >>>>