Hi Lou Thanks. Can you show the output of `mount` and `cat /proc/meminfo`?
Can you measure again with an increased buffer size? Thanks Adrian On 3/5/20 14:48, Lou DeGenaro wrote: > Hi Adrian, > > re: iperf > > iperf -s > iperf -c <hostname> > > re: bandwidth > > bandwidth is calculated offline, using the seconds printed and the 1 GB > known file size. The 1 GB file is generated using: > > base64 /dev/urandom | head -c 1073741824 > G1.txt > > and is stored into Crail using: > > bin/crail fs -copyFromLocal /root/G1.txt /G1.txt > > > re: Crail configuration > > crail.namenode.address crail://sgt-pepper:9060 > crail.cachepath /tmp/hugepages/cache > crail.cachelimit 1073741824 > crail.storage.tcp.interface eth0 > crail.storage.tcp.datapath /tmp/hugepages/data > crail.storage.tcp.storagelimit 1073741824 > > I did try using iobench sometime ago without success. See my newsgroup > post 2020/02/07 > > Thanks for your help and interest. > > Lou. > > > On Thu, Mar 5, 2020 at 8:26 AM Adrian Schuepbach < > [email protected]> wrote: > >> Hi Lou >> >> It is hard to say without knowing more details. >> >> Can you post the exact iperf command you used? >> >> Also the Crail code you show seems not to be the one you >> use for the measurements, at least I don't see where the code >> computes the bandwidth. Can you post the actual code you are using? >> >> Have you configured Crail to store data on hugepages? Or will >> it access disks? >> >> There is also iobench, a performance measurement tool that comes >> with Crail. Have you tried measuring with this one? >> >> >> Best >> Adrian >> >> On 3/5/20 14:01, Lou DeGenaro wrote: >>> Hello, >>> >>> I'm comparing Crail to iperf. VM-A is used to run the server(s). VM-B >> is >>> used to run the client. Both VMs are CentOS 7 with 8 GB memory + 2 CPUs. >>> Tests runs are non-overlapping. >>> >>> For the Crail case , a 1 GB file is posted to server and a simple Java >>> client is employed (see below). >>> >>> Results: >>> >>> iperf: 4.05 Gb/seb >>> Crail: 2.52 Gb/sec >>> >>> Why is iperf so much better?? >>> >>> Lou. >>> >>> ----- >>> >>> Crail Java Client: >>> >>> public static void main(String[] args) { >>> try { >>> //initialize >>> String filename = "/G1.txt"; >>> int filesize = 1024*1024*1024; >>> int bufsize = 1024*1024; >>> CrailBuffer buf = OffHeapBuffer.wrap(ByteBuffer.allocateDirect(bufsize)); >>> CrailConfiguration cconf = >> CrailConfiguration.createConfigurationFromFile(); >>> CrailStore cstore = CrailStore.newInstance(cconf); >>> CrailFile file = cstore.lookup(filename).get().asFile(); >>> CrailInputStream directStream = >>> file.getDirectInputStream(file.getCapacity()); >>> long sumbytes = 0; >>> //run test >>> long start = System.currentTimeMillis(); >>> while (sumbytes < filesize) { >>> buf.clear(); >>> CrailResult cr = directStream.read(buf).get(); >>> long ret = cr.getLen(); >>> sumbytes = sumbytes + ret; >>> } >>> long end = System.currentTimeMillis(); >>> //print result and clean-up >>> double executionTime = ((double) (end - start)) / 1000.0; >>> System.out.println("time: "+executionTime); >>> cstore.close(); >>> directStream.close(); >>> } >>> catch(Exception e) { >>> e.printStackTrace(); >>> } >>> } >>> >> -- >> Adrian Schüpbach, Dr. sc. ETH Zürich >> >> -- Adrian Schüpbach, Dr. sc. ETH Zürich
