Hi Adrian, re: iperf
iperf -s iperf -c <hostname> re: bandwidth bandwidth is calculated offline, using the seconds printed and the 1 GB known file size. The 1 GB file is generated using: base64 /dev/urandom | head -c 1073741824 > G1.txt and is stored into Crail using: bin/crail fs -copyFromLocal /root/G1.txt /G1.txt re: Crail configuration crail.namenode.address crail://sgt-pepper:9060 crail.cachepath /tmp/hugepages/cache crail.cachelimit 1073741824 crail.storage.tcp.interface eth0 crail.storage.tcp.datapath /tmp/hugepages/data crail.storage.tcp.storagelimit 1073741824 I did try using iobench sometime ago without success. See my newsgroup post 2020/02/07 Thanks for your help and interest. Lou. On Thu, Mar 5, 2020 at 8:26 AM Adrian Schuepbach < [email protected]> wrote: > Hi Lou > > It is hard to say without knowing more details. > > Can you post the exact iperf command you used? > > Also the Crail code you show seems not to be the one you > use for the measurements, at least I don't see where the code > computes the bandwidth. Can you post the actual code you are using? > > Have you configured Crail to store data on hugepages? Or will > it access disks? > > There is also iobench, a performance measurement tool that comes > with Crail. Have you tried measuring with this one? > > > Best > Adrian > > On 3/5/20 14:01, Lou DeGenaro wrote: > > Hello, > > > > I'm comparing Crail to iperf. VM-A is used to run the server(s). VM-B > is > > used to run the client. Both VMs are CentOS 7 with 8 GB memory + 2 CPUs. > > Tests runs are non-overlapping. > > > > For the Crail case , a 1 GB file is posted to server and a simple Java > > client is employed (see below). > > > > Results: > > > > iperf: 4.05 Gb/seb > > Crail: 2.52 Gb/sec > > > > Why is iperf so much better?? > > > > Lou. > > > > ----- > > > > Crail Java Client: > > > > public static void main(String[] args) { > > try { > > //initialize > > String filename = "/G1.txt"; > > int filesize = 1024*1024*1024; > > int bufsize = 1024*1024; > > CrailBuffer buf = OffHeapBuffer.wrap(ByteBuffer.allocateDirect(bufsize)); > > CrailConfiguration cconf = > CrailConfiguration.createConfigurationFromFile(); > > CrailStore cstore = CrailStore.newInstance(cconf); > > CrailFile file = cstore.lookup(filename).get().asFile(); > > CrailInputStream directStream = > > file.getDirectInputStream(file.getCapacity()); > > long sumbytes = 0; > > //run test > > long start = System.currentTimeMillis(); > > while (sumbytes < filesize) { > > buf.clear(); > > CrailResult cr = directStream.read(buf).get(); > > long ret = cr.getLen(); > > sumbytes = sumbytes + ret; > > } > > long end = System.currentTimeMillis(); > > //print result and clean-up > > double executionTime = ((double) (end - start)) / 1000.0; > > System.out.println("time: "+executionTime); > > cstore.close(); > > directStream.close(); > > } > > catch(Exception e) { > > e.printStackTrace(); > > } > > } > > > -- > Adrian Schüpbach, Dr. sc. ETH Zürich > >
