This is just a simple test, the more adequate one is needed in the future. Maybe this work can't be finished until most of the share lib is implemented.
Thanks Yang > -----Original Message----- > From: Liu Yuan [mailto:[email protected]] > Sent: Wednesday, May 27, 2015 4:45 PM > To: yuyang > Cc: sheepdog > Subject: Re: VDI r/w performance comparison test > > On Wed, May 27, 2015 at 04:24:29PM +0800, yuyang wrote: > > Hello everyone, > > > > Recently we need copy one VDI from sheepdog to another storage > > system(such as sheepdog or RAID). There are 2 ways to choose, > > 1. dog vdi r | dog vdi w,(only for sheepdog, or temp file is needed) > > 2. lib client, which is on the way. > > > > And then I did a R/W IO performance comparison. > > Now I want to share the results. > > > > In this test, we read from sheepdog and write to the same sheepdog. Or we > just call > > it "copy task". For the sake of simplicity, the sheepdog only has one node. > > > > The first step is to prepare the source VDI. The size of source VDI is 50GB, > > and the data is 10GB/25GB/40GB(we do three tests). > > > > The task by lib client is done by the following code: > > > > /****** copy.c *********/ > > #include <stdio.h> > > #include <stdlib.h> > > #include <sheepdog sheepdog.h=""> > > > > #define BUFSIZE (1024*1024*128) /**128M**/ > > > > void copy(struct sd_cluster *csrc, char *srcname, > > struct sd_cluster *cdst, char *dstname) > > { > > int ret = 0; > > struct sd_vdi * srcv = sd_vdi_open(csrc, srcname, 0, NULL); > > if (!srcv) { > > fprintf(stderr, "open %s failed\n", srcname); > > return ; > > } > > uint64_t size = srcv->inode->vdi_size; > > ret = sd_vdi_create(cdst, dstname, size, 0); > > if (ret != SD_RES_SUCCESS) { > > fprintf(stderr, "create failed %s\n", sd_error2str(ret)); > > return; > > } > > struct sd_vdi * dstv = sd_vdi_open(cdst, dstname, 0, NULL); > > if (!dstv) { > > fprintf(stderr, "open %s failed\n", dstname); > > return ; > > } > > char *buf = malloc(BUFSIZE); > > if ( !buf ) { > > printf("OOM\n"); > > return ; > > } > > uint64_t offset = 0, left = size, onew; > > while(offset < size) { > > left = size - offset; > > onew = left > BUFSIZE ? BUFSIZE : left; > > memset(buf, 0, BUFSIZE); > > > > ret = sd_vdi_read(srcv, buf, onew, offset); > > if (ret != SD_RES_SUCCESS) { > > fprintf(stderr, " read error: offset=%"PRIx64, offset); > > return; > > } > > ret = sd_vdi_write(dstv, buf, onew, offset); > > if (ret != SD_RES_SUCCESS) { > > fprintf(stderr, " write error: offset=%"PRIx64, offset); > > return; > > } > > offset += onew; > > } > > free(buf); > > } > > > > int main(int argc, char *argv[]) > > { > > char host1[256] = {}; > > char host2[256] = {}; > > memset(host1, 0, 256); > > memset(host2, 0, 256); > > if (argc != 5) { > > printf("too small argument\n"); > > printf("usage: copy srcIP srcvdiname dstIP > dstvdiname\n"); > > return 0; > > } > > > > strcpy(host1, argv[1]); > > strcpy(host2, argv[3]); > > > > strcat(host1, ":7000"); > > strcat(host2, ":7000"); > > > > struct sd_cluster *c1 = sd_connect(host1); > > struct sd_cluster *c2 = sd_connect(host2); > > > > if (!c1 || !c2) { > > printf("connect failed\n"); > > return 0; > > } > > copy(c1, argv[2], c1, argv[4]); > > return 0; > > } > > > > compile: gcc copy.c -lpthread -lsheepdog > > run it: > > ./a.out 127.0.0.1 $vlibsrc 127.0.0.1 $vlibdst > > > > we record the runtime of the programme. > > > > And the copy task by dog cmd is done by the following script: > > dog vdi create $vdogdst 50g > > dog vdi read $vdogsrc | dog vdi write $vdogdst 0 50g > > > > we also record the runtime. > > > > Then we prepare 2 new source VDI in which there is 25GB(and then 40GB) > > and repeat the steps above. > > The result is as follows: > > > > data(GB) lib/dog time(s) > > 10 lib 101 > > 10 dog 239 > > > > 25 lib 128 > > 25 dog 251 > > > > 40 lib 134 > > 40 dog 249 > > > > As is shown in the table above, we can see that most of the time, > > the lib client can speed up to more than 50% in VDI r/w. > > Thanks for the share. I guess he performance gain is linear to the buffer > size. > Could you please add a dimension 'buffer size' such as following? > > 128M 256M 512M 1024M > lib x y z .. > dog x' y' z' .. > > Thanks, > Yuan -- sheepdog mailing list [email protected] https://lists.wpkg.org/mailman/listinfo/sheepdog
