Re: [Qemu-devel] Question about drive mirror

2014-01-14 Thread rudy...@163.com 
Thank you for your detailed reply.
Now, I am modifying the code to change the reading and writing processes, 
My purpose is that after it send the writing requestion and the writing 
requestion 
is not come back, then it send the reading requestion. 
I think this way will be more reading and writing pressure than before. 

I don't know why in mirror_iteration, each iteration send the number of data is 
continuous dirty data, the maximum is the size of free_buffer(default is 10M),
I think the way will reduce the performance of IO,because the mirror will wait 
long 
time to send next reading requestion. I think each iteration send the number of 
data
should base on the granularity ,we can set the granularity  is 1M. This way may 
be 
let the reading  and writing is parallel. 




zhang min

From: Stefan Hajnoczi
Date: 2014-01-14 13:57
To: Fam Zheng
CC: rudy...@163.com; rudy.zhangmin; qemu-devel
Subject: Re: [Qemu-devel] Question about drive mirror
On Tue, Jan 14, 2014 at 09:41:41AM +0800, Fam Zheng wrote:
 On Mon, 01/13 23:44, rudy...@163.com wrote:
  I tested the capability of drive mirror, I found the IO is low. Then I read 
  the code, 
  The code mirror_run() will call mirror_iteration() to read the size of 
  buffer data  
  from source storage, when the read callback ,and then in 
  mirror_read_complete () 
  write the data to the target storage, It is serial. 

And it also uses throttling to avoid impacting the guest too much.  It's
considered a background job, i.e. not trying to saturate storage
bandwidth.

That said, improving performance while keeping I/O isolated from higher
priority guest I/O can be good.

  Now, I hope when it is writing the data to target storage ,we can send the 
  request 
  of reading data from source storage. Because of  using  coroutine to do it 
  ,there is 
  some troubles to achieve it. why not use Multi-thread? 
  Some one can give me some idea?

It can be parallelized by splitting the code into two separate loops:

A reader coroutine that reads data from the source device and places
buffers in a queue.

A writer coroutine that takes buffers from the queue and writes data to
the target device.

(In fact there could be multiple readers and writers.)

Stefan

[Qemu-devel] Question about drive mirror

2014-01-13 Thread rudy...@163.com 
Hi,everyone.
I tested the capability of drive mirror, I found the IO is low. Then I read the 
code, 
The code mirror_run() will call mirror_iteration() to read the size of buffer 
data  
from source storage, when the read callback ,and then in mirror_read_complete 
() 
write the data to the target storage, It is serial. 
Now, I hope when it is writing the data to target storage ,we can send the 
request 
of reading data from source storage. Because of  using  coroutine to do it 
,there is 
some troubles to achieve it. why not use Multi-thread? 
Some one can give me some idea?
Thanks.




zhang min

Re: [Qemu-devel] Question about drive mirror

2014-01-13 Thread Fam Zheng 
On Mon, 01/13 23:44, rudy...@163.com wrote:
 Hi,everyone.
 I tested the capability of drive mirror, I found the IO is low. Then I read 
 the code, 
 The code mirror_run() will call mirror_iteration() to read the size of buffer 
 data  
 from source storage, when the read callback ,and then in mirror_read_complete 
 () 
 write the data to the target storage, It is serial. 
 Now, I hope when it is writing the data to target storage ,we can send the 
 request 
 of reading data from source storage. Because of  using  coroutine to do it 
 ,there is 
 some troubles to achieve it. why not use Multi-thread? 
 Some one can give me some idea?

Hi,

QEMU block layer has been using coroutine as the program model even before
introducing these block jobs, so it was nature that block mirror followed this
style. With coroutines, I believe it is as possible as with multi-thread, to
have more outstanding IO requests, in order to speed up the mirroring. What's
the trouble you have?

Multi-threading needs much more synchronization mechanism than what we have in
block interface now. One day it may be possible to move block job to a thread,
but it will require quite some work.

Fam

Re: [Qemu-devel] Question about drive mirror

2014-01-13 Thread Stefan Hajnoczi 
On Tue, Jan 14, 2014 at 09:41:41AM +0800, Fam Zheng wrote:
 On Mon, 01/13 23:44, rudy...@163.com wrote:
  I tested the capability of drive mirror, I found the IO is low. Then I read 
  the code, 
  The code mirror_run() will call mirror_iteration() to read the size of 
  buffer data  
  from source storage, when the read callback ,and then in 
  mirror_read_complete () 
  write the data to the target storage, It is serial. 

And it also uses throttling to avoid impacting the guest too much.  It's
considered a background job, i.e. not trying to saturate storage
bandwidth.

That said, improving performance while keeping I/O isolated from higher
priority guest I/O can be good.

  Now, I hope when it is writing the data to target storage ,we can send the 
  request 
  of reading data from source storage. Because of  using  coroutine to do it 
  ,there is 
  some troubles to achieve it. why not use Multi-thread? 
  Some one can give me some idea?

It can be parallelized by splitting the code into two separate loops:

A reader coroutine that reads data from the source device and places
buffers in a queue.

A writer coroutine that takes buffers from the queue and writes data to
the target device.

(In fact there could be multiple readers and writers.)

Stefan