On Tue, Nov 17, 2015 at 03:28:29PM +0800, Fam Zheng wrote:
> On Tue, 11/17 14:58, Stefan Hajnoczi wrote:
> > On Mon, Nov 16, 2015 at 02:10:36PM +0800, Fam Zheng wrote:
> > > When a request R is absorbed by request M, it is appended to the
> > > "mr_next" queue led by M, and is completed together with the completion
> > > of M, in virtio_blk_rw_complete.
> > >
> > > With error policy equals stop, if M has an I/O error, now R also gets
> > > prepended to the per device DMA restart queue, which will be retried
> > > when VM resumes. It leads to a double completion (in symptoms of memory
> > > corruption or use after free).
> > >
> > > Adding R to the queue is superfluous, only M needs to be in the queue.
> > >
> > > Fix this by marking request R as "merged" and skipping it in
> > > virtio_blk_handle_rw_error.
>
> The commit message is outdated. "merged" requests are actually skipped in
> virtio_blk_handle_request.
>
> > >
> > > Cc: qemu-sta...@nongnu.org
> > > Signed-off-by: Fam Zheng <f...@redhat.com>
> > >
> > > ---
> > >
> > > v2: Don't lose the request in migration. [Paolo]
> > > ---
> > > hw/block/virtio-blk.c | 7 +++++++
> > > include/hw/virtio/virtio-blk.h | 1 +
> > > 2 files changed, 8 insertions(+)
> > >
> > > diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
> > > index e70fccf..5cdb06f 100644
> > > --- a/hw/block/virtio-blk.c
> > > +++ b/hw/block/virtio-blk.c
> > > @@ -36,6 +36,7 @@ VirtIOBlockReq *virtio_blk_alloc_request(VirtIOBlock *s)
> > > req->in_len = 0;
> > > req->next = NULL;
> > > req->mr_next = NULL;
> > > + req->merged = false;
> > > return req;
> > > }
> > >
> > > @@ -344,6 +345,7 @@ static inline void submit_requests(BlockBackend *blk,
> > > MultiReqBuffer *mrb,
> > > for (i = start + 1; i < start + num_reqs; i++) {
> > > qemu_iovec_concat(qiov, &mrb->reqs[i]->qiov, 0,
> > > mrb->reqs[i]->qiov.size);
> > > + mrb->reqs[i]->merged = true;
> > > mrb->reqs[i - 1]->mr_next = mrb->reqs[i];
> > > nb_sectors += mrb->reqs[i]->qiov.size / BDRV_SECTOR_SIZE;
> > > }
> > > @@ -511,6 +513,11 @@ void virtio_blk_handle_request(VirtIOBlockReq *req,
> > > MultiReqBuffer *mrb)
> > > - sizeof(struct virtio_blk_inhdr);
> > > iov_discard_back(in_iov, &in_num, sizeof(struct virtio_blk_inhdr));
> > >
> > > + if (req->merged) {
> > > + /* Enough for restarting a (migrated) merged request, no need to
> > > + * actually submit I/O. */
> > > + return;
> > > + }
>
> This is not enough.
>
> There is a risk that the coalesced requests being restarted here are also
> merged, which will leak the requests originally merged into it.
>
> > > type = virtio_ldl_p(VIRTIO_DEVICE(req->dev), &req->out.type);
> > >
> > > /* VIRTIO_BLK_T_OUT defines the command direction.
> > > VIRTIO_BLK_T_BARRIER
> > > diff --git a/include/hw/virtio/virtio-blk.h
> > > b/include/hw/virtio/virtio-blk.h
> > > index 6bf5905..db4adf4 100644
> > > --- a/include/hw/virtio/virtio-blk.h
> > > +++ b/include/hw/virtio/virtio-blk.h
> > > @@ -70,6 +70,7 @@ typedef struct VirtIOBlockReq {
> > > size_t in_len;
> > > struct VirtIOBlockReq *next;
> > > struct VirtIOBlockReq *mr_next;
> > > + bool merged;
> > > BlockAcctCookie acct;
> > > } VirtIOBlockReq;
> >
> > I'm not sure if this patch truly fixes the bug:
> >
> > virtio_blk_rw_complete() doesn't do req->mr_next = NULL. There is a
> > potential double-free if resubmitting failed requests doesn't overwrite
> > themr_next field.
> >
> > This can be fixed by adding req->mr_next = NULL to the loop in
> > virtio_blk_rw_complete().
> >
> > Is that enough to solve the bug? I don't think adding a new field is
> > necessary.
> >
> > If not, please explain the double-free.
>
> The first free is the expected one when the coalesced request is completed.
> The second free is because virtio_blk_rw_complete was also called on requests
> who have "merged == true", which is a mistake in virtio_blk_dma_restart_bh.
>
> I don't think adding req->mr_next in virtio_blk_rw_complete in the I/O error
> path is right or helpful.Question about your patch: How is the merged qiov retained across restart? The merged qiov is freed the first time virtio_blk_rw_complete() with an error gets called and you skip processing merged requests in virtio_blk_handle_request(). I think the result of your patch is that only the head request is resubmitted to the host OS. Non-head requests are not submitted to the host OS. The requests all complete at the virtio-blk level but only the head request actually gets data transferred. Like I said, I think the fix is much simpler. Forget about adding a merged field and just dissolve the relationship between merged requests in virtio_blk_rw_complete() by setting req->mr_next = NULL in virtio_blk_rw_complete()'s loop. The code was designed to work this way because virtio_blk_dma_restart_bh() parses and submits requests again from scratch at the virtio-blk level. Stefan
signature.asc
Description: PGP signature
