From: Yu Kuai <[email protected]> bfq_bio_bfqg() is called while bfqd->lock is held from the merge and request insertion paths. It walks bio->bi_blkg and its parent chain to find the closest online BFQ group, and also updates bio->bi_blkg when the original association points at an offline or otherwise unusable blkg.
After missing blkg creation is protected by q->blkcg_mutex, bio_associate_blkg_from_css() can sleep on lookup misses. BFQ must not call it while holding bfqd->lock. The blkg BFQ wants is already known from the existing bio->bi_blkg ancestry walk, so update bio->bi_blkg by swapping references to that existing blkg directly instead of looking it up again by css. Signed-off-by: Yu Kuai <[email protected]> --- block/bfq-cgroup.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index 5c2faf56c8ef..06c4ec6d5e35 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -604,6 +604,16 @@ static void bfq_link_bfqg(struct bfq_data *bfqd, struct bfq_group *bfqg) } } +static void bfq_bio_update_blkg(struct bio *bio, struct blkcg_gq *blkg) +{ + if (bio->bi_blkg == blkg) + return; + + blkg_get(blkg); + blkg_put(bio->bi_blkg); + bio->bi_blkg = blkg; +} + struct bfq_group *bfq_bio_bfqg(struct bfq_data *bfqd, struct bio *bio) { struct blkcg_gq *blkg = bio->bi_blkg; @@ -616,14 +626,13 @@ struct bfq_group *bfq_bio_bfqg(struct bfq_data *bfqd, struct bio *bio) } bfqg = blkg_to_bfqg(blkg); if (bfqg->pd.online) { - bio_associate_blkg_from_css(bio, &blkg->blkcg->css, false); + bfq_bio_update_blkg(bio, blkg); return bfqg; } blkg = blkg->parent; } - bio_associate_blkg_from_css(bio, - &bfqg_to_blkg(bfqd->root_group)->blkcg->css, - false); + blkg = bfqg_to_blkg(bfqd->root_group); + bfq_bio_update_blkg(bio, blkg); return bfqd->root_group; } -- 2.51.0
