When we are iterating the set bits in a word, we also need to factor in
the cleared bits. Don't call fn() unless the bit is also not set in
the cleared word.
Fixes: ea86ea2cdced ("sbitmap: ammortize cost of clearing bits")
Signed-off-by: Jens Axboe <[email protected]>
diff --git a/include/linux/sbitmap.h b/include/linux/sbitmap.h
index 92806a2dbab7..9f374fbcdba6 100644
--- a/include/linux/sbitmap.h
+++ b/include/linux/sbitmap.h
@@ -283,6 +283,11 @@ static inline void __sbitmap_for_each_set(struct sbitmap
*sb,
nr = find_next_bit(&word->word, depth, nr);
if (nr >= depth)
break;
+ /* if set in cleared, it's actually free */
+ if (test_bit(nr, &word->cleared)) {
+ nr++;
+ continue;
+ }
if (!fn(sb, (index << sb->shift) + nr, data))
return;
--
Jens Axboe