On Fri, Nov 21, 2025 at 04:17:39PM +0800, zhangshida wrote: > We have captured four instances of this corruption in our production > environment. > In each case, we observed a distinct pattern: > The corruption starts at an offset that aligns with the beginning of > an XFS extent. > The corruption ends at an offset that is aligned to the system's > `PAGE_SIZE` (64KB in our case). > > Corruption Instances: > 1. Start:`0x73be000`, **End:** `0x73c0000` (Length: 8KB) > 2. Start:`0x10791a000`, **End:** `0x107920000` (Length: 24KB) > 3. Start:`0x14535a000`, **End:** `0x145b70000` (Length: 8280KB) > 4. Start:`0x370d000`, **End:** `0x3710000` (Length: 12KB)
Do you have a somwhat isolate reproducer for this? > After analysis, we believe the root cause is in the handling of chained > bios, specifically related to out-of-order io completion. > > Consider a bio chain where `bi_remaining` is decremented as each bio in > the chain completes. > For example, > if a chain consists of three bios (bio1 -> bio2 -> bio3) with > bi_remaining count: > 1->2->2 > if the bio completes in the reverse order, there will be a problem. > if bio 3 completes first, it will become: > 1->2->1 > then bio 2 completes: > 1->1->0 > > Because `bi_remaining` has reached zero, the final `end_io` callback > for the entire chain is triggered, even though not all bios in the > chain have actually finished processing. This premature completion can > lead to stale data being exposed, as seen in our case. It sounds like there is a problem because bi_remaining is only incremented after already submittin a bio. Which code path do you see this with? iomap doesn't chain bios, so is this the buffer cache or log code? Or is there a remapping driver involved?
