Thank you for your contribution! Sashiko AI review found 7 potential
issues to consider:
- [High] drbd: partition offset ignored in drbd_dax_open() and
  drbd_dax_map()
- [High] drbd: sub-page offset and page-count truncation when PAGE_SIZE
  > 4096
- [High] drbd: missing cache writeback and ordering in
  drbd_dax_al_initialize()
- [High] drbd: __LC_DIRTY never cleared in drbd_dax_al_begin_io_commit()
- [High] drbd: link failure and wrong Kconfig dependency on
  CONFIG_DEV_DAX_PMEM
- [Medium] drbd: signed 32-bit overflow of al_slots in
  drbd_dax_al_initialize()
- [Medium] drbd: uncommitted and unlocked act_log read in
  drbd_dax_al_initialize()
--

commit 78fd73a53d10bb7aa94cbf5a0d1278098930c4df
Author: Christoph Böhmwalder <[email protected]>

drbd: add DAX/PMEM support for metadata access

This commit adds DAX/PMEM support for DRBD metadata access, allowing the
superblock, activity log, and resync bitmap to be mapped and updated
directly in persistent memory.

> diff --git a/drivers/block/drbd/Makefile b/drivers/block/drbd/Makefile
> index 9f05f74e619c1..8ec1ad3604499 100644
> --- a/drivers/block/drbd/Makefile
> +++ b/drivers/block/drbd/Makefile
> @@ -5,6 +5,7 @@ drbd-y += drbd_main.o drbd_strings.o drbd_nl.o
>  drbd-y += drbd_interval.o drbd_state.o
>  drbd-y += drbd_nl_gen.o
>  drbd-y += drbd_transport.o
> +drbd-$(CONFIG_DEV_DAX_PMEM) += drbd_dax_pmem.o
>  drbd-$(CONFIG_DEBUG_FS) += drbd_debugfs.o
>  
>  obj-$(CONFIG_BLK_DEV_DRBD)     += drbd.o

[Severity: High]
Does the build fail to link when CONFIG_BLK_DEV_DRBD=y and
CONFIG_DEV_DAX_PMEM=m?  In that configuration, this line expands to
drbd-m += drbd_dax_pmem.o, which Kbuild ignores when linking built-in
drbd.o, while IS_ENABLED(CONFIG_DEV_DAX_PMEM) in drbd_dax_pmem.h still
evaluates to true and declares drbd_dax_*() as external functions instead
of stubs.

In addition, drbd_dax_open() looks up the DAX device via
fs_dax_get_by_bdev(), which is gated on CONFIG_FS_DAX rather than
CONFIG_DEV_DAX_PMEM (the /dev/daxX.Y character device driver), so
drbd_dax_open() always returns -ENODEV if CONFIG_FS_DAX is disabled.

> diff --git a/drivers/block/drbd/drbd_dax_pmem.c 
> b/drivers/block/drbd/drbd_dax_pmem.c
> new file mode 100644
> index 0000000000000..1fb22e57cf5d0
> --- /dev/null
> +++ b/drivers/block/drbd/drbd_dax_pmem.c
[ ... ]
> +static pgoff_t md_sector_to_pgoff(sector_t sector)
> +{
> +     return sector >> (PAGE_SHIFT - SECTOR_SHIFT);
> +}
> +
> +static long md_sectors_to_pages(sector_t nr_sectors)
> +{
> +     return nr_sectors >> (PAGE_SHIFT - SECTOR_SHIFT);
> +}
> +
> +/* Signed, since the bitmap offset relative to the superblock may be 
> negative. */
> +static long md_sector_offset_to_bytes(long sector_offset)
> +{
> +     return sector_offset * SECTOR_SIZE;
> +}
> +
> +static int map_superblock_for_dax(struct drbd_backing_dev *bdev, struct 
> dax_device *dax_dev)
> +{
> +     long want = 1;
> +     pgoff_t pgoff = md_sector_to_pgoff(bdev->md.md_offset);
> +     void *kaddr;
> +     long len;
> +     int id;
> +
> +     id = dax_read_lock();
> +     len = dax_direct_access(dax_dev, pgoff, want, DAX_ACCESS, &kaddr, NULL);
> +     dax_read_unlock(id);
> +
> +     if (len < want)
> +             return -EIO;
> +
> +     bdev->md_on_pmem = kaddr;
> +
> +     return 0;
> +}
> +
> +/**
> + * drbd_dax_open() - Open device for dax and map metadata superblock
> + * @bdev: backing device to be opened
> + */
> +int drbd_dax_open(struct drbd_backing_dev *bdev)
> +{
> +     struct dax_device *dax_dev;
> +     int err;
> +     u64 part_off;
> +
> +     dax_dev = fs_dax_get_by_bdev(bdev->md_bdev, &part_off, NULL, NULL);
> +     if (!dax_dev)
> +             return -ENODEV;
> +
> +     err = map_superblock_for_dax(bdev, dax_dev);
> +     if (!err)
> +             bdev->dax_dev = dax_dev;
> +     else
> +             put_dax(dax_dev);
> +
> +     return err;
> +}
> +
> +void drbd_dax_close(struct drbd_backing_dev *bdev)
> +{
> +     put_dax(bdev->dax_dev);
> +}
> +
> +/**
> + * drbd_dax_map() - Map metadata for dax
> + * @bdev: backing device whose metadata is to be mapped
> + */
> +int drbd_dax_map(struct drbd_backing_dev *bdev)
> +{
> +     struct dax_device *dax_dev = bdev->dax_dev;
> +     sector_t first_sector = drbd_md_first_sector(bdev);
> +     sector_t al_sector = bdev->md.md_offset + bdev->md.al_offset;
> +     long want = md_sectors_to_pages(drbd_md_last_sector(bdev) + 1 - 
> first_sector);
> +     pgoff_t pgoff = md_sector_to_pgoff(first_sector);
> +     long md_offset_byte = md_sector_offset_to_bytes(bdev->md.md_offset - 
> first_sector);
> +     long al_offset_byte = md_sector_offset_to_bytes(al_sector - 
> first_sector);
> +     void *kaddr;
> +     long len;
> +     int id;
> +
> +     id = dax_read_lock();
> +     len = dax_direct_access(dax_dev, pgoff, want, DAX_ACCESS, &kaddr, NULL);
> +     dax_read_unlock(id);
> +
> +     if (len < want)
> +             return -EIO;
> +
> +     bdev->md_on_pmem = kaddr + md_offset_byte;
> +     bdev->al_on_pmem = kaddr + al_offset_byte;
> +
> +     return 0;
> +}

[Severity: High]
Does discarding part_off in drbd_dax_open() cause DRBD to read and write
metadata at the wrong offset when bdev->md_bdev is a partition?

fs_dax_get_by_bdev() returns the dax_device for the whole disk and writes
the partition's starting byte offset into part_off.  Because
map_superblock_for_dax() and drbd_dax_map() pass partition-relative
pgoff values to dax_direct_access() without adding part_off, metadata
accesses hit the beginning of the parent PMEM disk instead of the
partition, corrupting the partition table or earlier partitions.

[Severity: High]
Can md_sector_to_pgoff() and md_sectors_to_pages() misalign pointers and
truncate the mapping when PAGE_SIZE > 4096 (such as on arm64 or powerpc
with 64 KiB pages)?

DRBD metadata offsets and sizes are 4 KiB-aligned, not PAGE_SIZE-aligned;
for example, internal metadata places the 4 KiB superblock in the last 8
sectors of the device.  In map_superblock_for_dax(), md_sector_to_pgoff()
shifts off the sub-page bits of bdev->md.md_offset, so kaddr points to the
start of the page rather than the superblock.  In drbd_dax_map(), kaddr
also starts at the page boundary pgoff << PAGE_SHIFT, but md_offset_byte
and al_offset_byte are calculated relative to first_sector instead of the
start of pgoff, and md_sectors_to_pages() right-shifts the sector count
down (yielding want == 0 if the metadata region is smaller than a page).

[ ... ]

> +void drbd_dax_al_begin_io_commit(struct drbd_device *device)
> +{
> +     struct lc_element *e;
> +
> +     spin_lock_irq(&device->al_lock);
> +
> +     list_for_each_entry(e, &device->act_log->to_be_changed, list)
> +             drbd_dax_al_update(device, e);
> +
> +     lc_committed(device->act_log);
> +
> +     spin_unlock_irq(&device->al_lock);
> +}

[Severity: High]
Does leaving __LC_DIRTY set in device->act_log->flags after
drbd_dax_al_begin_io_commit() cause later activity log locking to hang?

When an activity log lookup misses, __lc_get() sets __LC_DIRTY in
lc->flags.  lc_committed() clears pending_changes, but __LC_DIRTY is only
cleared by lc_unlock().  Unlike drbd_al_begin_io_commit(), this function
does not acquire drbd_al_try_lock_for_transaction(), call
lc_unlock(device->act_log), or wake device->al_wait.  Once __LC_DIRTY is
set, lc_try_lock(device->act_log) (which requires lc->flags == 0) always
fails, so check_al_extents() hangs forever in wait_event(device->al_wait,
drbd_al_try_lock(device)) and drbd_try_suspend_al() always fails.

> +int drbd_dax_al_initialize(struct drbd_device *device)
> +{
> +     struct al_on_pmem *al_on_pmem = device->ldev->al_on_pmem;
> +     __be32 *slots = al_on_pmem->slots;
> +     /* The activity log area holds the magic followed by the slots. */
> +     int i, al_slots = device->ldev->md.al_size_4k * (SZ_4K / 
> sizeof(*slots)) - 1;
> +
> +     al_on_pmem->magic = cpu_to_be32(DRBD_AL_PMEM_MAGIC);
> +     /*
> +      * Initialize all slots rather than just the configured number in case
> +      * the configuration is later changed.
> +      */
> +     for (i = 0; i < al_slots; i++) {
> +             unsigned int extent_nr = i < device->act_log->nr_elements ?
> +                     lc_element_by_index(device->act_log, i)->lc_number :
> +                     LC_FREE;
> +             slots[i] = cpu_to_be32(extent_nr);
> +     }
> +
> +     return 0;
> +}

[Severity: High]
Does drbd_dax_al_initialize() need to flush the initialized activity log
with arch_wb_cache_pmem() and write al_on_pmem->magic only after slots[]
are persisted?

Unlike drbd_dax_al_update(), drbd_dax_al_initialize() never calls
arch_wb_cache_pmem(), and it stores DRBD_AL_PMEM_MAGIC before populating
slots[].  If a crash or power failure occurs before the slot cache lines
are evicted from volatile CPU caches, the activity log on PMEM can have a
valid magic header with uninitialized or stale slots.

[Severity: Medium]
Can al_slots overflow a signed 32-bit int when al_size_4k is larger than
8 GiB?

check_activity_log_stripe_size() allows al_size_4k up to 4,194,304
(16 GiB).  Multiplying by SZ_4K / sizeof(*slots) (1024) produces up to
0xffffffff, which becomes negative when stored in int al_slots and causes
the loop to be skipped without initializing any slots.

[Severity: Medium]
Can drbd_dax_al_initialize() write stale extent numbers to PMEM when
device->act_log->to_be_changed has uncommitted elements?

Unlike the non-DAX drbd_al_initialize() path, which holds device->al_lock,
writes lc_new_number for entries on to_be_changed, and calls
lc_committed(device->act_log), drbd_dax_al_initialize() reads lc_number
without holding device->al_lock or committing to_be_changed.  Any pending
element on to_be_changed has its old lc_number written to PMEM and remains
on to_be_changed after the caller unlocks device->act_log.

-- 
Sashiko AI review · 
https://sashiko.dev/#/patchset/[email protected]?part=6

Reply via email to