Hi Shengyong, Hi Jinbao,
Thank you for providing the patch. I have tested the patch and found a critical
regression that can lead to data loss or unexpected failure during a no-op
resize.
The issue arises because the fixed HDIO_GETGEO calculation, when applied to a
legacy image created with start_sector=4097s, causes the segment_count to
unexpectedly shrink due to the new alignment calculation.
1. Regression Reproduction Steps (The Volume Shrink Bug)
The issue happens when resizing an image created with a legacy mkfs.f2fs tool
(which assumed start_sector=0) located at a physical offset that causes a new
alignment shift (e.g., sector 4097).
A. Reproduction of Segment Count Reduction
This demonstrates the subtle change in geometry resulting in a reduction of
total segments from 32767 to 32766 due to below code:
while (new_seg0_blkaddr < old_seg0_blkaddr)
new_seg0_blkaddr += blks_per_seg;
# Initialize GPT and create a 64GB partition starting at sector 4097
parted -s /dev/sdb mklabel gpt
parted -s /dev/sdb mkpart primary 4097s 134221824s
# Create F2FS using the buggy, pre-fix mkfs (assuming start_sector=0)
mkfs.f2fs -f /dev/sdb1
# Run the patched resize.f2fs (which correctly identifies start_sector=4097)
# This will show segment count reducing from 32767 to 32766 in the new SB.
resize.f2fs -F -f -d1 /dev/sdb1
B. Reproduction of Resize Check Failure (Data Loss Risk)
This demonstrates that when the FS is full, the segment reduction causes
valid_block_count > user_block_count, making resize.f2fs fail prematurely.
# Initialize partition
parted -s /dev/sdb mklabel gpt
parted -s /dev/sdb mkpart primary 4097s 134221824s
# Create F2FS using the buggy, pre-fix mkfs (assuming start_sector=0)
mkfs.f2fs -f /dev/sdb1
mount /dev/sdb1 /mnt/f2fs-test
# Fill the filesystem to capacity
# (Use the provided dd commands to fill the volume)
dd if=/dev/urandom of=/mnt/f2fs-test/fill_34359738368.dat bs=4096 count=8388608
oflag=direct
dd if=/dev/urandom of=/mnt/f2fs-test/fill_17179869184.dat bs=4096 count=4194304
oflag=direct
dd if=/dev/urandom of=/mnt/f2fs-test/fill_8589934592.dat bs=4096 count=2097152
oflag=direct
dd if=/dev/urandom of=/mnt/f2fs-test/fill_4294967296.dat bs=4096 count=1048576
oflag=direct
dd if=/dev/urandom of=/mnt/f2fs-test/fill_2147483648.dat bs=4096 count=524288
oflag=direct
dd if=/dev/urandom of=/mnt/f2fs-test/fill_1073741824.dat bs=4096 count=262144
oflag=direct
dd if=/dev/urandom of=/mnt/f2fs-test/fill_134217728.dat bs=4096 count=32768
oflag=direct
dd if=/dev/urandom of=/mnt/f2fs-test/fill_67108864.dat bs=4096 count=16384
oflag=direct
dd if=/dev/urandom of=/mnt/f2fs-test/fill_16777216.dat bs=4096 count=4096
oflag=direct
dd if=/dev/urandom of=/mnt/f2fs-test/fill_4194304.dat bs=4096 count=1024
oflag=direct
dd if=/dev/urandom of=/mnt/f2fs-test/fill_2097152.dat bs=4096 count=512
oflag=direct
# Run the patched resize.f2fs
# This will fail the check f2fs_resize_check() and immediately abort.
resize.f2fs -F -f -d1 /dev/sdb1
2. Analysis and Proposed Solution
This issue confirms that the corrected geometry calculation can lead to
undesirable outcomes for legacy images. When the new correct alignment requires
a larger offset (zone_align_start_offset), it causes a reduction in the space
allocated for the main_area.
I observe that this volume shrink and subsequent failure primarily occur during
a no-op resize (resize.f2fs attempting to resize to the same size) when the
filesystem is near full capacity. When normally growing the partition size,
this issue can typically be avoided.
Could you please confirm if this failure during a no-op resize scenario is
considered acceptable behavior for resize.f2fs?
If this behavior is considered an issue that needs to be addressed, I have two
proposals regarding the geometric calculation for legacy images:
A. Proactive Geometry Override (Preferred Approach):
We should attempt to deduce the original start_sector from the existing
Superblock (specifically by referencing old_sb->segment0_blkaddr). If the
deduced legacy start_sector is inconsistent with the sector reported by
HDIO_GETGEO (e.g., if the image was created using the old buggy logic), we
should use the deduced/legacy start_sector to override the HDIO_GETGEO result
for the duration of the resize calculation. This ensures the geometric
calculation matches the existing on-disk layout, preventing the unexpected
shrink and failure.
B. Maintain Legacy Bug (If deduction is too complex):
If implementing the reliable deduction logic (Proposal A) proves to be overly
complex or risky, would it be better to maintain the original, incorrect
HDIO_GETGEO behavior for resize.f2fs to prioritize compatibility with existing
images over geometric correctness?
3. Resource Constraints & Test Coverage
The root cause of this issue is that after the bug in HDIO_GETGEO was fixed,
the obtained value for start_sector differs from its previous value (which was
implicitly 0). Consequently, the zone_align_start_offset calculation, which
depends on start_sector, changed. Many crucial fields within the superblock, in
turn, depend on zone_align_start_offset.
Further analysis reveals that the zone_align_start_offset exhibits a periodic
change relative to start_sector. For instance, the zone_align_start_offset
generated when start_sector is in the range of 0 to 4095 is identical to the
one generated when start_sector is in the range of 4096 to 8191.
Based on this analysis, I have developed a test script that iterates through
start_sector values from 4096 to 8191 to observe the behavior produced by
resize.f2fs. The script is as follows:
#!/bin/bash
# Test script for resize.f2fs validation
# set -e
SOURCE_DIR="./data"
MOUNT_POINT="/mnt/f2fs-tmp"
SSD_DEVICE="/dev/sdb"
OLD_MKFS="mkfs.f2fs"
NEW_RESIZE="/path/to/fixed/resize.f2fs"
# Size of each test partition in GB
PARTITION_SIZE_GB=64
# Log file for all output
LOG_FILE="./test_resize.log"
# Statistics file for superblock inconsistencies
STATS_FILE="./sb_inconsistencies.txt"
# Output functions (output is automatically logged via exec redirect)
error() {
echo "ERROR: $1" >&2
exit 1
}
info() {
echo "INFO: $1"
}
warn() {
echo "WARN: $1"
}
# Parse superblock info from print_raw_sb_info output
parse_sb_info() {
local sb_file="$1"
# Extract values using awk
# Note: Format is "field_name [0x hex :
decimal]"
# The hex value may have leading spaces, so we match more flexibly
awk '
/^segment_count[[:space:]]+\[0x[[:space:]]*[[:xdigit:]]+[[:space:]]*:[[:space:]]*[[:digit:]]+\]/
{
match($0,
/\[0x[[:space:]]*([[:xdigit:]]+)[[:space:]]*:[[:space:]]*([[:digit:]]+)\]/, arr)
print "segment_count=" arr[2]
}
/^segment_count_ckpt[[:space:]]+\[0x[[:space:]]*[[:xdigit:]]+[[:space:]]*:[[:space:]]*[[:digit:]]+\]/
{
match($0,
/\[0x[[:space:]]*([[:xdigit:]]+)[[:space:]]*:[[:space:]]*([[:digit:]]+)\]/, arr)
print "segment_count_ckpt=" arr[2]
}
/^segment_count_sit[[:space:]]+\[0x[[:space:]]*[[:xdigit:]]+[[:space:]]*:[[:space:]]*[[:digit:]]+\]/
{
match($0,
/\[0x[[:space:]]*([[:xdigit:]]+)[[:space:]]*:[[:space:]]*([[:digit:]]+)\]/, arr)
print "segment_count_sit=" arr[2]
}
/^segment_count_nat[[:space:]]+\[0x[[:space:]]*[[:xdigit:]]+[[:space:]]*:[[:space:]]*[[:digit:]]+\]/
{
match($0,
/\[0x[[:space:]]*([[:xdigit:]]+)[[:space:]]*:[[:space:]]*([[:digit:]]+)\]/, arr)
print "segment_count_nat=" arr[2]
}
/^segment_count_ssa[[:space:]]+\[0x[[:space:]]*[[:xdigit:]]+[[:space:]]*:[[:space:]]*[[:digit:]]+\]/
{
match($0,
/\[0x[[:space:]]*([[:xdigit:]]+)[[:space:]]*:[[:space:]]*([[:digit:]]+)\]/, arr)
print "segment_count_ssa=" arr[2]
}
/^segment_count_main[[:space:]]+\[0x[[:space:]]*[[:xdigit:]]+[[:space:]]*:[[:space:]]*[[:digit:]]+\]/
{
match($0,
/\[0x[[:space:]]*([[:xdigit:]]+)[[:space:]]*:[[:space:]]*([[:digit:]]+)\]/, arr)
print "segment_count_main=" arr[2]
}
/^section_count[[:space:]]+\[0x[[:space:]]*[[:xdigit:]]+[[:space:]]*:[[:space:]]*[[:digit:]]+\]/
{
match($0,
/\[0x[[:space:]]*([[:xdigit:]]+)[[:space:]]*:[[:space:]]*([[:digit:]]+)\]/, arr)
print "section_count=" arr[2]
}
/^cp_payload[[:space:]]+\[0x[[:space:]]*[[:xdigit:]]+[[:space:]]*:[[:space:]]*[[:digit:]]+\]/
{
match($0,
/\[0x[[:space:]]*([[:xdigit:]]+)[[:space:]]*:[[:space:]]*([[:digit:]]+)\]/, arr)
print "cp_payload=" arr[2]
}
/^segment0_blkaddr[[:space:]]+\[0x[[:space:]]*[[:xdigit:]]+[[:space:]]*:[[:space:]]*[[:digit:]]+\]/
{
match($0,
/\[0x[[:space:]]*([[:xdigit:]]+)[[:space:]]*:[[:space:]]*([[:digit:]]+)\]/, arr)
print "segment0_blkaddr=" arr[2]
}
/^cp_blkaddr[[:space:]]+\[0x[[:space:]]*[[:xdigit:]]+[[:space:]]*:[[:space:]]*[[:digit:]]+\]/
{
match($0,
/\[0x[[:space:]]*([[:xdigit:]]+)[[:space:]]*:[[:space:]]*([[:digit:]]+)\]/, arr)
print "cp_blkaddr=" arr[2]
}
/^sit_blkaddr[[:space:]]+\[0x[[:space:]]*[[:xdigit:]]+[[:space:]]*:[[:space:]]*[[:digit:]]+\]/
{
match($0,
/\[0x[[:space:]]*([[:xdigit:]]+)[[:space:]]*:[[:space:]]*([[:digit:]]+)\]/, arr)
print "sit_blkaddr=" arr[2]
}
/^nat_blkaddr[[:space:]]+\[0x[[:space:]]*[[:xdigit:]]+[[:space:]]*:[[:space:]]*[[:digit:]]+\]/
{
match($0,
/\[0x[[:space:]]*([[:xdigit:]]+)[[:space:]]*:[[:space:]]*([[:digit:]]+)\]/, arr)
print "nat_blkaddr=" arr[2]
}
/^ssa_blkaddr[[:space:]]+\[0x[[:space:]]*[[:xdigit:]]+[[:space:]]*:[[:space:]]*[[:digit:]]+\]/
{
match($0,
/\[0x[[:space:]]*([[:xdigit:]]+)[[:space:]]*:[[:space:]]*([[:digit:]]+)\]/, arr)
print "ssa_blkaddr=" arr[2]
}
/^main_blkaddr[[:space:]]+\[0x[[:space:]]*[[:xdigit:]]+[[:space:]]*:[[:space:]]*[[:digit:]]+\]/
{
match($0,
/\[0x[[:space:]]*([[:xdigit:]]+)[[:space:]]*:[[:space:]]*([[:digit:]]+)\]/, arr)
print "main_blkaddr=" arr[2]
}
/^log_blocks_per_seg[[:space:]]+\[0x[[:space:]]*[[:xdigit:]]+[[:space:]]*:[[:space:]]*[[:digit:]]+\]/
{
match($0,
/\[0x[[:space:]]*([[:xdigit:]]+)[[:space:]]*:[[:space:]]*([[:digit:]]+)\]/, arr)
print "log_blocks_per_seg=" arr[2]
}
' "$sb_file"
}
# Fill partition with files using largest possible 2^n sizes
fill_partition() {
local mount_point="$1"
info "Filling partition at $mount_point"
# Get available space in bytes
local available_bytes
available_bytes=$(df -B1 "$mount_point" | tail -1 | awk '{print $4}')
if [ -z "$available_bytes" ] || [ "$available_bytes" -le 0 ]; then
error "Failed to get available space or no space available"
fi
info "Available space: $((available_bytes / 1024 / 1024 / 1024))GB
($available_bytes bytes)"
# Find the largest 2^n that is <= available space
# Start from 1 byte and multiply by 2 until we exceed available space, then
go back one step
local start_size_bytes=1 # Start from 1 byte
# Find the largest 2^n that fits (1, 2, 4, 8, 16, 32, 64, 128, 256, 512,
1K, 2K, 4K, 8K, ...)
while [ $((start_size_bytes * 2)) -le $available_bytes ]; do
start_size_bytes=$((start_size_bytes * 2))
done
# Display the starting size in appropriate units
if [ $start_size_bytes -ge $((1024 * 1024 * 1024)) ]; then
local start_size_gb=$((start_size_bytes / 1024 / 1024 / 1024))
info "Starting with ${start_size_gb}GB (largest 2^n that fits)"
elif [ $start_size_bytes -ge $((1024 * 1024)) ]; then
local start_size_mb=$((start_size_bytes / 1024 / 1024))
info "Starting with ${start_size_mb}MB (largest 2^n that fits)"
elif [ $start_size_bytes -ge 1024 ]; then
local start_size_kb=$((start_size_bytes / 1024))
info "Starting with ${start_size_kb}KB (largest 2^n that fits)"
else
info "Starting with ${start_size_bytes} bytes (largest 2^n that fits)"
fi
# Now fill with files: start from the calculated start_size_bytes and go
down
# For each size, try once. If space is not enough or creation fails, skip
to next smaller size
local file_index=0
local size_bytes=$start_size_bytes # Start from dynamically calculated size
local min_size=$((1024 * 512))
while [ $size_bytes -gt $min_size ]; do
# Check if we have enough space
local current_available
current_available=$(df -B1 "$mount_point" | tail -1 | awk '{print $4}')
if [ "$current_available" -lt "$size_bytes" ]; then
# Not enough space, skip this size and try next smaller size
info "Skipping ${size_bytes} bytes (not enough space:
$current_available available)"
size_bytes=$((size_bytes / 2))
continue
fi
# Try to create file using dd with 4KB block size
local file_path="$mount_point/fill_${file_index}_${size_bytes}.dat"
local size_gb=$((size_bytes / 1024 / 1024 / 1024))
local size_mb=$((size_bytes / 1024 / 1024))
local block_count=$((size_bytes / 4096)) # 4KB blocks
# Use dd with 4KB block size and direct I/O to bypass page cache
# oflag=direct: use direct I/O for output, bypassing page cache
if sudo dd if=/dev/urandom of="$file_path" bs=4096 count=$block_count
oflag=direct 2>/dev/null; then
if [ $size_gb -gt 0 ]; then
info "Created file: fill_${file_index}_${size_bytes}.dat
(${size_gb}GB)"
else
info "Created file: fill_${file_index}_${size_bytes}.dat
(${size_mb}MB)"
fi
# Sync to flush dirty pages and free page cache to reduce memory
pressure
sync
info "Synced file: fill_${file_index}_${size_bytes}.dat"
file_index=$((file_index + 1))
# Continue with next smaller size
size_bytes=$((size_bytes / 2))
else
# Failed to create (likely out of space), skip this size and try
next smaller
sudo rm -f "$file_path" 2>/dev/null
info "Failed to create ${size_bytes} bytes file, trying next
smaller size"
size_bytes=$((size_bytes / 2))
fi
done
info "Partition filled. Created $file_index files."
}
# Verify data integrity
verify_data() {
info "Verifying data integrity in $MOUNT_POINT"
# Check if both directories exist
if [ ! -d "$SOURCE_DIR" ]; then
error "Source directory $SOURCE_DIR does not exist"
fi
if [ ! -d "$MOUNT_POINT" ]; then
error "Mount point $MOUNT_POINT is not accessible"
fi
# Use diff -rq to quickly compare entire directory trees
# This is much faster than comparing files one by one
# -r: recursive, -q: quiet (only report if files differ)
local diff_output
if ! diff_output=$(diff -rq "$SOURCE_DIR" "$MOUNT_POINT" 2>&1); then
error "Data integrity check failed. Differences found:\n$diff_output"
fi
info "Data integrity verified: all files match between $SOURCE_DIR and
$MOUNT_POINT"
}
# Calculate expected values for variable fields
calculate_expected_values() {
local old_sb="$1"
local start_sector="$2"
# Load old SB values
eval $(parse_sb_info "$old_sb")
# Check if parsing succeeded - all required fields must be present
if [ -z "$segment0_blkaddr" ] || [ -z "$log_blocks_per_seg" ] || [ -z
"$segment_count_ckpt" ] || \
[ -z "$segment_count_sit" ] || [ -z "$segment_count_nat" ] || [ -z
"$segment_count_ssa" ]; then
error "Failed to parse superblock from $old_sb. File contents:\n$(head
-20 "$old_sb")"
fi
local old_segment0_blkaddr=$segment0_blkaddr
local blks_per_seg=$((2**log_blocks_per_seg))
# Print parsed values that are actually used (to stderr to avoid being
captured by eval)
echo "INFO: calculate_expected_values:
old_segment0_blkaddr=$old_segment0_blkaddr,
segment_count_ckpt=$segment_count_ckpt, segment_count_sit=$segment_count_sit,
segment_count_nat=$segment_count_nat, segment_count_ssa=$segment_count_ssa,
log_blocks_per_seg=$log_blocks_per_seg, blks_per_seg=$blks_per_seg" >&2
# Calculate alignment_bytes and zone_align_start_offset
local alignment_bytes=$((4096 * 512))
local zone_align_start_offset=$(( (start_sector * 512 + 2 * 4096 +
alignment_bytes - 1) / alignment_bytes * alignment_bytes - start_sector * 512 ))
# Calculate new segment0_blkaddr
# segment0_blkaddr = zone_align_start_offset / 4096, then round up to >=
old_segment0_blkaddr
local new_segment0_blkaddr=$((zone_align_start_offset / 4096))
while [ "$new_segment0_blkaddr" -lt "$old_segment0_blkaddr" ]; do
new_segment0_blkaddr=$((new_segment0_blkaddr + 512))
done
# Calculate other addresses based on new_segment0_blkaddr
local new_cp_blkaddr=$new_segment0_blkaddr
local new_sit_blkaddr=$((new_cp_blkaddr + segment_count_ckpt *
blks_per_seg))
local new_nat_blkaddr=$((new_sit_blkaddr + segment_count_sit *
blks_per_seg))
local new_ssa_blkaddr=$((new_nat_blkaddr + segment_count_nat *
blks_per_seg))
local new_main_blkaddr=$((new_ssa_blkaddr + segment_count_ssa *
blks_per_seg))
echo "expected_segment0_blkaddr=$new_segment0_blkaddr"
echo "expected_cp_blkaddr=$new_cp_blkaddr"
echo "expected_sit_blkaddr=$new_sit_blkaddr"
echo "expected_nat_blkaddr=$new_nat_blkaddr"
echo "expected_ssa_blkaddr=$new_ssa_blkaddr"
echo "expected_main_blkaddr=$new_main_blkaddr"
}
# Record inconsistency to stats file
record_inconsistency() {
local start_sector="$1"
local field="$2"
local old_value="$3"
local new_value="$4"
echo "[start_sector=$start_sector] $field: old=$old_value, new=$new_value"
>> "$STATS_FILE"
warn "Inconsistency detected: $field changed from $old_value to $new_value
(start_sector=$start_sector)"
}
# Verify superblock consistency
verify_sb_consistency() {
local old_sb_file="$1"
local new_sb_file="$2"
local start_sector="$3"
info "Verifying superblock consistency"
# Parse both superblocks
eval $(parse_sb_info "$old_sb_file")
# Check if parsing succeeded - all required fields must be present
if [ -z "$segment_count" ] || [ -z "$segment_count_ckpt" ] || [ -z
"$segment_count_sit" ] || \
[ -z "$segment_count_nat" ] || [ -z "$segment_count_ssa" ] || [ -z
"$segment_count_main" ] || \
[ -z "$section_count" ] || [ -z "$cp_payload" ]; then
error "Failed to parse old superblock from $old_sb_file. File
contents:\n$(head -20 "$old_sb_file")"
fi
local old_segment_count=$segment_count
local old_segment_count_sit=$segment_count_sit
local old_segment_count_nat=$segment_count_nat
local old_segment_count_ssa=$segment_count_ssa
local old_segment_count_main=$segment_count_main
local old_section_count=$section_count
local old_cp_payload=$cp_payload
local old_segment_count_ckpt=$segment_count_ckpt
eval $(parse_sb_info "$new_sb_file")
# Check if parsing succeeded - all required fields must be present
if [ -z "$segment_count" ] || [ -z "$segment_count_ckpt" ] || [ -z
"$segment_count_sit" ] || \
[ -z "$segment_count_nat" ] || [ -z "$segment_count_ssa" ] || [ -z
"$segment_count_main" ] || \
[ -z "$section_count" ] || [ -z "$cp_payload" ]; then
error "Failed to parse new superblock from $new_sb_file. File
contents:\n$(head -20 "$new_sb_file")"
fi
local new_segment_count=$segment_count
local new_segment_count_sit=$segment_count_sit
local new_segment_count_nat=$segment_count_nat
local new_segment_count_ssa=$segment_count_ssa
local new_segment_count_main=$segment_count_main
local new_section_count=$section_count
local new_cp_payload=$cp_payload
local new_segment_count_ckpt=$segment_count_ckpt
# Print parsed values for debugging
info "Parsed old SB: segment_count=$old_segment_count,
segment_count_sit=$old_segment_count_sit,
segment_count_nat=$old_segment_count_nat,
segment_count_ssa=$old_segment_count_ssa,
segment_count_main=$old_segment_count_main, section_count=$old_section_count,
cp_payload=$old_cp_payload, segment_count_ckpt=$old_segment_count_ckpt"
info "Parsed new SB: segment_count=$new_segment_count,
segment_count_sit=$new_segment_count_sit,
segment_count_nat=$new_segment_count_nat,
segment_count_ssa=$new_segment_count_ssa,
segment_count_main=$new_segment_count_main, section_count=$new_section_count,
cp_payload=$new_cp_payload, segment_count_ckpt=$new_segment_count_ckpt"
# Verify unchanged fields (record inconsistencies instead of exiting)
if [ "$old_segment_count" != "$new_segment_count" ]; then
record_inconsistency "$start_sector" "segment_count"
"$old_segment_count" "$new_segment_count"
fi
if [ "$old_segment_count_sit" != "$new_segment_count_sit" ]; then
record_inconsistency "$start_sector" "segment_count_sit"
"$old_segment_count_sit" "$new_segment_count_sit"
fi
if [ "$old_segment_count_nat" != "$new_segment_count_nat" ]; then
record_inconsistency "$start_sector" "segment_count_nat"
"$old_segment_count_nat" "$new_segment_count_nat"
fi
if [ "$old_segment_count_ssa" != "$new_segment_count_ssa" ]; then
record_inconsistency "$start_sector" "segment_count_ssa"
"$old_segment_count_ssa" "$new_segment_count_ssa"
fi
if [ "$old_segment_count_main" != "$new_segment_count_main" ]; then
record_inconsistency "$start_sector" "segment_count_main"
"$old_segment_count_main" "$new_segment_count_main"
fi
if [ "$old_section_count" != "$new_section_count" ]; then
record_inconsistency "$start_sector" "section_count"
"$old_section_count" "$new_section_count"
fi
if [ "$old_cp_payload" != "$new_cp_payload" ]; then
record_inconsistency "$start_sector" "cp_payload" "$old_cp_payload"
"$new_cp_payload"
fi
if [ "$old_segment_count_ckpt" != "$new_segment_count_ckpt" ]; then
record_inconsistency "$start_sector" "segment_count_ckpt"
"$old_segment_count_ckpt" "$new_segment_count_ckpt"
fi
info "Unchanged fields verified"
}
# Verify variable fields
verify_variable_fields() {
local old_sb_file="$1"
local new_sb_file="$2"
local start_sector="$3"
info "Verifying variable fields"
# Parse new SB values
eval $(parse_sb_info "$new_sb_file")
# Check if parsing succeeded - all required fields must be present
if [ -z "$segment0_blkaddr" ] || [ -z "$cp_blkaddr" ] || [ -z
"$sit_blkaddr" ] || \
[ -z "$nat_blkaddr" ] || [ -z "$ssa_blkaddr" ] || [ -z "$main_blkaddr"
]; then
error "Failed to parse new superblock from $new_sb_file. File
contents:\n$(head -20 "$new_sb_file")"
fi
local new_segment0_blkaddr=$segment0_blkaddr
local new_cp_blkaddr=$cp_blkaddr
local new_sit_blkaddr=$sit_blkaddr
local new_nat_blkaddr=$nat_blkaddr
local new_ssa_blkaddr=$ssa_blkaddr
local new_main_blkaddr=$main_blkaddr
# Calculate expected values (stdout for eval, stderr for info messages)
eval $(calculate_expected_values "$old_sb_file" "$start_sector")
# Print parsed and expected values for debugging
info "Parsed new SB variable fields:
segment0_blkaddr=$new_segment0_blkaddr, cp_blkaddr=$new_cp_blkaddr,
sit_blkaddr=$new_sit_blkaddr, nat_blkaddr=$new_nat_blkaddr,
ssa_blkaddr=$new_ssa_blkaddr, main_blkaddr=$new_main_blkaddr"
info "Expected variable fields :
segment0_blkaddr=$expected_segment0_blkaddr, cp_blkaddr=$expected_cp_blkaddr,
sit_blkaddr=$expected_sit_blkaddr, nat_blkaddr=$expected_nat_blkaddr,
ssa_blkaddr=$expected_ssa_blkaddr, main_blkaddr=$expected_main_blkaddr"
# Record inconsistencies instead of exiting
if [ "$new_segment0_blkaddr" != "$expected_segment0_blkaddr" ]; then
record_inconsistency "$start_sector" "segment0_blkaddr"
"$expected_segment0_blkaddr" "$new_segment0_blkaddr"
fi
if [ "$new_cp_blkaddr" != "$expected_cp_blkaddr" ]; then
record_inconsistency "$start_sector" "cp_blkaddr"
"$expected_cp_blkaddr" "$new_cp_blkaddr"
fi
if [ "$new_sit_blkaddr" != "$expected_sit_blkaddr" ]; then
record_inconsistency "$start_sector" "sit_blkaddr"
"$expected_sit_blkaddr" "$new_sit_blkaddr"
fi
if [ "$new_nat_blkaddr" != "$expected_nat_blkaddr" ]; then
record_inconsistency "$start_sector" "nat_blkaddr"
"$expected_nat_blkaddr" "$new_nat_blkaddr"
fi
if [ "$new_ssa_blkaddr" != "$expected_ssa_blkaddr" ]; then
record_inconsistency "$start_sector" "ssa_blkaddr"
"$expected_ssa_blkaddr" "$new_ssa_blkaddr"
fi
if [ "$new_main_blkaddr" != "$expected_main_blkaddr" ]; then
record_inconsistency "$start_sector" "main_blkaddr"
"$expected_main_blkaddr" "$new_main_blkaddr"
fi
info "Variable fields verified"
}
# Cleanup function
cleanup() {
sudo umount "$MOUNT_POINT" 2>/dev/null || true
}
# Main test function for a single partition
test_partition() {
local start_sector="$1"
local partition_dev="${SSD_DEVICE}1" # Always use partition 1
local old_sb_file="old_sb_${start_sector}.txt"
local new_sb_file="new_sb_${start_sector}.txt"
# Setup cleanup trap
trap "cleanup" EXIT INT TERM
info "Testing partition starting at sector $start_sector (partition:
$partition_dev)"
# Calculate partition end sector (GB to sectors: GB * 1024 * 1024 * 1024 /
512)
# end_sector is inclusive, so we need to subtract 1
local end_sector=$((start_sector + PARTITION_SIZE_GB * 1024 * 1024 * 1024 /
512 - 1))
# Check if partition would exceed device size
local device_size_sectors=$(sudo blockdev --getsz "$SSD_DEVICE" 2>/dev/null
|| echo "0")
if [ "$device_size_sectors" -gt 0 ] && [ "$end_sector" -ge
"$device_size_sectors" ]; then
error "Partition end sector $end_sector exceeds device size
$device_size_sectors"
fi
# Delete existing partition if it exists (we'll always use partition 1)
sudo parted -s "$SSD_DEVICE" rm 1 2>/dev/null || true
# Create new partition starting at start_sector (always as partition 1)
info "Creating partition: start=${start_sector}s, end=${end_sector}s"
local parted_output
parted_output=$(sudo parted -s "$SSD_DEVICE" mkpart primary
${start_sector}s ${end_sector}s 2>&1)
local parted_status=$?
if [ $parted_status -ne 0 ]; then
error "Failed to create partition (start=${start_sector}s,
end=${end_sector}s): $parted_output"
fi
# Wait for kernel to recognize the new partition
sudo partprobe "$SSD_DEVICE" 2>/dev/null || true
# Verify partition exists (always partition 1)
partition_dev="${SSD_DEVICE}1"
if [ ! -b "$partition_dev" ]; then
error "Partition $partition_dev does not exist after creation"
fi
# Verify partition start and end sectors match our configuration
# Use parted -m for machine-readable output
local actual_info=$(sudo parted -s -m "$SSD_DEVICE" unit s print | grep
"^1:" || error "Failed to get partition info")
local actual_start=$(echo "$actual_info" | cut -d: -f2 | sed 's/s$//')
local actual_end=$(echo "$actual_info" | cut -d: -f3 | sed 's/s$//')
if [ -z "$actual_start" ] || [ -z "$actual_end" ]; then
error "Failed to parse partition start/end sectors"
fi
if [ "$actual_start" != "$start_sector" ]; then
error "Partition start sector mismatch: expected $start_sector, got
$actual_start"
fi
if [ "$actual_end" != "$end_sector" ]; then
error "Partition end sector mismatch: expected $end_sector, got
$actual_end"
fi
info "Partition verified: start=$actual_start, end=$actual_end"
# Create filesystem with old mkfs
info "Creating filesystem on $partition_dev"
local mkfs_output
mkfs_output=$(sudo "$OLD_MKFS" -f "$partition_dev" 2>&1)
local mkfs_status=$?
if [ $mkfs_status -ne 0 ]; then
error "Failed to create filesystem on $partition_dev: $mkfs_output"
fi
info "Filesystem created successfully"
# Mount and fill partition with data
sudo mkdir -p "$MOUNT_POINT"
sudo mount -t f2fs "$partition_dev" "$MOUNT_POINT" || error "Failed to
mount"
# Check if SOURCE_DIR already exists (from previous test)
if [ -d "$SOURCE_DIR" ] && [ -n "$(ls -A "$SOURCE_DIR" 2>/dev/null)" ]; then
# SOURCE_DIR exists and has files, copy from SOURCE_DIR to mount point
info "SOURCE_DIR already exists, copying files from $SOURCE_DIR to
$MOUNT_POINT"
sudo cp -r "$SOURCE_DIR"/* "$MOUNT_POINT/" || error "Failed to copy
files from $SOURCE_DIR to $MOUNT_POINT"
else
# SOURCE_DIR doesn't exist, fill partition and copy to SOURCE_DIR
fill_partition "$MOUNT_POINT"
# Copy filled files to SOURCE_DIR as backup for verification
info "Copying filled files to $SOURCE_DIR for verification"
rm -rf "$SOURCE_DIR"
mkdir -p "$SOURCE_DIR"
sudo cp -r "$MOUNT_POINT"/* "$SOURCE_DIR/" || error "Failed to copy
files to $SOURCE_DIR"
sudo chown -R $(whoami) "$SOURCE_DIR" || true # Change ownership so we
can access later
fi
sudo umount "$MOUNT_POINT" || error "Failed to umount"
# Run resize and capture output
# -F: skip interactive input, -f: force, -d1: debug level 1
# Output is automatically logged via exec redirect, but we also save to
file for parsing
info "Running resize for partition starting at sector $start_sector"
sudo "$NEW_RESIZE" -F -f -d1 "$partition_dev" 2>&1 | tee
"resize_output_${start_sector}.txt" || error "Failed to resize"
# Ensure resize output contains three superblocks (mount + old + new)
local sb_total
sb_total=$(grep -c '^\| Super block' "resize_output_${start_sector}.txt" ||
echo 0)
if [ "$sb_total" -lt 3 ]; then
error "resize output incomplete for sector $start_sector: expected at
least 3 super blocks, got $sb_total"
fi
# Extract superblock info from output
# Use "| Super block" as unique identifier to avoid conflicts with other
output
# Skip the first superblock (from mount), extract the 2nd (old) and 3rd
(new) ones
awk '
BEGIN {
sb_count=0
in_sb=0
prev_line=""
}
/^\| Super block/ {
# Start of new superblock section
sb_count++
# Skip the first superblock (from mount), only process 2nd and 3rd
if (sb_count == 1) {
in_sb=0
prev_line = $0
next
}
# Process 2nd (old) and 3rd (new) superblocks
in_sb=1
# Include the separator line before "| Super block"
if (prev_line ~
/^\+--------------------------------------------------------\+/) {
print prev_line > (sb_count == 2 ? "'"$old_sb_file"'" :
"'"$new_sb_file"'")
}
print > (sb_count == 2 ? "'"$old_sb_file"'" : "'"$new_sb_file"'")
prev_line = $0
next
}
in_sb && sb_count >= 2 {
# Collect all lines until next "| Super block" or end of file
print > (sb_count == 2 ? "'"$old_sb_file"'" : "'"$new_sb_file"'")
prev_line = $0
next
}
{
prev_line = $0
}
' "resize_output_${start_sector}.txt"
# Check if files were created and have content
if [ ! -s "$old_sb_file" ]; then
error "Failed to extract old superblock. Output file size: $(wc -l <
"$old_sb_file" 2>/dev/null || echo 0) lines. First 20 lines of resize
output:\n$(head -20 "resize_output_${start_sector}.txt")"
fi
if [ ! -s "$new_sb_file" ]; then
error "Failed to extract new superblock. Output file size: $(wc -l <
"$new_sb_file" 2>/dev/null || echo 0) lines"
fi
# Verify superblock consistency
verify_sb_consistency "$old_sb_file" "$new_sb_file" "$start_sector"
# Verify variable fields
verify_variable_fields "$old_sb_file" "$new_sb_file" "$start_sector"
# Mount and verify data
sudo mount -t f2fs "$partition_dev" "$MOUNT_POINT" || error "Failed to
mount after resize"
verify_data
sudo umount "$MOUNT_POINT" || error "Failed to umount after resize"
# Cleanup files (partition cleanup handled by trap)
# rm -f "$old_sb_file" "$new_sb_file" "resize_output_${start_sector}.txt"
# Remove trap
trap - EXIT INT TERM
info "Partition test $start_sector completed successfully"
}
# Main execution
main() {
# Redirect all output to both terminal and log file
exec > >(tee "$LOG_FILE") 2>&1
echo "=== Test started at $(date) ==="
echo ""
echo "=== Configuration ==="
echo "SOURCE_DIR: $SOURCE_DIR"
echo "MOUNT_POINT: $MOUNT_POINT"
echo "SSD_DEVICE: $SSD_DEVICE"
echo "OLD_MKFS: $OLD_MKFS"
echo "NEW_RESIZE: $NEW_RESIZE"
echo "PARTITION_SIZE_GB: $PARTITION_SIZE_GB"
echo "LOG_FILE: $LOG_FILE"
echo "STATS_FILE: $STATS_FILE"
echo ""
# Check prerequisites
command -v "$OLD_MKFS" >/dev/null 2>&1 || error "mkfs.f2fs command not
found: $OLD_MKFS"
[ -f "$NEW_RESIZE" ] || error "New resize.f2fs not found: $NEW_RESIZE"
# Check if SSD_DEVICE is set
if [ -z "$SSD_DEVICE" ]; then
error "SSD_DEVICE is not set. Please set it to your SSD device (e.g.,
/dev/sdb)"
error "Usage: SSD_DEVICE=/dev/sdb $0"
exit 1
fi
# Verify SSD device exists
if [ ! -b "$SSD_DEVICE" ]; then
error "SSD device $SSD_DEVICE does not exist or is not a block device"
fi
# Warn user about data destruction
warn "WARNING: This script will create and destroy partitions on
$SSD_DEVICE"
warn "All existing data on $SSD_DEVICE will be lost!"
echo ""
read -p "Press Enter to continue or Ctrl+C to abort..."
info "Starting resize.f2fs test suite"
# Initialize statistics file
echo "=== Superblock Inconsistencies ===" > "$STATS_FILE"
echo "Format: [start_sector=X] field: old=value, new=value" >> "$STATS_FILE"
echo "" >> "$STATS_FILE"
# Initialize partition table (only once)
warn "Initializing partition table on $SSD_DEVICE"
sudo parted -s "$SSD_DEVICE" mklabel gpt 2>/dev/null || error "Failed to
create partition table"
# Test each partition (start from sector 4096, as GPT requires partitions
to start after sector 34)
local total_tests=4096
local current_test=0
for start_sector in {4096..8191}; do
current_test=$((current_test + 1))
info "Progress: $current_test/$total_tests (sector $start_sector)"
test_partition "$start_sector"
done
info "All tests completed successfully"
}
# Run main function
main "$@"
But as an individual researcher with limited hardware, my testing environment
is currently constrained by my academic work.
Please note that I only have access to a single SSD around 100GB for my
testing, which limits the scope of my regression validation (especially for
large-scale performance or capacity checks). Given your corporate environment,
your team would be much better positioned to run extensive, large-scale
regression tests on the final fix. Of course, whenever I have spare time, I
will also continue running this script for further testing.
I kindly request that you use these reproducers and the test script to validate
the fix on your end once implemented. If there is anything further I can do to
assist with the analysis or implementation, please do not hesitate to reach out.
Note on Script Validation Logic:
The validation logic within this script for checking new vs. old superblock
fields (specifically the difference in field values) may not be perfectly
accurate or comprehensive. However, this is acceptable, as these checks are
primarily for informational purposes and will not be used as the definitive
criteria for judging the bug.
The true criteria for bug judgment remain:
1. Whether resize.f2fs executes successfully without failure.
2. More importantly, whether the original created files remains fully
consistent after the execution of resize.f2fs.
Please feel free to reach out to me if you have any questions or require
clarification regarding the script or the expected outcomes.
Thanks,
Xiaole He
_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
