> On 23 Jan 2026, at 16:33, Ming Lei <[email protected]> wrote:
>
> On Fri, Jan 23, 2026 at 03:59:31PM +0200, Alexander Atanasov wrote:
>> On 23 Jan 2026, at 15:33, Ming Lei <[email protected]> wrote:
>>>
>>> On Fri, Jan 23, 2026 at 11:20:36AM +0000, Alexander Atanasov wrote:
>>>> Create a temp dir for temporary files and use it instead of
>>>> placing them inside source tree.
>>>
>>> Many temporary files are backing files of file storage target, so far
>>> the code requires O_DIRECT, or the size could be a bit big.
>>>
>>> In case of ramfs/tmpfs of temp dir, it may cause problem for tests.
>>>
>>
>> I am aware of O_DIRECT problem but you can export different TMPDIR that has
>> working O_DIRECT.
>
> Can you share how to export TMPDIR capable of O_DIRECT?
+export TMPDIR=$(mktemp -d ${TMPDIR:-/tmp}/ublktest-dir.XXXXXX)
I made the tests to run in own TMPDIR. Which is under already set TMPDIR or
if TMPDIR is not set it is defaults to /tmp.
export TMPDIR=/path/to/odirect/capable
Before running and tests will run in:
/path/to/odirect/capable/ublktest-dir.XXXXXX
>
>>
>> I use sshfs mount of the build to run the tests and that is a problem
>> sshfs/fuse does not
>> do O_DIRECT too.
>>
>> I think test_generic_06.sh is the only one that fails due to this(thou I
>> still have to investigate).
>>
>> If O_DIRECT is required by the tests it may be possible to go thru a RAM
>> disk which does support it,
>> so it works eveerywhere
>>
>> Other option is to preserve working in source tree as it is now, and just
>> add a variable to specify working directory -
>> UBLK_TMPDIR or something.
>>
>>
>> I get a lot of out of order io - between 0 and 10 on average on my test
>> setup:
>> tools/testing/selftests/ublk/test_generic_01.sh
>> Attached 3 probes
>> io_out_of_order: exp 564688 actual 564648
>> io_out_of_order: exp 564648 actual 565584
>> io_out_of_order: exp 565584 actual 564688
>> io_out_of_order: exp 565592 actual 564688
>> io_out_of_order: exp 566328 actual 565592
>> io_out_of_order: exp 882256 actual 882248
>> io_out_of_order: exp 883032 actual 882912
>> io_out_of_order: exp 882912 actual 883040
>> io_out_of_order: exp 883040 actual 883032
>>
>>
>> generic_01 : [FAIL]
>>
>> All rq-s are there just reordered , AFAIK blk-mq does not guarantee that
>> requests will be completed in order, what’s the idea to catch this and
>
> If there is just 0 ~ 10, it could be fine. But if all are reorderd,
> something must be wrong. One improvement could be check if there is too
> many reorder...
>
> Actually what I am trying to test is to make sure same order is observed
> from both ublk driver dispatch code path and ublk target io handling code
> path, because io_uring task work schedule uses llist, which may introduce io
> reorder.
There are for sure other places where a reordering can be introduced, so the
code should be ready and expecting
It. (For my case see bellow) Is preserving the order required for some reason
for ublk?
>
> However, that involves ublk kprobe/kfunc trace, which may not be stable,
> so I simply check the end-to-end IO order. Sometimes blk-mq IO queue/dispatch
> may re-order IO.
>
> I guess the following change may avoid the re-order, but batch IO case may
> not be covered:
>
> diff --git a/tools/testing/selftests/ublk/test_generic_01.sh
> b/tools/testing/selftests/ublk/test_generic_01.sh
> index 21a31cd5491a..5805da4c84c5 100755
> --- a/tools/testing/selftests/ublk/test_generic_01.sh
> +++ b/tools/testing/selftests/ublk/test_generic_01.sh
> @@ -29,14 +29,8 @@ if ! kill -0 "$btrace_pid" > /dev/null 2>&1; then
> exit "$UBLK_SKIP_CODE"
> fi
>
> -# run fio over this ublk disk
> -fio --name=write_seq \
> - --filename=/dev/ublkb"${dev_id}" \
> - --ioengine=libaio --iodepth=16 \
> - --rw=write \
> - --size=512M \
> - --direct=1 \
> - --bs=4k > /dev/null 2>&1
> +taskset -c 0 dd if=/dev/zero of=/dev/ublkb"${dev_id}" bs=1M count=256
> oflag=direct > /dev/null 2>&1
> +
>
>
>> consider it an error? (Latest tree with batch io and batch io fixes on top
>> of if that matters)
>
> Never observe generic_01 failure in my test VM and hardware.
>
> My kernel config is based on Fedora, maybe scheduler config option makes the
> difference.
Fedora 43 default config with some debugging options enabled, but no changes in
schedulers.
Test VM storage is on a networked NAS over iSCSI - both boxes VM host and NAS
have two NICs,
I get the errors when I load the network. So I believe the requests really
complete out of
order due to the network in my case. All tests that have the bpftrace check
fail on occasion.
Regards,
Alexander Atanasov
