Public bug reported:
## Summary
After upgrading from kernel 6.17.0-1019-aws to 7.0.0-1009-aws, the NFS
client can no longer list (readdir/opendir) any subdirectory on an
NFSv4.1 mount. All subdirectory listings return "Remote I/O error"
(EREMOTEIO). File reads, stat(), and file creation in the root directory
continue to work. Rolling back to 6.17.0-1019-aws immediately resolves
the issue.
## Affected version
- linux-image-7.0.0-1009-aws 7.0.0-1009.9~24.04.1
- Ubuntu 24.04.4 LTS (Resolute HWE kernel)
- Architecture: amd64 (AWS EC2)
## Last known working version
- linux-image-6.17.0-1019-aws 6.17.0-1019.19~24.04.1
## NFS server
- NFS-Ganesha V7.3 (inside Longhorn share-manager v1.11.2 container)
- NFSv4.1, TCP, VFS FSAL
- Backing filesystem: XFS on iSCSI block device
## NFS client mount options
nfs4
(rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,softerr,softreval,noresvport,fatal_neterrors=none,proto=tcp,timeo=600,retrans=5,sec=sys,local_lock=none)
## Steps to reproduce
1. Boot with kernel 7.0.0-1009-aws
2. Mount an NFSv4.1 export from NFS-Ganesha (Longhorn RWX volume share-manager)
3. Create or access any subdirectory on the mount
## Observed behavior
```
# Root directory works
$ ls /mnt/nfs/
bootstrap_api_key configs mnesia testfile
# stat on subdirectory works
$ stat /mnt/nfs/configs/
File: /mnt/nfs/configs/
Size: 4096 Blocks: 8 IO Block: 1048576 directory
# Reading a file inside a subdirectory works
$ cat /mnt/nfs/configs/app.config
[{ekka, ... # file content returned successfully
# But listing ANY subdirectory fails
$ ls /mnt/nfs/configs/
ls: cannot open directory '/mnt/nfs/configs/': Remote I/O error
$ ls /mnt/nfs/mnesia/
ls: cannot open directory '/mnt/nfs/mnesia/': Remote I/O error
# Even a freshly created directory fails
$ mkdir /mnt/nfs/testdir # succeeds
$ ls /mnt/nfs/testdir/
ls: cannot access '/mnt/nfs/testdir/..': Remote I/O error
total 0
drwxr-xr-x 2 root root 6 Jul 30 14:06 .
d????????? ? ? ? ? ? ..
```
## Expected behavior
ls on subdirectories should return directory contents, as it does with kernel
6.17.0-1019-aws.
## Key observations
- opendir() / getdents64() on subdirectories returns EREMOTEIO
- stat() on subdirectories works (NFS GETATTR succeeds)
- open() + read() on files inside subdirectories works (NFS LOOKUP + OPEN +
READ succeeds)
- mkdir() and touch in the root directory work (NFS CREATE succeeds)
- The NFS server (NFS-Ganesha) is unaffected — local access from the server pod
works perfectly
- The underlying block device has no I/O errors
- No errors in dmesg related to NFS
- Only affects NFSv4.1 mounts (RWX volumes); iSCSI-direct mounts (RWO volumes)
are unaffected
## Impact
This regression breaks any workload using Kubernetes ReadWriteMany (RWX)
PersistentVolumeClaims backed by Longhorn, which uses NFS-Ganesha for RWX
access. In our case, EMQX (MQTT broker) enters CrashLoopBackOff because its
RocksDB/mnesia storage requires directory listing to initialize.
## Workaround
Roll back to kernel 6.17.0-1019-aws and hold the 7.0 package:
```
sudo sed -i 's/^GRUB_DEFAULT=.*/GRUB_DEFAULT="Advanced options for
Ubuntu>Ubuntu, with Linux 6.17.0-1019-aws"/' /etc/default/grub
sudo update-grub
sudo apt-mark hold linux-image-7.0.0-1009-aws linux-image-aws
sudo reboot
```
## Environment details
- 3-node Kubernetes cluster (kubeadm, server v1.35.6 / kubelet v1.35.2)
- Longhorn v1.11.2 (storage)
- NFS-Ganesha inside longhornio/longhorn-share-manager:v1.11.2
- kured auto-rebooted nodes into the new kernel between 02:00-06:00 UTC on
2026-07-29/30
- Two independent clusters (opa-lab and opa-dev) both affected simultaneously
after the kernel upgrade
** Affects: linux-aws (Ubuntu)
Importance: Undecided
Status: New
** Tags: kernel-bug-exists-upstream regression-release
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2162365
Title:
linux-aws 7.0.0-1009: NFS readdir on NFSv4.1 subdirectories returns
EREMOTEIO
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-aws/+bug/2162365/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
