I have been using BackupPC very successfully for years. Then a few days
ago I noticed that there were no recent backups. On investigation, I
found that the pool filesystem had been mounted read-only.
My BackupPC server is running version 4.4.0 on a Proxmox lxc container.
The container has a mount point at /var/backuppc to pass the pve host
mount point at /nfs/bpc
The pve host uses autofs to automount micros:/srv/nfs/backuppc at /nfs/bps
micros is a file server running OpenSuSE 15.2 with kernel version 5.3.18
It was the kernel on micros that had remounted the filesystem on
/dev/vgdata/lvbackuppc as read-only.
On further investigation of the system journal on micros, I found the
following entries:
Apr 22 18:25:53 micros kernel: Aborting journal on device dm-8-8.
Apr 22 18:25:53 micros kernel: EXT4-fs error (device dm-8):
ext4_journal_check_start:61: Detected aborted journal
Apr 22 18:34:09 micros kernel: EXT4-fs (dm-8): mounted filesystem
without journal. Opts: acl,user_xattr
I checked for failing discs on the raid 5 array that underpins the
logical volumes and their filesystems that this server exports.
Discovered one drive with uncorrectable errors and another with pending
reallocated sectors.
I replaced both of these drives, but still had the problem of the
aborting journal.
Removing the journal completely worked, but unacceptable.
I created an external journal and this has worked for the last few days.
The external journal is on /dev/vgsys/lvbkpcjnl
lvcreate -L 1G -n lvbkpcjnl vgsys
mke2fs -O journal_dev /dev/vgsys/lvbkpcjnl
tune2fs -j -J device=/dev/vgsys/lvbkpcjnl /dev/vgdata/lvbackuppc
When mounting this filesystem, I use the mount option data=journal
(although the problem was still present with data=ordered when the
journal was internal).
None of the other filesystems on this file server exhibit the same
issue, even though they are all ext4 with internal journals.
Has anyone come across this situation?
Thanks,
Graham.
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
