Hi Uwe, Josip, all,
Am 31.10.2022 um 14:37 schrieb Josip Deanovic:
On 2022-10-31 12:46, Uwe Schuerkamp wrote:
Hi folks,
due to user error part of a file system on one of our backup clients
was compressed by a find command gone haywire. Among them are
/usr/bin/bash & others (kernel, initrd, grub, root, authorized_keys
etc), leading to a situation where we cannot log into the system
remotely nor via the vmware console function anymore.
I would assume that also quite a few libraries are now compressed.
I've tried to restore /usr/bin/bash by its uncompressed version as the
bacula-fd on the client is still running (today's incr. backup worked
fine, too), but once the restore starts I'm getting the following
error:
31-Oct 12:23 deniol2378-sd JobId 23316: Forward spacing Volume
"deXXXXX-0183" to addr=269
31-Oct 12:23 deniol2378-sd JobId 23316: Elapsed time=00:00:01,
Transfer rate=484.9 K Bytes/second
31-Oct 12:22 bacula-fd JobId 23316: Error: create_file.c:227 Could not
create /usr/bin/bash: ERR=Permission denied
That looks like a real challenge... however, for practical reasons, I
would tend to agree with Josip: Boot from recovery media, and start
fixing from there (possibly even reverting the find-triggered
compression instead of restoring).
The actual problem might be related to any sort of libraries or
configuration no longer usable -- user authentication, permission
checks, ACL processing may all need to do all sorts of getpw* calls.
...
Replace: Always
Start time: 31-Oct-2022 12:23:02
End time: 31-Oct-2022 12:23:02
Elapsed time: 1 sec
Files Expected: 1
Files Restored: 1
Bytes Restored: 0 (0 B)
Rate: 0.0 KB/s
FD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Restore OK
That does not look correct, although a system that messed up is no good
foundation for a bug report :-)
I'm clutching at straws here naturally, however before we rebuild the
VM in question from a full backup I'd like to try as many options as
possible.
What makes this case even worse is that LVM is being used for / &
other fileystems, so we cannot simply mount those drives on another VM
(3 physical volumes) to repair the damage, right?
Might actually be possible, although a bit risky. But there are
snapshots for that purpose :-)
Thanks for reading this far & for your help,
Hello,
It might be that your file systems are now mounted read-only and
bacula-fd cannot write to file systems.
Are you able to connect to the system and run any command?
Unlikely, I think.
If not, you will have to boot the system using boot iso image,
prepare network, prepare LVM, mount file systems, copy the necessary
bacula files, chroot into the file system, start bacula-fd and
restore the data.
Or give systemrescuecd a try. You should be able to boot that from an
image / virtual CD, and volume management, file system maintenance, and
(de)compression tools should be available.
This sort of problem is possibly more quickly recovered from using a new
system installation, FD installation, and then restoration of the actual
data. The necessary forensic work is just too time consuming (for me at
least...)
Good luck!
Arno
--
Arno Lehmann
IT-Service Lehmann
Sandstr. 6, 49080 Osnabrück
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
