odd phantom directory
Hi all, I ran into a rather odd issue this morning with my FreeBSD 9.0-Release system running ZFS v28. This system serves as an RSYNC host which all of our other systems back up to each night. Last night, I started getting the following error: file has vanished: /backup/ldap1/etc/pki Now, usually when I get a file has vanished error during an RSYNC run, it indicates that the source file/directory on the system that is sending the rsync backup has been deleted or moved before rsync got a chance to actually send it. That doesn't appear to be the case here. /backup/ldap1/etc/pki is the destination directory on my Freebsd/ZFS server. I take a look in /backup/ldap1/etc on my Freebsd server and the pki subdirectory is no longer listed. Ok, so I run mkdir /backup/ldap1/etc/pki and get the following error: mkdir: /backup/ldap1/etc/pki: File exists. Odd Just to double check, I run ls -la /backup/ldap1/etc/pki and get the following: ls: /backup/ldap1/etc/pki: No such file or directory Alright, how about a simple touch? touch: /backup/ldap1/etc/pki: No such file or directory Fine. Maybe there is something funky about the /backup/ldap1/etc directory that is preventing me from doing any of this. mkdir /backup/ldap1/etc/pki2. That works just fine. What the heck? Looking at the output of my daily security run, I see the following: Checking setuid files and devices: find: /backup/ldap1/etc/fonts/conf.avail: No such file or directory find: /backup/ldap1/etc/fonts/conf.d/30-metric-aliases.conf: No such file or directory find: /backup/ldap1/etc/pki: No such file or directory So, it looks like there are a few files/directories in /backup/ldap1/etc that were affected. Looking through dmesg and /var/log/messages, I don't see anything out of the ordinary. I'm running a zpool scrub now just to be on the safe side, but I haven't seen any checksum or other errors so far. Any thoughts as to what might be causing this? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
RE: odd phantom directory
It looks like this may be the same issue as reported here: http://lists.freebsd.org/pipermail/freebsd-current/2011-October/027902.html but that thread seems to have just died off about a year ago. Zfs scrub is still running, but not reported errors so far. I'm going to run a zdb -ccv backup once that is done. From looking over this other thread, I tried just a simple ls /backup/ldap1/etc and /backup/ldap1/etc/pki does show up if I do ls without any arguments. If I do an ls -l then it doesn't show up. -Original Message- From: owner-freebsd-questi...@freebsd.org [mailto:owner-freebsd-questi...@freebsd.org] On Behalf Of Brian Gold Sent: Friday, November 16, 2012 9:37 AM To: freebsd-questions@freebsd.org Subject: odd phantom directory Hi all, I ran into a rather odd issue this morning with my FreeBSD 9.0-Release system running ZFS v28. This system serves as an RSYNC host which all of our other systems back up to each night. Last night, I started getting the following error: file has vanished: /backup/ldap1/etc/pki Now, usually when I get a file has vanished error during an RSYNC run, it indicates that the source file/directory on the system that is sending the rsync backup has been deleted or moved before rsync got a chance to actually send it. That doesn't appear to be the case here. /backup/ldap1/etc/pki is the destination directory on my Freebsd/ZFS server. I take a look in /backup/ldap1/etc on my Freebsd server and the pki subdirectory is no longer listed. Ok, so I run mkdir /backup/ldap1/etc/pki and get the following error: mkdir: /backup/ldap1/etc/pki: File exists. Odd Just to double check, I run ls -la /backup/ldap1/etc/pki and get the following: ls: /backup/ldap1/etc/pki: No such file or directory Alright, how about a simple touch? touch: /backup/ldap1/etc/pki: No such file or directory Fine. Maybe there is something funky about the /backup/ldap1/etc directory that is preventing me from doing any of this. mkdir /backup/ldap1/etc/pki2. That works just fine. What the heck? Looking at the output of my daily security run, I see the following: Checking setuid files and devices: find: /backup/ldap1/etc/fonts/conf.avail: No such file or directory find: /backup/ldap1/etc/fonts/conf.d/30-metric-aliases.conf: No such file or directory find: /backup/ldap1/etc/pki: No such file or directory So, it looks like there are a few files/directories in /backup/ldap1/etc that were affected. Looking through dmesg and /var/log/messages, I don't see anything out of the ordinary. I'm running a zpool scrub now just to be on the safe side, but I haven't seen any checksum or other errors so far. Any thoughts as to what might be causing this? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
RE: odd phantom directory
Ok, really confused now. I just ran an rm -rf /backup/ldap1, which errored out when trying to rm /backup/ldap1/etc/pki, /backup/ldap1/etc/fonts/conf.d/30-metric-aliases.conf, and /backup/ldap1/etc/fonts/conf.avail. Everything else got purged correctly, except for those phantom files. I then reran my rsync script, which DIDN'T error this time, shipped all the files over, and I can now read those phantom files/folders just fine. -Original Message- From: owner-freebsd-questi...@freebsd.org [mailto:owner-freebsd-questi...@freebsd.org] On Behalf Of Brian Gold Sent: Friday, November 16, 2012 11:23 AM To: freebsd-questions@freebsd.org Subject: RE: odd phantom directory It looks like this may be the same issue as reported here: http://lists.freebsd.org/pipermail/freebsd-current/2011-October/027902.html but that thread seems to have just died off about a year ago. Zfs scrub is still running, but not reported errors so far. I'm going to run a zdb -ccv backup once that is done. From looking over this other thread, I tried just a simple ls /backup/ldap1/etc and /backup/ldap1/etc/pki does show up if I do ls without any arguments. If I do an ls -l then it doesn't show up. -Original Message- From: owner-freebsd-questi...@freebsd.org [mailto:owner-freebsd-questi...@freebsd.org] On Behalf Of Brian Gold Sent: Friday, November 16, 2012 9:37 AM To: freebsd-questions@freebsd.org Subject: odd phantom directory Hi all, I ran into a rather odd issue this morning with my FreeBSD 9.0-Release system running ZFS v28. This system serves as an RSYNC host which all of our other systems back up to each night. Last night, I started getting the following error: file has vanished: /backup/ldap1/etc/pki Now, usually when I get a file has vanished error during an RSYNC run, it indicates that the source file/directory on the system that is sending the rsync backup has been deleted or moved before rsync got a chance to actually send it. That doesn't appear to be the case here. /backup/ldap1/etc/pki is the destination directory on my Freebsd/ZFS server. I take a look in /backup/ldap1/etc on my Freebsd server and the pki subdirectory is no longer listed. Ok, so I run mkdir /backup/ldap1/etc/pki and get the following error: mkdir: /backup/ldap1/etc/pki: File exists. Odd Just to double check, I run ls -la /backup/ldap1/etc/pki and get the following: ls: /backup/ldap1/etc/pki: No such file or directory Alright, how about a simple touch? touch: /backup/ldap1/etc/pki: No such file or directory Fine. Maybe there is something funky about the /backup/ldap1/etc directory that is preventing me from doing any of this. mkdir /backup/ldap1/etc/pki2. That works just fine. What the heck? Looking at the output of my daily security run, I see the following: Checking setuid files and devices: find: /backup/ldap1/etc/fonts/conf.avail: No such file or directory find: /backup/ldap1/etc/fonts/conf.d/30-metric-aliases.conf: No such file or directory find: /backup/ldap1/etc/pki: No such file or directory So, it looks like there are a few files/directories in /backup/ldap1/etc that were affected. Looking through dmesg and /var/log/messages, I don't see anything out of the ordinary. I'm running a zpool scrub now just to be on the safe side, but I haven't seen any checksum or other errors so far. Any thoughts as to what might be causing this? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: odd phantom directory
Brian Gold bgold at simons-rock.edu writes: Hi all, I ran into a rather odd issue this morning with my FreeBSD 9.0-Release system running ZFS v28. This system serves as an RSYNC host which all of our other systems back up to each night. Last night, I started getting the following error: file has vanished: /backup/ldap1/etc/pki Now, usually when I get a file has vanished error during an RSYNC run, it indicates that the source file/directory on the system that is sending the rsync backup has been deleted or moved before rsync got a chance to actually send it. That doesn't appear to be the case here. /backup/ldap1/etc/pki is the destination directory on my Freebsd/ZFS server. I take a look in /backup/ldap1/etc on my Freebsd server and the pki subdirectory is no longer listed. Ok, so I run mkdir /backup/ldap1/etc/pki and get the following error: mkdir: /backup/ldap1/etc/pki: File exists. Odd Just to double check, I run ls -la /backup/ldap1/etc/pki and get the following: ls: /backup/ldap1/etc/pki: No such file or directory ... There have been cases like that reported in the past. One was dated 2006: http://lists.freebsd.org/pipermail/freebsd-bugs/2006-April/018069.html I assume the backup host was on UFS. This comment seems to be interesting: Such behavior usually caused by lost vnode reference and/or bugs in the vnode traversal code. ... Next dated 2011: http://freebsd.1045724.n5.nabble.com/carsten-c-otto-de-ftpsync-freebsd-ftp-ftp-1013-rsync-ERROR-on-2011-03-04-09-23-00-td4073512.html I assume the backup host was on UFS2. There was a fix commited: ...John Baldwin commited very promising MFC yesterday, see http://svn.freebsd.org/changeset/base/219744 . Next dated 2011: http://lists.freebsd.org/pipermail/freebsd-current/2011-October/027902.html The backup host was on ZFS. Yours is similar to the last one. Perhaps looking for the solution to this problem should start at top VFS layer ? The description in /usr/src/sys/sys/vnode.h is a good reference. I would suggest you file a PR# to get VFS and fs devs have a look at it. jb ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
