On Sat, Apr 28, 2012 at 12:36 PM, Alejandro Imass <aim...@yabarana.com> wrote: > On Sat, Apr 28, 2012 at 11:39 AM, Robert Bonomi > <bon...@mail.r-bonomi.com> wrote: >> >> Alejandro Imass <aim...@yabarana.com> wrote: >>> On Sat, Apr 28, 2012 at 3:22 AM, Wojciech Puchar >>> <woj...@wojtek.tensor.gdynia.pl> wrote: >>> >> I somewhat agree, but it wasn't a person. I am the only administrator, >>> >> the only one with root access. The jails were effectively moved to the >>> >> /usr/local/etc/apache22 of the single that survived at the top level. >>> >> I'm thinking something between mount, EzJail, the journal and the way >>> >> MySQL created a great deal of head contention, so something must have >>> >> gotten corrupted at the directory level like you state, but the >>> >> strange part is no _data_ corruption as such, because I was able to >>> >> physically archive the jails, move them to the correct directory and >>> > >>> > >>> > no matter what you do FreeBSD DOES NOT ramdomly move directories. if you >>> > are >>> > sure you didn't move it yourself then it must be machine hardware problem >>> > but still unlikely. >>> >>> After a little more research, ___it it NOT unlikely at all___ that >>> under high distress and a hard boot, UFS could have somehow corrupted >>> the directory structure, whilst maintaining the data intact. >> >> This is techically accurate, *BUT* the specifics of the quote "corruption" >> unquote in the case under discussion make it *EXTREMELY* unlikely that this >> is what happened. >> >> 99.99+++% of all UFS filesystem "corruption' issues are the result of a >> system crash _between_ the time cached 'meta-data' is updated in memory >> and that data is flushed to disk (a deferred write). >> >> The second most common (and vanishingly rare) failure mode is a powerfail >> _as_ a sector of disk is being written -- resulting in 'garbage data' >> being written to disk. >> >> The next possibility is 'cosmic rays'. If running on 'cheap' hardware (i.e., >> without 'ECC' memory), this can cause a *SINGLE-BIT* error in data being >> output. >> >> The fact that the 'corrupted' filesystem passed fsck -without- any reported >> errors shows that everything in the filesystem meta-data was consistent >> >> Given *that*, there are precisely *TWO* ways that the 'results' that have >> been reported could have happened. >> >> 1) "Something" did a mv(2) of the various jail directories 'from' their >> original location to the 'apache' diretory. This involves simply >> *copying* the diretory entry from the jail's 'parent directory' to >> the apache directory, and then marking the entry in the original >> parent as 'unused'. Nothing other than the directory whre the jail >> 'used to live', and the directory 'where it was found' are touched. >> This occured _through_ the system 'mv' function, so all the normal >> 'housekeeping' was done properly. >> >> 2) it was -not- done though mv(2) -- but that requires that a whole >> *series* of "corruptions" of the filesystem, _ALL_ of which had to >> occur in 'exactly' the right way. They are: > > [...] > >> I think it is safe to conclude that the probabilities -greatly- favor >> alternative #1. >> > > OK. So after your comments and further research I concur with you on > the mv but if it wasn't a human, then this might be exposing a serious > security flaw in the jail system or the way EzJail implements it. The > whole point of using jails is to protect things like this from > happening. Given that the only jail that survived was the front-end > Apache Web server/reverse proxy, then it is also safe to suspect the > apache (or other) process running on it was able to perform a mv of > the rest of the jails to it's own /usr/local/etc/apache22 directory. > > Is there no possibility is that after the system crash, the journal > recocery process and/or fsck could have moved this directories ? >
Also note that even the EzJail basejail was moved also, so it could be a security hole in the way nullfs is used or in nullfs itself. but the curious thing is that the basejail is supposed to be mounted read-only so how did that get moved to the http-proxy jail?? That is why I suspect it could have been something in the boot process like the journal recovery, fsck or something else with that kind of privilege and when the EzJail filesystems were unmounted. -- Alejandro _______________________________________________ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
