Re: Log every access to a file
On Mon, 2003-10-27 at 03:35, Matthew Seaman wrote: On Mon, Oct 27, 2003 at 11:57:31AM +0800, [EMAIL PROTECTED] wrote: How do you create/add a system log to monitor every access to a specific file (say a database file accessed through samba)? A sample line for syslog.conf would be greatly appreciated ?? :-) Samba has extensive logging capabilities itself -- which generally bypass syslog entirely, although there are options available to use syslog. It will certainly log who is accessing the server and from what machines. I don't think it has the capability to monitor accesses down to the level of a particular file though, but read the manuals carefully to be sure. If you really need to log all accesses to the file, then probably your best bet is to only make the file available via a web interface, which can be set to require passwords before it will allow access and will supply the logs you require. Alternatively, databases such as postgres or mysql can keep detailed logs of all queries run against them. Actually, there are two options that will allow you to monitor accesses of any particular file. The first is to periodically stat that file. This is incredibly simple to do. The disadvantage is that if the file is being accessed very often, you may miss some accesses (if you're checking every second, and two people access in the same second, you'll only see one access); if the file is being accessed very rarely, it's a bit of a waste of cpu and disk time to keep checking it. But, nonetheless, this is sometimes the best way to go. I've attached a script statlog.py (requires python 2.3) that will do it for you. It read a list of filenames (one per line) from /usr/local/etc/statlog.conf, and begins monitoring each one, and outputs to /var/log/statlog.log any time there's been a change to A/M/C time. By default, it checks once/second, but you can change this with the -f flag (./statlog -f 5 means five times/second, ./statlog -f 0.5 means every two seconds, ./statlog -f 0 means as often as you can--which you probably only want to use in conjunction with nice or idprio_. The second is to use fam. I should mention that I've only used fam under linux, and, after a brief glance, it looks like the FreeBSD port (/usr/ports/devel/fam) is not as powerful--in particular, FreeBSD apparently doesn't provide imon support (a way for the filesystem to make a callback to a usermode app like fam--no dnotify or anything similar, either, apparently). Which implies that it's probably just a heavier-weight way of doing the exact same thing--periodically stat'ing a list of files--and that there is no better solution available. But I could be wrong, and it's probably worth testing to see if it works better for you. Also, if the files are stored on nfs-mounted drives (and this may be true for smb also, but I don't know), and the nfs server is running fam, the checks are passed off to the server, which makes them faster (and, if the server is running linux or another imon-capable OS, gets around the worries mentioned above). You should have no problem getting fam itself working if you follow the instructions in the message you get when installing the port/package. Anyway, the second script, famlog.py, is a slightly-modified version of a script that I've used for a similar purpose in linux. It reads the filenames in /usr/local/etc/famlog.conf, tells fam to monitor all of those files, and sends its output to /var/log/famlog.log. If either of these is useful to me, let me know. If you need help automating stuff (making an rc.d/famlog.sh wrapper, and maybe a logrotater), modifying either script to use syslog instead of its own log file (should be a one-line change), etc., just ask. (NOTE: The attachments are scrubbed from the copy of this message sent to the list; if anyone besides the original author wants them, let me know.) ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Log every access to a file
On Wed, Oct 29, 2003 at 01:07:26AM -0800, andi payn wrote: The second is to use fam. I should mention that I've only used fam under linux, and, after a brief glance, it looks like the FreeBSD port (/usr/ports/devel/fam) is not as powerful--in particular, FreeBSD apparently doesn't provide imon support (a way for the filesystem to make a callback to a usermode app like fam--no dnotify or anything similar, either, apparently). Which implies that it's probably just a heavier-weight way of doing the exact same thing--periodically stat'ing a list of files--and that there is no better solution available. Check the kevent(2) man page. It's a generic mechanism for having the kernel message your process when some condition occurs, such as modification of a file. Unfortunately other than knowing something happened, it doesn't tell you a great deal else, like who it was that made the alteration. Even so, fam(1) has apparently not been patched to use kevent(2) under FreeBSD, so, yes, it's probably going to operate by polling the file status every so often. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 26 The Paddocks Savill Way PGP: http://www.infracaninophile.co.uk/pgpkey Marlow Tel: +44 1628 476614 Bucks., SL7 1TH UK pgp0.pgp Description: PGP signature
Re: Log every access to a file
On Wed, 29 Oct 2003 10:00:15 + Matthew Seaman [EMAIL PROTECTED] wrote: On Wed, Oct 29, 2003 at 01:07:26AM -0800, andi payn wrote: The second is to use fam. I should mention that I've only used fam under linux, and, after a brief glance, it looks like the FreeBSD port (/usr/ports/devel/fam) is not as powerful--in particular, FreeBSD apparently doesn't provide imon support (a way for the filesystem to make a callback to a usermode app like fam--no dnotify or anything similar, either, apparently). Which implies that it's probably just a heavier-weight way of doing the exact same thing--periodically stat'ing a list of files--and that there is no better solution available. Check the kevent(2) man page. It's a generic mechanism for having the kernel message your process when some condition occurs, such as modification of a file. Unfortunately other than knowing something happened, it doesn't tell you a great deal else, like who it was that made the alteration. And for a way to easily use this facility from shell scripts, check out sysutils/wait_on, in the ports tree. -Chris ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Log every access to a file
On Wed, 2003-10-29 at 02:00, Matthew Seaman wrote: On Wed, Oct 29, 2003 at 01:07:26AM -0800, andi payn wrote: The second is to use fam. I should mention that I've only used fam under linux, and, after a brief glance, it looks like the FreeBSD port (/usr/ports/devel/fam) is not as powerful--in particular, FreeBSD apparently doesn't provide imon support (a way for the filesystem to make a callback to a usermode app like fam--no dnotify or anything similar, either, apparently). Which implies that it's probably just a heavier-weight way of doing the exact same thing--periodically stat'ing a list of files--and that there is no better solution available. Check the kevent(2) man page. It's a generic mechanism for having the kernel message your process when some condition occurs, such as modification of a file. Thanks. This is pretty cool, but it's missing a few things. First, it works only on UFS filesystems. That's no big deal; fam could use kevent on UFS and poll other filesystems. However, that does seem to rule it out for the original poster's purpose (the file he wanted to monitor was mounted via smb). More importantly, kevent apparently doesn't notify you when a file is accessed (or does changing the atime trigger NOTE_ATTRIB? in that case, it does--but to distinguish between an access and a change you'd have to stat the file). This seems to make it unusable for fam, as well as for the original poster's purpose (he wanted to see all accesses to a file). It also doesn't seem to notify you when a file in a directory you're watching is changed. This might would make fam much more complicated. However, I think the same is true of dnotify, so the extra code can probably be borrowed from the dnotify patch Unfortunately other than knowing something happened, it doesn't tell you a great deal else, like who it was that made the alteration. Well, fam doesn't tell you anything more either; neither does imon nor dnotify--or stating a file directly. The OS and filesystem don't keep track of things like who was the last user or process to touch a file. If you want that, you could take fstat snapshots and hope to catch everyone who has the file open (this should work if users tend to access it for a long period of time, say a second or more). Or you could get a lease on a file and see who breaks it (although many programs might try to write the file without breaking your lock first, and just fail and give up--and besides, I don't know if that would work in FreeBSD the way it would in linux). Or you could put a fifo/socket/loopback filesystem/something between the actual file and the pathname people use to access it, and thereby control/monitor all access to it; etc. Even so, fam(1) has apparently not been patched to use kevent(2) under FreeBSD, so, yes, it's probably going to operate by polling the file status every so often. Well, it looks like it would be a pretty easy patch to fam (especially since someone's already generalized the imon code to allow using dnotify instead), but if it can't provide information on accesses, it'd also be a useless patch (which might explain why nobody's done it yet?). Another issue: In linux, you can open a file O_NOACCESS (== O_ACCMODE, if you don't #include anything extra), which gets you an fd to pass to fstat/fcntl/whatever even without read or write access to the file. I assume the dnotify patch to fam uses this. This doesn't seem to work in FreeBSD. If you try to open a file O_ACCMODE, you get EACCES. Therefore, you'd have to open the file O_RDONLY to get an fd for kevent--which means you can't monitor a file that you can't read, which would be another limit placed on fam by using kevent. Plus, kevent doesn't seem to give you information on what's been changed within a directory, so fam would have to do some dirty work on each access (or monitor all files within the directory...). I think the same is true with dnotify; if so the kevent patch could use the same code as the dnotify patch, which is probably fine. Still, it's a good idea; I'll look into this a little deeper. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Log every access to a file
How do you create/add a system log to monitor every access to a specific file (say a database file accessed through samba)? A sample line for syslog.conf would be greatly appreciated ?? :-) Serve this file from an NFS-mounted partition and have nfsd log all file accesses. Is there a better way? Perhaps some kind of debugging option in the VFS or UFS (1 or 2) code? Or a modified union-fs or nullfs layer, which would intercept all filesystem calls and log them? Hmmm... -- Cordula's Web. http://www.cordula.ws/ ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Log every access to a file
Hello, How do you create/add a system log to monitor every access to a specific file (say a database file accessed through samba)? A sample line for syslog.conf would be greatly appreciated ?? :-) Thanks in advance. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
