[PATCH] file change notification

2003-02-06 Thread Hal Roberts
Attached is a patch that adds support for reporting 
individual files during file change notification.  The patch 
keeps a table of the stats of all files in a watched 
directory and then compares the current stats of the files 
with the stored stats whenever there is a notification 
event.  Note that all of the code is in the generic notify.c 
file, so it works regardless of whether the server is using 
hash_notify.c or kernel_notify.c.

The costs are 1) the memory cost of storing the tdb info in 
memory and 2) the performance cost of having to check each 
file to figure out which one changed everytime there is a 
file notification event.  The memory use could be reduced by 
storing the table on disk instead of in memory, but that 
would incur a cost in performance and could result in a lot 
of *tdb files sitting around (one per directory watched). 
The only way to reduce the performance cost is to make the 
kernel stuff return a file name with its notification.  This 
change wouldn't be too hard to add to the kernel, but I'd 
rather not require a hacked kernel to make samba work as it 
should.

The only thing that doesn't work quite as it should is that 
the code does not report a file name change event as such, 
but instead just treats a name change as a file remove and 
then a file add.  Supporting this would make the code a lot 
more complex (I'd have to keep a separate, inode indexed 
table of file stats in addition to the file name indexed 
one), and I can't imagine when a remove/add instead of a 
move would break a client, though it might cause performance 
issues.

Also, though this is a separate issue, the file change stuff 
still doesn't support deep notifications as it should as far 
as I can tell (which is to say not at all).

-hal
diff -u -r samba-2.2.7a.dist/source/include/smb.h samba-2.2.7a/source/include/smb.h
--- samba-2.2.7a.dist/source/include/smb.h  Wed Dec  4 12:16:36 2002
+++ samba-2.2.7a/source/include/smb.h   Wed Jan 22 16:26:56 2003
@@ -1202,6 +1202,16 @@
 #define FILE_NOTIFY_CHANGE_SECURITY0x100
 #define FILE_NOTIFY_CHANGE_FILE_NAME   0x200
 
+/* file notification actions */
+#define FILE_ACTION_ADDED  0x0001
+#define FILE_ACTION_REMOVED0x0002
+#define FILE_ACTION_MODIFIED   0x0003
+#define FILE_ACTION_RENAMED_OLD_NAME   0x0004
+#define FILE_ACTION_RENAMED_NEW_NAME   0x0005
+#define FILE_ACTION_ADDED_STREAM   0x0006
+#define FILE_ACTION_REMOVED_STREAM 0x0007
+#define FILE_ACTION_MODIFIED_STREAM0x0008
+
 /* where to find the base of the SMB packet proper */
 #define smb_base(buf) (((char *)(buf))+4)
 
diff -u -r samba-2.2.7a.dist/source/smbd/notify.c samba-2.2.7a/source/smbd/notify.c
--- samba-2.2.7a.dist/source/smbd/notify.c  Fri Feb  1 17:14:47 2002
+++ samba-2.2.7a/source/smbd/notify.c   Thu Feb  6 16:08:07 2003
@@ -25,6 +25,16 @@
 static struct cnotify_fns *cnotify;
 
 /
+This structure holds a list of files and associated notification actions.
+*/
+struct file_action {
+   struct file_action *next, *prev;
+   int action;
+   char *filename;
+   int filename_length;
+};
+
+/
  This is the structure to queue to implement NT change
  notify. It consists of smb_size bytes stored from the
  transact command (to keep the mid, tid etc around).
@@ -38,14 +48,250 @@
uint32 flags;
char request_buf[smb_size];
void *change_data;
+   TDB_CONTEXT *file_data;
+   struct file_action *file_actions;
 };
 
 static struct change_notify *change_notify_list;
 
 /
+ Return a file action struct with the given filename and fileaction
+*/
+struct file_action *change_notify_get_file_action(char *filename,
+ int fileaction) {
+
+   struct file_action *fa;
+  
+   if (!(fa = (struct file_action *)malloc(sizeof(struct file_action {
+   DEBUG(0, (malloc failed!));
+   }
+
+   fa-action = fileaction;
+   fa-filename = strdup(filename);
+   fa-filename_length = strlen(filename);
+
+   return fa;
+}
+
+/
+ Check to make sure that the file in the given cnbp.file_data record
+ still exists.
+*/
+int change_notify_file_data_exists(TDB_CONTEXT *tdb, TDB_DATA key,
+  TDB_DATA data, void *status) {
+  
+   struct file_action *fa;
+   struct change_notify *cnbp = (struct change_notify *)status;
+   char *filename;
+   char 

file change notification issues

2003-01-23 Thread Hal Roberts
The purpose of this mail is to make sure no one else is 
working on / has already finished hacking samba's file 
change notification support to support notification of 
individual file changes.  If not, I plan on doing so forthwith.

More details:

I've been wrestling with samba for the past couple of weeks 
trying to get it to play nicely with IIS.  I've got IIS 
running with a samba share as the root directory, and 
everything works well except for asp caching.  When running 
from a samba share, IIS refuses to invalidate any cached 
asps, even if the cached asp is modified or even deleted or 
moved.

I've finally pegged the problem as samba file change 
notification support.  The cifs reference at:

http://www.snia.org/tech_activities/CIFS/

indicates that samba should send back information about 
which files triggered a notification and why for any 
directory with notification running (for instance, 
/test/test.asp and FILE_ACTION_MODIFIED).  Samba should send 
back a list of such file/action records with one record for 
each file action that triggered the notification, up to the 
maximum as determined by the parameter count field in the 
request.  If the number of records would be greater that the 
max allowed, the samba server should return a 
STATUS_NOTIFY_ENUM_DIR error.

The samba notification stuff as written does not keep track 
of which specific files were changed and just *always* 
returns the STATUS_NOTIFY_ENUM_DIR error.  This approach 
seems to work fine for handling windows explorer updates, 
but IIS and presumably other applications just drop these 
replies as errors.

I hacked the notify stuff to return a hard coded file / 
action record of a file I knew I was changing to test this 
theory, and the hack did indeed fix the problem -- when I 
changed the hard coded file, IIS immediately invalidated the 
asp cache for that file and loaded up the new file instead.

The big cost of adding individual file notifications will be 
the need to keep a bunch of file information in memory 
somewhere.  The current code just does a checksum of the 
file modification times of the monitored directory or relies 
on the dnotify kernel stuff, which only alerts on whole 
directories.  To figure which file changed, the system will 
have to keep the state of the monitored files in memory. 
There is a project called fam 
(http://oss.sgi.com/projects/fam/) that implements a daemon 
that handles this sort of thing for the general case, but it 
does not support all of the notifications that cifs requires 
(it does not support notifications of file access, for 
example), so the only way of implementing this correctly 
seems to be to write it from scratch.

Thoughts?

-hal