RE : Detecting directory changes.
Without knowing what is using the internal routine, I do not know if it will remain, will be removed, or if changes to RMS will leave it totally or partially non-functional. Actually, this internal routine uses a feature of the XQP, not of RMS, but it's a detail. It may work for now, but as it is an internal routine, it is unsupported, and may break with out notice, even from the installation of an ECO kit. Well, this is hardly what happened in the past, however. As Dave Jones pointed out, this internal routine is dated 1993 in the sources, and uses an XQP feature (using the VMS lock manager in a certain way) that seems to be present since the very beginning of the XQP itself (middle of the 80s, if I remember correctly), precisely for managing the validation of the directory XQP cache in a cluster environment. In my opinion, the probability of a sudden break out for this particular feature sounds low. Note that my old method of looking at the lock value block of the XQP volume lock used another XQP feature that is hardly more documented or supported... Caching directory and file (stat) information enhances Samba/VMS performances very much, that's a fact. May be that the price to pay is to take the risk of using some undocumented/unsupported VMS feature. PLEASE READ THIS IMPORTANT ETIQUETTE MESSAGE BEFORE POSTING: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Detecting directory changes.
If another way of doing it is desired, perhaps the following could be considered. The general description of the method is to use the lock manager in a way that is compatible with how RMS uses it. What you'd do is to take out a lock in PR (protected read) mode on the RMS resource that represents the directory file. These resources have names that are built using a known pattern (4 character prefix of RMS$, 6 byte FID, 16 character devlockname). So after you read the directory data into your cache you enqueue a PR mode lock to the resource and close the file, at which time this lock should be granted (possible refinement - the lock could initially be requested in NL (null) mode with LCK$M_EXPEDITE specified, then a conversion to PR mode enqueued; this may have some advantages). You specify a blocking AST with the lock. The PR mode lock will not block any read-only access to the file, as they should PR (or maybe CR) mode locks. Any open allowing writes to the file should trigger your blocking AST as these will involve a PW (protected write) or EX (exclusive) mode lock which will be blocked by this lock. In the blocking AST you enqueue a conversion of lock to NL mode and set a flag for the cache indicating that it is invald. The next time the cache is needed the program checks the validity flag and finds that it is not valid so the cache is refreshed from disk and the lock converted back to PR mode. (Note that the resources used by the RMS locks all have a minimum of executive mode access and are all system locks. This makes enqueueing them just a bit more complicated.) The advantage is that you wouldn't need to poll the sequence number via F11X$POSIX_FASTRDSEQNOS every time you want to read from the cache, possibly giving some slight (possibly trivial) increase in performance. (Instead of polling for the value, you get notified via the blocking AST.) The disadvantage is, I think, that you had best be very carefull when messing around with RMS or you could have an adverse effect on the system as a whole. It is also possible that you will be invalidating the cache more often than is really necessary since other accesses to the directory file for reading it may not always be done in a read-only type of way (defaults for opening files in C, for example, is no sharing which presumably causes an EX mode lock to be used which would trigger the blocking AST and invalidate the cache). You should note that the Frontport library already does this sort of thing with RMS locks, thus Samba V2.0.6 is also already doing this sort of thing (and it works OK). Frontport's useage of this is, as I recall, somewhat simpler - I think it only enqueues locks to RMS resources to find out who is blocking you, and those test locks are immediately dequeued. This is part of its increase to fcntl() functionality regarding locks. It's a possibility to consider, anyway. --- Carl PLEASE READ THIS IMPORTANT ETIQUETTE MESSAGE BEFORE POSTING: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Detecting directory changes
How does Pathworks do this? Since HP/Compaq will probably maintain Pathworks as a layered product, the Pathworks method will continue to be supported internally (unless there is a major change in Pathworks itself). Mike Ober. PLEASE READ THIS IMPORTANT ETIQUETTE MESSAGE BEFORE POSTING: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Detecting directory changes.
That's actually a way to consider, but I am not too much confident, because the problem is to know when directory contents are changed. You are talking here of RMS locks, and changes in the directories are not made by RMS, but by the XQP. I mean that there is no calls to RMS $OPEN service when changing the directory contents, so there is probably no RMS$xxx resource either to be AST blocked for that purpose. I keep forgeting that even though the $RENAME service is part of RMS, it won't take out RMS locks (at least, I'm pretty sure it doesn't). OK, I have done some research. It turns out that the XQP uses serialization locks with names with a prefix of F11B$s followed by 4 bytes of binary data that are 2 bytes of FID file number and 2 bytes of RVN/NMX data (the only example I have found doesn't use the NMX data, but it is from 1994), essentially it is the FID with the middle two bytes of file sequence number removed. This is a child lock on a volume allocation lock - a lock taken out on a resource with a prefix of F11B$v followed by 12 bytes of volume name (space filled). So instead of blocking ASTs via a (probably never used) RMS resource, you could do it on the F11B$s serialization resource. According to the only mention of this on the Ask the Wizard pages, this is used by various software, including 3rd party softwre, for doing this sort of thing. I found an old (1994) program using this (it doesn't use the NMX byte to get the full range of file numbers, instead using only the RVN byte, so it will not always work as advertised on a more modern system - it will tend to operate on the wrong file if the desired file uses the NMX bits to extend its file number past the 16 bit limit). This program was for doing something a lot like SAMBA need - it was intended to replace the scanning of the contents of a directory to see if anything had changed with a way of being notified when it changed. See: http://www.eight-cubed.com/watchdir.zip which has a parent document of: http://www.eight-cubed.com/downloads.html The lock's resoure name's format is also mentioned in a few other places such as http:// www.geocities.com/keithparris/decus_presentations/ s2002_dist_lock_mgr_perf.ppt which includes this text in an example: 'F11B$vAPP2' 202020202020202032505041762442313146 Files-11 Volume Allocation lock for volume APP2 'F11B$sH...' 0148732442313146 Files-11 File Serialization lock for file [328,*,0] on volume APP2 The a lock on the former resouce would be the parent of the lock on the latter resource. (The parent lock evidently just needs to be in NL mode.) Figuring out some of this would be easier for someone with the VMS File Systems Internals book. Unfortunately that has not been updated since 1990 and would therefore be missing some information (like the NMX bits in the FID, probably). --- Carl PLEASE READ THIS IMPORTANT ETIQUETTE MESSAGE BEFORE POSTING: http://www.catb.org/~esr/faqs/smart-questions.html
RE : Detecting directory changes.
Gents, From your discussion on this topic, it sounds like there are 2 things that can be most useful at this point: 1) Use the unsupported F11X$POSIX_FASTRDSEQNOS API in the coding of Samba on VMS for now. 2) Someone with an OpenVMS support contract put in a request for a supported API to provide the desired alert on directory changes. John has indicated that this is a prime time for such a suggestion. --Bryan From: SMTP%[EMAIL PROTECTED] COLLOT Jean-Yves 13-MAR-2003 12:26:56.76 To: [EMAIL PROTECTED] CC: [EMAIL PROTECTED] Subj: RE : Detecting directory changes. So instead of blocking ASTs via a (probably never used) RMS resource, you could do it on the F11B$s serialization resource That's exactly that resource that F11X$POSIX_FASTRDSEQNOS uses. The question is : is it really useful to re-write in C a routine that is already available in the system, even if it's an undocumented one ? PLEASE READ THIS IMPORTANT ETIQUETTE MESSAGE BEFORE POSTING: http://www.catb.org/~esr/faqs/smart-questions.html
RE : PB with filename
We have many filenames with double underscores ( __ ) under VMS and they do not appear under Windows explorer. I have a look on Internet it seem to be normal under Samba Vms !!! The best idea I have is to use ODS-5 file system on your disk. Another way is to define the logical name : $ DEFINE/SYSTEM SAMBA_FILESPEC_ENCODE NEVER If you do that, your files will appear again, but the non-compatible-with-VMS filenames (such as names with blanks inside, or with the ~ character, for example) will not be permitted via Samba/VMS. May be you can accept that. PLEASE READ THIS IMPORTANT ETIQUETTE MESSAGE BEFORE POSTING: http://www.catb.org/~esr/faqs/smart-questions.html
RE : Detecting directory changes.
So instead of blocking ASTs via a (probably never used) RMS resource, you could do it on the F11B$s serialization resource That's exactly that resource that F11X$POSIX_FASTRDSEQNOS uses. The question is : is it really useful to re-write in C a routine that is already available in the system, even if it's an undocumented one ? Since this could work on a VAX, but the routine doesn't exist there, then it might be useful if you want it to work equally well on a VAX. It also switches polling the lock value block using that routine every time you want to read from the cache to something that is not polling that resource. It is going from: Doing whatever F11X$POSIX_FASTRDSEQNOS does. This is one of two things - it either enqueues a lock and dequeues the lock in a lock mode such that it gets a copy of the current lock value block, or it directly accesses the memory structures used by the lock manager to locate and read the lock value block. Since it is called FASTRDSEQNOS and direct access would probably be faster, I assume that this is what it does (probably in kernel mode). to: Checking a local variable. This variable is set by the blocking AST to indicate that the cache is invalid. This uses blocking ASTs and pays no attention to the sequence number in the lock value block. I tend to prefer notification via AST over polling. It is more efficient and should be faster: walking the lock manager's data structures (in kernel mode, I expect) to get the data in the lock value block and comparing it to the saved value vs. checking a local variable to see if it has been changed from TRUE to FALSE (or whatever) by an AST. You do still have to worry about version compatability. I would guess that this should at least work on any version of VMS that uses the XQP, but maybe not. A concern is how the other, system controlled, locks behave. If they have possible modes that indicate read or write access, then this should work well (e.g. they could use PR mode locks when reading and PW or EX mode locks when writing). If they don't (e.g. they just get set to EX mode even for a read access), then it won't work so well - it would probably require switching to using the lock value block. If that is the case, then it is not so usefull except maybe just using it to get the contents of the lock value block on a VAX (assuming that it sets this, but just not provide the F11X$POSIX_FASTRDSEQNOS routine to read it), which just requries taking out the lock and converting it back and forth between modes to get the data stored in the lock value block every time you need it. If the VAX flavor of VMS doesn't set this value block, then it wouldn't be useful on the VAX for that and would probably be slower than F11X$POSIX_FASTRDSEQNOS on the Alpha (and thus not useful there either). So the usefullness depends on the behavior of VMS's use of these locks and also whether or not the VAX version uses the lock value block to store sequence number information. --- Carl PLEASE READ THIS IMPORTANT ETIQUETTE MESSAGE BEFORE POSTING: http://www.catb.org/~esr/faqs/smart-questions.html
