Matt Benjamin wrote:
File Server/Cache Manager Extended Status InformationMotivation The FetchStatus/BulkStatus and Callback mechanisms provided for by the current AFS-3 protocol are not sufficiently rich to support server-coordinated byte-range locking [1]. Our analysis is that implementation of, in particular, granular and asynchronous file change notifications in the AFS protocol would provide an important general improvement independent of new features. Specifically, the current file status notification mechanism requires clients to discard an entire file when a small portion of the file may have changed. Changes to file access permissions are added, because they were enumerated by Jeff Altman as of interest. Some details in this area are not finalized, and feedback/discussion is requested. Locking specific change notifications are omitted (just the NotificationType flag is included) to simplify discussion. We expect to send a separate proposal on locking interfaces.
You misunderstood our discussion. ACL changes are not an area of interest. They are one of the reasons that BreakCallback() is executed in the file server. The full list includes: * StoreACL * StoreData * StoreStatus * RemoveFile * CreateFile * RenameFile * Symlink * Link * MakeDir * RemoveDir * ReleaseLock * when the callback is dropped by the file server due to lack of space
Proposed Change This proposal would supplement the current CallBack interface and protocol contract with a granular chunk or range invalidate mechanism, reducing network traffic and improving cache coherence. The specifics in this protocol draft are submitted for discussion. We feel that the following features of the proposal are key: 1. granular notifications from the file server to cache managers, specifically, invalidation of byte ranges in a file for which it owns a CallBack 2. backward compatiblity with existing clients and servers (this is provided for using the *_GetCapabilties RPC mechansims) 3. asynchronous notification from the file server to cache managers, where cache manager and file server agree to support it. Protocol Negotiation Prior to issuing any FetchStatus RPCs, clients using this feature will issue a GetCapabilities() RPC on the file server of interest. A new capability flag, cacheable by the client, will indicate the file server is capable of delivering extended status notifications.
I'm not sure that this is required.
Should the client wish to receive extended status notifications, it will so notify the file server by issuing an extended version of the FS FetchStatus RPC, containing a filter mask parameter specifying extended status notifications the client is willing to receive (cf. Data Types).
Why should the client have to request an extended callback notification? The client already sends the file server a capabilities list as part of the TellMeAboutYourself response. If the client supports the extended callbacks it can notify the file server in the TellMeAboutYourself response and the file server can simply send the extended callbacks.
I do not understand this level of complexity. A callback is issued because aHypothetically, this call might look like: FetchStatusEx( ~ IN AFSFid *Fid, ~ IN afs_int32 NotificationTypes, ~ OUT AFSFetchStatus *OutStatus, ~ AFSCallBack *CallBack, ~ AFSVolSync *Sync ) = 132; Data Types The following data types support the extended FetchStatus RPC in the file server interface, and a set of extended status notification callback RPCs in the cache manager interface. They are presented in rxgen format. *** /* callback notification classes, used by a client to specify a set ~ * of notifications it is interested in, and by the file server to ~ * specify notification type it knows how to provide */ %#define AFSCB_IND_FileInvalidate 0x01 /* traditional callback */%#define AFSCB_IND_RangeInvalidate 0x02 /* range in file has changed */ %#define AFSCB_IND_AccessChange 0x04 /* ACL or mode has changed *//* lock management notification interfaces -- notification structures ~ * and suggested prototypes for these calls are not included here to ~ * reduce scope of discussion, at GK request */%#define AFSCB_IND_LockAsyncResult 0x08 /* lock request completions */%#define AFSCB_IND_LockDelegation 0x16 /* lock delegation offers* and revocations are requested *//* new file-change notification types */ %#define AFSCB_FChange 0x01 /* range is invalidated */ %#define AFSCB_FTrunc 0x02 /* file reset to [0..Length] */ struct AFSCBRange { ~ afs_int64 Offset; ~ afs_int64 Length; }; struct AFSRangeInvalidateCallback { ~ afs_uint32 NotificationType; ~ afs_uint32 NewDataVersion; ~ AFSCBRange Range; }; typedef AFSRangeInvalidateCallBack AFSRangeCallBackSeq<AFSCBMAX>; ***
change has occurred on the file server. The problem that the cache managers have with the existing callback mechanism is that the cache manager has no idea what the trigger was. From my perspective, the cache manager should simply be told the reason for the trigger. For example: * StoreData (newDV, offset, length) * StoreACL (newDV) * StoreStatus (newDV, status data) * CreateFile (newDV, filename, fid) * RemoveFile (newDV, filename, fid) * RenameFile (newDV, oldname, newname, fid) * Symlink (newDV, name, fid) * Link (newDV, name, fid) * MakeDir (newDV, name, fid) * RemoveDir (newDV, name, fid) * ReleaseLock (newDV, type, fid) * Cancelled By doing so the cache managers obtain the data they need without requiringany architectural changes to the file server. These changes could be deployed
today.
Remote Procedure Calls
The following RPCs are added to the cache manager call back
interface. They are issued only when
1. a valid CallBack promise exists on a file for the called-back
client
2. the client has registered an interest in receiving extended
notifications
Proc RangeInvalidateCallBack
This RPC communicates a vector of file changes, of byte-range
granularity, for a corresponding vector of CallBacks.
proc RangeInvalidateCallBack {
~ IN AFSCBFids *Fids_Array,
~ AFSRangeCallbackSeq *CallBacks_Array
} multi = 65539;
Why is there a range of byte range changes? A StoreData operation does not take a list of independent commits. Each StoreData willresult in a BreakCallBack().
Proc AccessChangeCallBack
This RPC communicates a vector of file/directory access changes.
These changes may be to ACLs or Unix mode bits on the object.
NB., we are aware that currently, the OpenAFS cache manager tracks
user access right on files, not ACLs. This formated is used to
stimulate discussion--the cache manager and file server may not
currently share sufficient information (by design) to permit trivially
sending changes to the access structures cached by clients.
proc AccessChangeCallBack {
~ IN AFSCBFids *Fids_Array,
~ AFSAccessCallbackSeq *CallBacks_Array
} multi = 65540;
The Cache Manager does not track ACLs because it does not have enforcement responsibility and has no interaction with the Protection Service. The purpose of the StoreACL BreakCallBack is to instruct the cache manager to flush the access rights history for the file. Jeffrey Altman
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ AFS3-standardization mailing list [email protected] http://michigan-openafs-lists.central.org/mailman/listinfo/afs3-standardization
