In my experience, using the file systems time stamp on files is more reliable and robust than what can be done in user space.  

A alphabetic naming approach would be inherently slower than just using the time stamp because the time stamp is already automatically created.  In order to coordinate file names, we would have to do directory scans on each put() to coordinate the next filename to generate, or using some sort of static method in a routine (and that wouldn't survive across system reboots). 

If multiple files were being written at the same time by multiple processes, some sort of locking mechanism to determine what file name to generate would be required in order to make the naming reliable.   That isn't necessary with the file system time stamps, since it's automatic.

The put() routine's pretty simple - it's just four lines of code.   Hard to get much or faster or reliable than that.

It's my suggestion to use what we have written for DC1, and make a determination (either during DC1 or afterwards) how well the current routines work and make changes then if required.

Steve

On May 26, 2006, at 1:20 PM, Russell E Owen wrote:

I suggest that the DC1 queue code could be simplified and made more robust by insisting that files be ordered alphabetically, rather than measuring the creation date and picking the earliest file.


The reason this may be more robust (if the naming can be done reliably) is that file creation date can be either unreliable or too chunky for some kinds of file systems.


It is simpler because there is no need to query for file creation name. Instead, one simply sorts the list of found files -- an operation that is blazingly fast in python (it has a superb sort).


-- Russell


_______________________________________________
LSST-data mailing list
[email protected]
http://www.lsstmail.org/mailman/listinfo/lsst-data

Reply via email to