Re: Race condition..., once again
Adam Osuchowski wrote: fd = open(file); read(fd, buffer, size); close(fd); copy(file, file~); [...here user edits file...] fd = open(file.tmp, O_WRONLY|O_CREAT|O_TRUNC); write(fd, new content of file); close(fd); chmod(file.swp, mode); chown(file.swp, owner, group); [...if need be, setting other parameters like ACLs...] rename(file.swp, file); Sure, it's my mistake, it should be file.tmp not file.swp: fd = open(file); read(fd, buffer, size); close(fd); copy(file, file~); [...here user edits file...] fd = open(file.tmp, O_WRONLY|O_CREAT|O_TRUNC); write(fd, new content of file); close(fd); chmod(file.tmp, mode); chown(file.tmp, owner, group); [...if need be, setting other parameters like ACLs...] rename(file.tmp, file); -- You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php
Race condition..., once again
Some time ago I wrote here about race condition during file saving (http://groups.google.com/group/vim_dev/tree/browse_frm/month/2009-01/0049693b73a6a1e6). Some of you didn't share my opinion about this problem and said File systems is not databases with ACID properties. Ok, I agree and know it is rather POSIX semantic's fault. But my case has come back to me. In that thread, I suggested that rename() syscall should improve bahaviour beacuse of atomical nature of this syscall. One of argument against using rename() was that it would break hard and soft links. In soft links case, I can't imagine how rename() could break it. If process atomically exchange file which is pointed to by symlink, nothing will break down. Even, if it is chain of symlinks. In hard links case, really there is a problem, and this case should be treated individually (fortunately, hardlinks are rather relatively rare case). But in other cases, rename() should be safe. Furthermore, even now, vim changes inode of edited file: # ls -li testfile 2955134269 -rw--- 1 root root 15 Jan 25 22:44 testfile # vim testfile [...edit file here...] # ls -li testfile 2955134271 -rw--- 1 root root 20 Jan 25 22:44 testfile rename() would behave identically and moreover would be atomic. So, I have a little request to developers. Please, read it again and try to understand me. Atomic rename() is better solution. It's not so stupid as it may appear at first look and shouldn't break to much. :-) Regards. -- You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php
One more thought about vim race condition during file saving
Did you think about use leases to exclusively lock file during save? I mean fcntl() with F_SETLEASE. It acquires mandatory lock and doesn't break hard links, which you afraid of. Processes which try to read file during saving are blocked on open() until lease is removed, so it doesn't change behaviour from other processes' point of view. On the other hand, it is not possible to obtain write lease on file already opened by another process, and in such a situation (which IMHO shouldn't happen often with files which are edited by hand) vim should retry acquiring lease after short sleep or fallback to current behaviour. This method was originally developed for use in samba, but it is available in Linux kernel since 2002 (or even earlier), so it could be said that it is standard. Certainly, it is Linux specific feature, but if it is available, why shouldn't it be used. Any pros or cons? Regards. --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: Race condition during file saving
Matt Wozniski wrote: rename(2) doesn't do everything needed. Right, but current behaviour is even worse. We can't protect if somebody create file while vim saves it due to system limitations, but we can protect against completely lack of file or situation when it is partially written. rename() would break the link, which probably isn't what you want an editor to do... Right, but definitely there are fewer multi hard linked files than singles. Again, we can't protect against such situation because of POSIX syscalls nature, so maybe vim should identify if there is hard link and unless, it will do atomic file replacement. I know, there is another race condition (between stat() and rename()) but it is more unlikely case. --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: Race condition during file saving
Bram Moolenaar wrote: That's all taken care of when 'backupcopy' is auto. If you want the original file to always exist set 'backupcopy' to yes. Saving files will be slower then, since Vim needs to both write a copy and write the actual file. Not quite. Of course, with 'backupcopy' set to yes, there are not moment when another process find file missing, but still file may be empty or not completely written: open(testfile, O_WRONLY|O_CREAT|O_TRUNC, 0600) = 3 write(3, test test test\n..., 15) = 15 fsync(3) = 0 close(3) = 0 Since you are overwriting the file there always is a moment it's empty. Unless use of rename() syscall, which replace it atomically (with exact to hard link cases). I know, that these problems result from flawed POSIX file system syscalls behaviour, but IMHO it may be made better than it is done now. --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: Race condition during file saving
Nikolai Weibull wrote: Either way, I really don't think we have a problem to fix. So what is your advice? Ignore it? It's very comfortable to call flows features, we have perfect situation and don't need to worry about consequences. If you're writing to a file that another program critically needs /that's/ your problem. Configuration file, for example, is critically for almost every daemon. Do you think that it is only my problem? No, it is very real scenario which could happen to everyone who use vim. --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: Race condition during file saving
Charles E. Campbell, Jr. wrote: Likely areas for problems like this concern cooperative editing (ie. multiple people editing the same file) and editing log files (or other files which are potentially being written to by some other program). Vim isn't designed for cooperative editing; I seem to recall it being on a wishlist, though. Editing log files is problematic because they generally aren't using mandatory file locking. I didn't tell about multiple people editing the same file or editing log files. I told about _ONE_ person editing file which could be read by another process at the same time. Isn't vim designed for it too? Don't think so. To avoid the need for cooperative editing, use cvs/git/etc and use separate copies and repositories. And keep all /etc files in cvs repo? It's only pinning the blame on another application (cvs, in this case). --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Race condition during file saving
There is a race condition in vim 7.2 (and probably in earlier too) on POSIX
platforms. Below, there is fragment of strace output related to this problem.
stat64(testfile~, 0xbfc35dbc) = -1 ENOENT (No such file or directory)
stat64(testfile, {st_mode=S_IFREG|0600, st_size=12, ...}) = 0
unlink(testfile~) = -1 ENOENT (No such file or directory)
rename(testfile, testfile~) = 0
open(testfile, O_WRONLY|O_CREAT|O_TRUNC, 0600) = 3
write(3, test test test\n..., 15) = 15
fsync(3)= 0
stat64(testfile, {st_mode=S_IFREG|0600, st_size=15, ...}) = 0
stat64(testfile, {st_mode=S_IFREG|0600, st_size=15, ...}) = 0
close(3)= 0
Problematic situation take place during file saving between rename and open,
open and write as well as write and close syscalls. There are points at
which another process attempting to access file may run into trouble.
There is, for example, possibility to:
- if application, which expects existence of file, would try to open it
between vim rename and open syscalls, it will fail due to lack of this
file,
- if other process creates file with the same name between vim rename and
open syscalls, it will be overriden by vim (it works with symlinks too,
so it can be used by attacker to damage other files),
- if application will read file while vim will write to it, the contents
may be badly read due to temporarily partially record.
Vim rather should create new, its own, temporary file with unique name,
write content, close it and then, atomically rename it to original name.
Regards.
--~--~-~--~~~---~--~~
You received this message from the vim_dev maillist.
For more information, visit http://www.vim.org/maillist.php
-~--~~~~--~~--~--~---
Re: Race condition during file saving
Tony Mechelynck wrote: See :help backup :help 'backup' :help 'writebackup' :help 'backupcopy' :help timestamp I try different settings of this variables and there was always the same situation: open(testfile, O_WRONLY|O_CREAT|O_TRUNC, 0600) = 3 write(3, test test test\n..., 15) = 15 So, still there are points in time when file could be empty (after opening with O_TRUNC) or partially written (between multiple write syscalls). Could you give me a concrete example of values of these settings, which could prevent such situations? --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: Race condition during file saving
Tony Mechelynck wrote: I don't know. There're only one keyboard and one display on this machine, and I try to avoid having several programs modify a single file simultaneously outside each other's knowledge. The rare case is /var/spool/mail/root which is appended to by my cron jobs, and truncated by SeaMonkey after downloading the mail to its own mailbox in its profile. Vim doesn't intervene there. Vim will try to detect when its editfile has been modified by another program, but it is not meant to be used in an environment where anything can be modified simultaneously by any number of actors. If something goes wrong, you can try to |recover|. Cases like mbox files are not so rare. There are many examples of simultaneously access to single file, but problem exists even without concurrent modification. A simple example: editing config file for some daemon. When vim truncates this file and program read it at the same time (because, for example, it will be restarted by cron, other administrator or even by itself) it will be problematic situation. The same matter is if a program will be run from cron or other program (for example, procmail from sendmail). Recover option in vim does not help here. Do you suggest turn off all processes during editing config files? Vim is used on multiuser and multiprocess systems, so limiting the working users or simultaneously processes to single one is misunderstanding. Why do you disrespect problem, especially if there is solution in the form of proper use of rename(2) syscall, which I mentioned about? --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
