Re: `notmuch setup` replaces `~/.notmuch-config` instead of truncating it
On Tue, Nov 16, 2010 at 22:42, Daniel Kahn Gillmor wrote: > On 11/16/2010 03:37 PM, Daniel Kahn Gillmor wrote: >> On 11/16/2010 03:26 PM, Ciprian Dorin, Craciun wrote: >>> So in the light of the above quoted "glitches", my question is: >>> due to the small chance of a power loss happening right when we write >>> such a small file, doesn't the inconvenience weight more than the >>> (fairly remote probable) file loss? >> >> What inconvenience? The inconvenience of writing the code correctly? > > Ah sorry -- on re-reading, i see you're probably referring to the > inconvenience of breaking hardlinks, dropping permissions, ACLs and > other metadata. Yes, exactly this is what I'm referring to: the meta-data that is not copied when a new file is created. > That's an open question, as far as i'm concerned. it'd be nice if there > was a way to "clone" a file's permissions and metadata to get the best > of both worlds. maybe someone knows some tricks to do that? > > --dkg ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: `notmuch setup` replaces `~/.notmuch-config` instead of truncating it
On Tue, Nov 16, 2010 at 22:37, Daniel Kahn Gillmor wrote: > On 11/16/2010 03:26 PM, Ciprian Dorin, Craciun wrote: >> P.S.: I say "pseudo" atomic because only the rename is atomic, >> thus in order to override file `a` for the target file `b` which >> exists, we must execute two **non-atomic** operations as a whole, but >> each atomic in part, rename operations: make `b` -> `c`, and then >> rename `a` -> `b`. So there is actually a small time-frame when I can >> be left with two files (`a` and `c`), none of which is my config file >> `b`. (This can be solved when opening the config file by checking if >> there isn't any leftover `c` or `a` file, in which case I take the `a` >> file and complete the rename.) > > There is only one ".notmuch-config" entry in the inode directory that is > your homedir. it points either to the old file, or the new file. it > cannot point to both, and it will not point to anything but those two > possibilities. This is what the atomicity of the operation is expected > to guarantee. > > --dkg Actually I've been wrong about this... I've thought that the way the file is "overwritten" is actually done by either two `rename` calls or by `link`, followed by another `link`, and then finally an `unlink` (this is what I've tried to explain in my previous email). In fact I've thought that the `rename` OS call can't overwrite a file if it exists. But after reading the man page of `rename(2)` -- quoted below -- I was indeed wrong to call the atomicity as being "pseudo". int rename(const char *oldpath, const char *newpath); ... If newpath already exists it will be atomically replaced (subject to a few conditions; see ERRORS below), so that there is no point at which another process attempting to access newpath will find it missing. ... If newpath exists but the operation fails for some reason rename() guarantees to leave an instance of newpath in place. So indeed the behavior is completely atomic. Ciprian. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: `notmuch setup` replaces `~/.notmuch-config` instead of truncating it
On 11/16/2010 03:37 PM, Daniel Kahn Gillmor wrote: > On 11/16/2010 03:26 PM, Ciprian Dorin, Craciun wrote: >> So in the light of the above quoted "glitches", my question is: >> due to the small chance of a power loss happening right when we write >> such a small file, doesn't the inconvenience weight more than the >> (fairly remote probable) file loss? > > What inconvenience? The inconvenience of writing the code correctly? Ah sorry -- on re-reading, i see you're probably referring to the inconvenience of breaking hardlinks, dropping permissions, ACLs and other metadata. That's an open question, as far as i'm concerned. it'd be nice if there was a way to "clone" a file's permissions and metadata to get the best of both worlds. maybe someone knows some tricks to do that? --dkg signature.asc Description: OpenPGP digital signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: `notmuch setup` replaces `~/.notmuch-config` instead of truncating it
On 11/16/2010 03:26 PM, Ciprian Dorin, Craciun wrote: > So in the light of the above quoted "glitches", my question is: > due to the small chance of a power loss happening right when we write > such a small file, doesn't the inconvenience weight more than the > (fairly remote probable) file loss? What inconvenience? The inconvenience of writing the code correctly? that's a small one-time cost compared to *any* risk of a user ending up with a damaged .notmuch-config (not to mention possible concurrent uses of notmuch seeing an intermediate version of the file) > P.S.: I say "pseudo" atomic because only the rename is atomic, > thus in order to override file `a` for the target file `b` which > exists, we must execute two **non-atomic** operations as a whole, but > each atomic in part, rename operations: make `b` -> `c`, and then > rename `a` -> `b`. So there is actually a small time-frame when I can > be left with two files (`a` and `c`), none of which is my config file > `b`. (This can be solved when opening the config file by checking if > there isn't any leftover `c` or `a` file, in which case I take the `a` > file and complete the rename.) There is only one ".notmuch-config" entry in the inode directory that is your homedir. it points either to the old file, or the new file. it cannot point to both, and it will not point to anything but those two possibilities. This is what the atomicity of the operation is expected to guarantee. --dkg signature.asc Description: OpenPGP digital signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: `notmuch setup` replaces `~/.notmuch-config` instead of truncating it
On Tue, Nov 16, 2010 at 21:09, Carl Worth wrote: > On Tue, 16 Nov 2010 15:33:30 +0200, "Ciprian Dorin, Craciun" > wrote: >> So my question is: is this behaviour (of deleting the file and >> creating a new one) deliberate? If not, could it be fixed (I could >> provide a patch) to just update the file in place? > > Daniel gave the perfect answer later in the thread. It is intentional to > replace the file with a new, complete version, (to avoid loss/corruption > of the file if "notmuch setup" is interrupted). But we should fix this > to replace the target of any symlinks. > > -Carl I understand now the reason for your file replacement choice. (I'll look over tomorrow to see if I can provide a patch on the line Daniel has described). But -- in general, and totally overlooking the "pseudo" atomic effect obtained from of POSIX file semantics -- doesn't this practice mislead some software (like backup systems, etc.) that would rely maybe on the inode number as part of the identity of the file? Moreover what if the user has set any ACL's or extended attributes on the file, wouldn't these be lost? (Wouldn't also SELinux be bothered?) So after browsing through the source code, I've found inside `notmuch-config.c`, inside the function `notmuch_config_save`, the call which actually overrides the file: `g_file_set_contents (config->filename, data, length, &error)`. Now searching for the documentation of this function, I've stumbled upon, from which I cite (I've never used glib before, so maybe the link is not the best one): http://library.gnome.org/devel/glib/unstable/glib-File-Utilities.html On Unix, if filename already exists hard links to filename will break. Also since the file is recreated, existing permissions, access control lists, metadata etc. may be lost. If filename is a symbolic link, the link itself will be replaced, not the linked file. So in the light of the above quoted "glitches", my question is: due to the small chance of a power loss happening right when we write such a small file, doesn't the inconvenience weight more than the (fairly remote probable) file loss? (I must admit I've lost once the `/etc/network/interfaces` file after an edit and immediately after a quick cold reboot, but it was my fault as I've not sync-ed the file system.) Ciprian. P.S.: I say "pseudo" atomic because only the rename is atomic, thus in order to override file `a` for the target file `b` which exists, we must execute two **non-atomic** operations as a whole, but each atomic in part, rename operations: make `b` -> `c`, and then rename `a` -> `b`. So there is actually a small time-frame when I can be left with two files (`a` and `c`), none of which is my config file `b`. (This can be solved when opening the config file by checking if there isn't any leftover `c` or `a` file, in which case I take the `a` file and complete the rename.) ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: `notmuch setup` replaces `~/.notmuch-config` instead of truncating it
On Tue, 16 Nov 2010 15:33:30 +0200, "Ciprian Dorin, Craciun" wrote: > First congratulations for the nice software! Thanks so much! Welcome to notmuch. > So my question is: is this behaviour (of deleting the file and > creating a new one) deliberate? If not, could it be fixed (I could > provide a patch) to just update the file in place? Daniel gave the perfect answer later in the thread. It is intentional to replace the file with a new, complete version, (to avoid loss/corruption of the file if "notmuch setup" is interrupted). But we should fix this to replace the target of any symlinks. -Carl -- carl.d.wo...@intel.com pgpOpuqnY5Bcy.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: `notmuch setup` replaces `~/.notmuch-config` instead of truncating it
On 11/16/2010 08:38 AM, Jameson Rollins wrote: > On Tue, 16 Nov 2010 15:33:30 +0200, "Ciprian Dorin, Craciun" > wrote: >> So my question is: is this behaviour (of deleting the file and >> creating a new one) deliberate? If not, could it be fixed (I could >> provide a patch) to just update the file in place? > > Hi, Ciprian. I had not noticed this, but now that you mention it, I see > that the same thing happened to me. This behavior is surely not > deliberate, and is definitely undesirable. A patch would be welcome. It often is desirable to do a replacement instead of truncation or appending -- atomic replacement guarantees that a legitimate copy is always available. It eliminates a window of time when the config file would be in an intermediate state. Readers of the file always get either the old state or the new state -- nothing in between. However, you can address both concerns by detecting whether ~/.notmuch-config is a symlink and readlink()ing until you get to the underlying file. Then atomically replace the target instead of the symlink itself. --dkg signature.asc Description: OpenPGP digital signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: `notmuch setup` replaces `~/.notmuch-config` instead of truncating it
On Tue, 16 Nov 2010 15:33:30 +0200, "Ciprian Dorin, Craciun" wrote: > But in my particular case `~/.notmuch-config` is symlinked to an > applications configuration directory which is versioned. Thus I've > expected than when notmuch updates the config, it opens it for > read-write, but with the truncation flag (which as a consequence would > have modified the symlinked file). But instead it deleted the symlink, > and replaced it with a newly created file (thus breaking my custom > configuration backup system.) > > So my question is: is this behaviour (of deleting the file and > creating a new one) deliberate? If not, could it be fixed (I could > provide a patch) to just update the file in place? Hi, Ciprian. I had not noticed this, but now that you mention it, I see that the same thing happened to me. This behavior is surely not deliberate, and is definitely undesirable. A patch would be welcome. jamie. pgpwRjKRcdOdx.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
