Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-16 Thread Antoine Pitrou
On Thu, 16 May 2019 18:05:39 +0300 Serhiy Storchaka wrote: > 16.05.19 17:05, Antoine Pitrou пише: > > On Thu, 16 May 2019 16:04:36 +0300 > > Serhiy Storchaka > > wrote: > >> 16.05.19 14:33, Antoine Pitrou пише: > >>> On Thu, 16 May 2019 13:05:48 +0300 > >>> Serhiy Storchaka > >>> wrote: >

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-16 Thread Serhiy Storchaka
16.05.19 17:05, Antoine Pitrou пише: On Thu, 16 May 2019 16:04:36 +0300 Serhiy Storchaka wrote: 16.05.19 14:33, Antoine Pitrou пише: On Thu, 16 May 2019 13:05:48 +0300 Serhiy Storchaka wrote: 16.05.19 11:28, Barry Scott пише: To replace one symlink with another atomically is possible by

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-16 Thread Antoine Pitrou
On Thu, 16 May 2019 16:04:36 +0300 Serhiy Storchaka wrote: > 16.05.19 14:33, Antoine Pitrou пише: > > On Thu, 16 May 2019 13:05:48 +0300 > > Serhiy Storchaka > > wrote: > >> 16.05.19 11:28, Barry Scott пише: > >>> To replace one symlink with another atomically is possible by using > >>>

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-16 Thread Tom Hale
Let me prefix all of this by saying that a shutil.symlink_with_overwrite is useful (see below), but limited in scope. Please see my next post on emulating POSIX ln. On 16/5/19 2:02 pm, Serhiy Storchaka wrote: This may be an interesting project on PyPi, but I do not think that we need to

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-16 Thread Serhiy Storchaka
16.05.19 14:33, Antoine Pitrou пише: On Thu, 16 May 2019 13:05:48 +0300 Serhiy Storchaka wrote: 16.05.19 11:28, Barry Scott пише: To replace one symlink with another atomically is possible by using rename() or renameat() something like: os.symlink( src, tmp_symlink )

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-16 Thread Antoine Pitrou
On Thu, 16 May 2019 13:05:48 +0300 Serhiy Storchaka wrote: > 16.05.19 11:28, Barry Scott пише: > > To replace one symlink with another atomically is possible by using > > rename() or renameat() > > something like: > > > > os.symlink( src, tmp_symlink ) > > os.rename( tmp_symlink, dst )

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-16 Thread Barry Scott
> On 16 May 2019, at 11:05, Serhiy Storchaka wrote: > > 16.05.19 11:28, Barry Scott пише: >> To replace one symlink with another atomically is possible by using rename() >> or renameat() >> something like: >> os.symlink( src, tmp_symlink ) >> os.rename( tmp_symlink, dst ) > >

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-16 Thread Serhiy Storchaka
16.05.19 11:28, Barry Scott пише: To replace one symlink with another atomically is possible by using rename() or renameat() something like: os.symlink( src, tmp_symlink ) os.rename( tmp_symlink, dst ) Somebody can replace tmp_symlink between os.symlink() and os.rename().

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-16 Thread Barry Scott
To replace one symlink with another atomically is possible by using rename() or renameat() something like: os.symlink( src, tmp_symlink ) os.rename( tmp_symlink, dst ) Use dir_fd's to taste. I'm sure there is a lot more to cover all the corner cases and attack vectors. I'm not

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-16 Thread Serhiy Storchaka
14.05.19 19:50, Steven D'Aprano пише: On Tue, May 14, 2019 at 02:43:03PM +0300, Serhiy Storchaka wrote: Sorry, but I do not understand what problem do you try to solve. If somebody can create a file named link_name between unlink and symlink, he can also remove and create a file named link_name

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-14 Thread eryk sun
On 5/14/19, Steven D'Aprano wrote: > > On posix systems, you should be able to use chattr +i to make the file > immutable, so that the attacker cannot remove or replace it. Minor point of clarification. File attributes, and APIs to access them, are not in the POSIX standard. chattr is a Linux

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-14 Thread Steven D'Aprano
On Tue, May 14, 2019 at 10:26:28AM +0300, Serge Matveenko wrote: > How about introducing `force=False` argument to > `pathlib.Path.symlink_to` method? > It looks like a good place for this as `pathlib` is actually the place > where higher-level things to operate on paths live already I don't

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-14 Thread Anders Hovmöller
> On 14 May 2019, at 14:13, Steven D'Aprano wrote: > >> On Tue, May 14, 2019 at 10:37:57AM +0300, Serge Matveenko wrote: >> >> My point was that in case of `os.symlink` vs `shutil.symlink` it is >> not obvious how they are different even taking into account their >> namespaces. > > Okay,

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-14 Thread Steven D'Aprano
On Tue, May 14, 2019 at 10:37:57AM +0300, Serge Matveenko wrote: > My point was that in case of `os.symlink` vs `shutil.symlink` it is > not obvious how they are different even taking into account their > namespaces. Okay, but that's not what you said. I can't respond to what you meant to say

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-14 Thread Serhiy Storchaka
13.05.19 12:38, Tom Hale пише: As suggested by Toshio Kuratomi at https://bugs.python.org/issue36656, I am raising this here for inclusion in the shutil module. Mimicking POSIX, os.symlink() will raise FileExistsError if the link name to be created already exists. A common use case is

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-14 Thread eryk sun
On 5/14/19, Serge Matveenko wrote: > > My point was that in case of `os.symlink` vs `shutil.symlink` it is > not obvious how they are different even taking into account their > namespaces. I prefer to reserve POSIX system call names if possible, unless it's a generic name such as "open" or

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-14 Thread Serge Matveenko
On Tue, May 14, 2019 at 4:34 AM Steven D'Aprano wrote: > You "see ... no obvious difference" between two functions that live in > completely different modules? > > > > The bottom line is that it is completely normal and not uncommon for > functions to be distinguished by the namespace they are

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-14 Thread Serge Matveenko
On Tue, May 14, 2019 at 6:19 AM Steven D'Aprano wrote: > The problem is a lack of a symlink function that safely overwrites an > existing file or symlink. We're just bike-shedding over its spelling > and where it lives: > > - modify os.symlink and give it a "force" parameter > - add a new

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-13 Thread Anders Hovmöller
> On 14 May 2019, at 04:24, Steven D'Aprano wrote: > > It's not "cryptic" that Paris, France is difference from Paris, Texas. > Let's not abuse the word "cryptic" for things which are easy to resolve. I'm sorry I thought you were the one arguing FOR using the same word for two very

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-13 Thread Anders Hovmöller
> On 14 May 2019, at 04:24, Steven D'Aprano wrote: > > *wink*. Function names are mnemonics, not documentation. Certainly not with that attitude. But it easily could be. Maybe you would be fine with a language where all function names are just hex values? *wink* (yes I think this was rude

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-13 Thread Stephen J. Turnbull
Steven D'Aprano writes: > Of course it is enough to presume that if they come from different > modules, they are probably different unless proven otherwise. That mere presumption is not helpful because YOLO. Most of us have limited time, and that includes doc-reading time. > As I have

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-13 Thread Anders Hovmöller
> On 14 May 2019, at 03:33, Steven D'Aprano wrote: > >> On Tue, May 14, 2019 at 12:22:05AM +0300, Serge Matveenko wrote: >> >> As a regular library user, I see and expect no obvious difference >> between `os.symlink` and `shutil.symlink`. > > You "see ... no obvious difference" between two

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-13 Thread Steven D'Aprano
On Tue, May 14, 2019 at 11:43:32AM +0900, Stephen J. Turnbull wrote: [...] > > Isn't the fact that they live in *different modules* obvious > > enough? > > Please, Steve, you didn't need to go there. > > Because it's not enough. Of course it is enough to presume that if they come from

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-13 Thread Stephen J. Turnbull
Steven D'Aprano writes: > On Tue, May 14, 2019 at 12:22:05AM +0300, Serge Matveenko wrote: > > > As a regular library user, I see and expect no obvious difference > > between `os.symlink` and `shutil.symlink`. > > You "see ... no obvious difference" between two functions that live in >

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-13 Thread Steven D'Aprano
On Mon, May 13, 2019 at 08:23:34PM +0200, Anders Hovmöller wrote: > > Being found in different modules should always be treated as a sign that > > the functions may be different. (If they're the same, why do we need > > both?) > > Assuming all users know there is another. How do users know

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-13 Thread Steven D'Aprano
On Tue, May 14, 2019 at 12:22:05AM +0300, Serge Matveenko wrote: > As a regular library user, I see and expect no obvious difference > between `os.symlink` and `shutil.symlink`. You "see ... no obvious difference" between two functions that live in completely different modules? Isn't the fact

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-13 Thread Serge Matveenko
On Mon, May 13, 2019 at 9:24 PM Anders Hovmöller wrote: > > On 13 May 2019, at 19:38, Steven D'Aprano wrote: > >> On Mon, May 13, 2019 at 12:31:08PM +0200, Anders Hovmöller wrote: > >> An optional "overwrite_if_exists=False" flag seems much nicer. > > > > Aside from the argument name being too

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-13 Thread Anders Hovmöller
> On 13 May 2019, at 19:38, Steven D'Aprano wrote: > >> On Mon, May 13, 2019 at 12:31:08PM +0200, Anders Hovmöller wrote: >> >> >>> On 13 May 2019, at 11:38, Tom Hale wrote: >>> >>> * os.symlink() - the new link name is expected to NOT exist >>> * shutil.symlink() - the new symlink

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-13 Thread Steven D'Aprano
On Mon, May 13, 2019 at 04:38:08PM +0700, Tom Hale wrote: > As suggested by Toshio Kuratomi at https://bugs.python.org/issue36656, I > am raising this here for inclusion in the shutil module. > > Mimicking POSIX, os.symlink() will raise FileExistsError if the link > name to be created already

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-13 Thread Steven D'Aprano
On Mon, May 13, 2019 at 12:31:08PM +0200, Anders Hovmöller wrote: > > > > On 13 May 2019, at 11:38, Tom Hale wrote: > > > > * os.symlink() - the new link name is expected to NOT exist > > * shutil.symlink() - the new symlink replaces an existing file > > This seems error prone to me. There is

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-13 Thread Anders Hovmöller
> On 13 May 2019, at 11:38, Tom Hale wrote: > > * os.symlink() - the new link name is expected to NOT exist > * shutil.symlink() - the new symlink replaces an existing file This seems error prone to me. There is nothing about those names that hint at the difference in behavior. An optional

[Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

2019-05-13 Thread Tom Hale
As suggested by Toshio Kuratomi at https://bugs.python.org/issue36656, I am raising this here for inclusion in the shutil module. Mimicking POSIX, os.symlink() will raise FileExistsError if the link name to be created already exists. A common use case is overwriting an existing file (often a