Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

[Anders Hovmöller]

> On 14 May 2019, at 04:24, Steven D'Aprano  wrote:
> 
> It's not "cryptic" that Paris, France is difference from Paris, Texas. 
> Let's not abuse the word "cryptic" for things which are easy to resolve.

I'm sorry I thought you were the one arguing FOR using the same word for two 
very different things? *wink*

The real argument is that it's cryptic because the reader of the code won't 
know they need to resolve it. That's the pitfall. 

/ Anders 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

[Anders Hovmöller]

> On 14 May 2019, at 04:24, Steven D'Aprano  wrote:
> 
> *wink*. Function names are mnemonics, not documentation.

Certainly not with that attitude. But it easily could be. Maybe you would be 
fine with a language where all function names are just hex values? *wink* (yes 
I think this was rude but that's my point). 

> mnemonics

Agreed, and like keys in a dict they work badly if there's a collision. 

/ Anders ___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

[Stephen J. Turnbull]

Steven D'Aprano writes:

 > Of course it is enough to presume that if they come from different 
 > modules, they are probably different unless proven otherwise.

That mere presumption is not helpful because YOLO.  Most of us have
limited time, and that includes doc-reading time.

 > As I have shown (perhaps you stopped reading and didn't bother
 > reading through the rest of my post) it is not uncommon for
 > different namespaces to define radically different functions with
 > the same name.

I read the whole post, as I always do for your posts.

 > I don't think you have understood the issue here. Perhaps you 
 > should re-read the original post in the thread.

The issue I'm responding to is your attitude, which is amply displayed
here as well.  So let me rephrase.

 > We're just bike-shedding over its spelling and where it lives[.]

And the color of the bikeshed turns on the connotations of the name.
You (like me) spend a lot of effort on documentation and precise
semantics.  IME most developers do not spend nearly as much.  And even
for me, even if I recognize vaguely that shutils.symlink and
os.symlink are different, I'm unlikely to know which has the semantics
I need or even that the difference is relevant to me, unless I'm
looking for it specifically.  Serge's concern is valid, but perhaps
not the most important.  Even if it's not, your answer to Anders was
much better (even if perhaps a bit sarcastic):

Steven D'Aprano writes:
 > On Mon, May 13, 2019 at 08:23:34PM +0200, Anders Hovmöller wrote:

 > > And if you "from x import y" the call site is identical for the
 > > two different functions. This is just asking for trouble.

 > Clearly your threshhold for "trouble" is a lot more sensitive than
 > mine.

Whose threshold is closest to *the rest of the Python community* is
the relevant consideration, and an empirical question.  My guess is
that Serge and Anders are closer to it than you and I are.

Whether I'm right about that or not, I am now asking you to show more
respect for that diversity in the way you phrase responses.

Regards,
Steve

-- 
Associate Professor  Division of Policy and Planning Science
http://turnbull.sk.tsukuba.ac.jp/ Faculty of Systems and Information
Email: turnb...@sk.tsukuba.ac.jp   University of Tsukuba
Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

[Anders Hovmöller]

> On 14 May 2019, at 03:33, Steven D'Aprano  wrote:
> 
>> On Tue, May 14, 2019 at 12:22:05AM +0300, Serge Matveenko wrote:
>> 
>> As a regular library user, I see and expect no obvious difference
>> between `os.symlink` and `shutil.symlink`. 
> 
> You "see ... no obvious difference" between two functions that live in 
> completely different modules?

This happens due to historical accidents all the time. 

> Isn't the fact that they live in *different modules* obvious enough?

No. See above. 

> If os.symlink was exactly the same as shutil.symlink, we would not need 
> them both.

Sure we could. See above. 

> Do you also see and expect no obvious difference between list.remove and 
> os.remove? If so, today is your opportunity to learn something and 
> become a better programmer! *wink*

That's rude. That might work in person but I'd doubt it. Over email the "wink" 
doesn't change it enough. 

/ Anders 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (meta) Broken link to gmane.org archive of python-ideas

[Brett Cannon]

Bugs about the website should be reported to
https://github.com/python/pythondotorg .

On Mon, May 13, 2019 at 7:42 AM Tom Hale  wrote:

> On 13/5/19 5:21 pm, Steven D'Aprano wrote:
> > If you don't like the mailman archives here
> >
> > https://mail.python.org/pipermail/python-ideas/
> >
> > you might try ActiveState:
> >
> > https://code.activestate.com/lists/python-ideas/
>
> Would https://bugs.python.org/ be an appropriate place for suggesting
> the updated links, or is there a specific channel for web updates?
>
> --
> Tom Hale
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

[Steven D'Aprano]

On Tue, May 14, 2019 at 11:43:32AM +0900, Stephen J. Turnbull wrote:

[...]
>  > Isn't the fact that they live in *different modules* obvious
>  > enough?
> 
> Please, Steve, you didn't need to go there.
> 
> Because it's not enough.  

Of course it is enough to presume that if they come from different 
modules, they are probably different unless proven otherwise.

As I have shown (perhaps you stopped reading and didn't bother reading 
through the rest of my post) it is not uncommon for different namespaces 
to define radically different functions with the same name.


> It's not unheard of for modules to include
> functionality from other modules with "thin wrappers", merely to
> change the calling convention, or even just to avoid an import.  Even
> within the stdlib.

Um, yes? And how is that relevant? If there's a "thin wrapper" 
changing the calling convention, then the APIs are different 
and they are not the same. That supports my position: functions with the 
same name coming from different namespaces are likely to be different. 
Even if a few happen to be the same that doesn't justify *assuming* that 
all such functions will be the same.

If they're just an import, then *unless documented otherwise* they are 
considered an implementation detail not a public part of the API.


> The point is that subtle issues like the presence
> or absence of race conditions require careful reading of the
> documentation, when some of said documentation may not even be in
> Python in the case of modules like os.

Lack of documentation means we should fix the documentation.


> If one of these has a race condition and the other doesn't, I think it
> would be reasonable for the documentation of the racy one to
> acknowledge the bug and point to the non-racy one.

I don't think you have understood the issue here. Perhaps you 
should re-read the original post in the thread.

There is no buggy existing version of symlink with a race condition. If 
the destination exists, os.symlink correctly does the right thing: it 
raises an error.

py> os.symlink('/tmp/spam', '/tmp/eggs')
Traceback (most recent call last):
  File "", line 1, in 
FileExistsError: [Errno 17] File exists: '/tmp/spam' -> '/tmp/eggs'

And there is currently no shutils version at all.

The problem is a lack of a symlink function that safely overwrites an 
existing file or symlink. We're just bike-shedding over its spelling 
and where it lives:

- modify os.symlink and give it a "force" parameter
- add a new function into shutils



-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

[Stephen J. Turnbull]

Steven D'Aprano writes:
 > On Tue, May 14, 2019 at 12:22:05AM +0300, Serge Matveenko wrote:
 > 
 > > As a regular library user, I see and expect no obvious difference
 > > between `os.symlink` and `shutil.symlink`. 
 > 
 > You "see ... no obvious difference" between two functions that live in 
 > completely different modules?
 > 
 > Isn't the fact that they live in *different modules* obvious
 > enough?

Please, Steve, you didn't need to go there.

Because it's not enough.  It's not unheard of for modules to include
functionality from other modules with "thin wrappers", merely to
change the calling convention, or even just to avoid an import.  Even
within the stdlib.  The point is that subtle issues like the presence
or absence of race conditions require careful reading of the
documentation, when some of said documentation may not even be in
Python in the case of modules like os.

If one of these has a race condition and the other doesn't, I think it
would be reasonable for the documentation of the racy one to
acknowledge the bug and point to the non-racy one.  I'm not sure it's
a great idea to put lots of effort into fixing shutils.  I just assume
that one is as risky (racy, risque) as any shell!


-- 
Associate Professor  Division of Policy and Planning Science
http://turnbull.sk.tsukuba.ac.jp/ Faculty of Systems and Information
Email: turnb...@sk.tsukuba.ac.jp   University of Tsukuba
Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

[Steven D'Aprano]

On Mon, May 13, 2019 at 08:23:34PM +0200, Anders Hovmöller wrote:

> > Being found in different modules should always be treated as a sign that 
> > the functions may be different. (If they're the same, why do we need 
> > both?)
> 
> Assuming all users know there is another.

How do users know the os module exists in the first place? How do they 
know to use list.append rather than list += [item]? If you're going to 
cater only for the most ignorant, know-nothing subset of users, that's 
an argument against providing *any* functionality at all since they 
won't know it exists.

We ought to assume competent users who know how to *find stuff out* by 
reading the docs, asking on Stackoverflow or other forums, or by reading 
other people's code and learning from what they see.


> And if you "from x import y" the call site is identical for the two 
> different functions. This is just asking for trouble. 

Clearly your threshhold for "trouble" is a lot more sensitive than mine.

Using `from module import ...` is always a compromise between verbosity 
and clarity. We don't want to be like Java:

configs.config.server-config.monitoring-service.module-monitoring-levels

(I copied that example from the Oracle docs) or even worse. But short 
names can sometimes be less clear. If the coder is using `from module 
import` they are explicitly preferring brevity, *and that is okay*.

Any even slightly experienced Python programmer will know what to do 
when you see an unfamiliar function in code:

syslink(...)

Most of the time we don't need to drill down in fine detail. "It makes 
symlinks" is enough for us to move on and we really don't want names 
which explicitly document everything about the function:


create_symlink_to_file_on_posix_filesystems_or_windows_vista_or_better_but_not_if_destination_already_exists(...)

*wink*. Function names are mnemonics, not documentation.

But if you do need to drill down and look at the function in fine 
detail, we know what to do: look for a local function definition, or a 
from import, and take it from there. In the worst case, you might have 
both in the same namespace so you need to resolve which one wins.

This is called programming. We all do it. It isn't any trouble except in 
cases of deliberately or accidently obfuscated code.


> >> An optional "overwrite_if_exists=False" flag seems much nicer.
> > 
> > Aside from the argument name being too verbose, that violates the rule 
> > of thumb "avoid constant bool flags" design principle.
> 

> Verbose is better than cryptic. Having the exact same name as 
> something that does something else is pretty cryptic.

It really isn't. That's why we have namespaces (packages, modules, 
classes) so we don't have to use unique names for everything.

It's not "cryptic" that Paris, France is difference from Paris, Texas. 
Let's not abuse the word "cryptic" for things which are easy to resolve.
This is cryptic:

https://britishlibrary.typepad.co.uk/digitisedmanuscripts/2015/08/help-us-decipher-this-inscription.html

not having two symlink functions in different modules.



> Then a different name seems also to be in order. symlink/setsymlink 
> seems a lot better to me for example.

"set" adds nothing to the function name that isn't already implied. You 
might as well just prefix it with any random three characters:

symlink  # sets a symlink
gwqsymlink   # sets a symlink

Among those who don't read the docs, all you are doing is giving them 
three extra characters to type when they post to Stackoverflow:

"What's the difference between os.symlink and shutil.[set]symlink?"

Arguably "forcesymlink" or "force_symlink" would be better, but given 
that option, I'd flip my preference to a force parameter:

+1   shutil.symlink
+0   os.symlink with force=False parameter
-0.5 shutil.force_symlink
-1   shutil.setsymlink



-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

[Steven D'Aprano]

On Tue, May 14, 2019 at 12:22:05AM +0300, Serge Matveenko wrote:

> As a regular library user, I see and expect no obvious difference
> between `os.symlink` and `shutil.symlink`. 

You "see ... no obvious difference" between two functions that live in 
completely different modules?

Isn't the fact that they live in *different modules* obvious enough? If 
os.symlink was exactly the same as shutil.symlink, we would not need 
them both.

Do you also see and expect no obvious difference between list.remove and 
os.remove? If so, today is your opportunity to learn something and 
become a better programmer! *wink*

Modules and classes are both namespaces, and we have namespaces with 
different names precisely so that they can do something different. This 
is why math.sqrt and cmath.sqrt are different:

py> math.sqrt(-2)
Traceback (most recent call last):
  File "", line 1, in 
ValueError: math domain error
py> cmath.sqrt(-2)
1.4142135623730951j

This is why *all of these* are different:

- os.open
- gzip.open
- bz2.open
- aifc.open
- shelve.open
- tokenize.open
- wave.open
- webbrowser.open

and none of those are the same as the builtin open.

A few more examples to prove this isn't an isolated thing:

* re.split, shlex.split and str.split
* dis.dis, pickletools.dis
* copy.copy, shutil.copy

The bottom line is that it is completely normal and not uncommon for 
functions to be distinguished by the namespace they are found in. It is 
both unreasonable and unnecessary to force objects in different 
namespaces to have unique names.


> Probably they should have
> different names if the behavior is not the same.

The behaviour is broadly the same. They both create a symlink. Only the 
fine details are different.


> As a constant Linux user, I'd expect a `symlink` function to do
> something similar to `ln -s` which also could be used as `ln -sf`. So,
> something like `symlink(force:bool=False)` looks like an expected and
> "guessable".

That's not an unreasonable suggestion. But the Windows mklink command 
does not seem to take a "force" parameter:

https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/mklink

so not so guessable to Windows users.

The ln command on my Linux system takes sixteen options, and we surely 
don't expect to match all of them.

In my previous response, I already explained why constant bool 
parameters were (often) an interface anti-pattern. I'm on the fence with 
this one: I don't think that os.symlink(... force=False) is awful, but I 
still prefer the more complex functionality to go into shutils and 
leave the os version to be a thin wrapper around the basic OS 
functionality.

If there's no consensus here, I guess the final decision will be that of 
the person doing the work.


-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

[Serge Matveenko]

On Mon, May 13, 2019 at 9:24 PM Anders Hovmöller  wrote:
> > On 13 May 2019, at 19:38, Steven D'Aprano  wrote:
> >> On Mon, May 13, 2019 at 12:31:08PM +0200, Anders Hovmöller wrote:
> >> An optional "overwrite_if_exists=False" flag seems much nicer.
> >
> > Aside from the argument name being too verbose, that violates the rule
> > of thumb "avoid constant bool flags" design principle.
>
> Verbose is better than cryptic. Having the exact same name as something that 
> does something else is pretty cryptic.

As a regular library user, I see and expect no obvious difference
between `os.symlink` and `shutil.symlink`. Probably they should have
different names if the behavior is not the same.

As a constant Linux user, I'd expect a `symlink` function to do
something similar to `ln -s` which also could be used as `ln -sf`. So,
something like `symlink(force:bool=False)` looks like an expected and
"guessable".

Thanks!
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

[Anders Hovmöller]

> On 13 May 2019, at 19:38, Steven D'Aprano  wrote:
> 
>> On Mon, May 13, 2019 at 12:31:08PM +0200, Anders Hovmöller wrote:
>> 
>> 
>>> On 13 May 2019, at 11:38, Tom Hale  wrote:
>>> 
>>> * os.symlink() - the new link name is expected to NOT exist
>>> * shutil.symlink() - the new symlink replaces an existing file
>> 
>> This seems error prone to me. There is nothing about those names that 
>> hint at the difference in behavior. 
> 
> One of them is in the `os` module, and therefore we should expect that 
> it will be a thin wrapper around the OS functionality.
> 
> The other is in the `shutil` module, and therefore we should expect that 
> it will be a shell utility function of arbitrary complexity.
> 
> Being found in different modules should always be treated as a sign that 
> the functions may be different. (If they're the same, why do we need 
> both?)

Assuming all users know there is another. And if you "from x import y" the call 
site is identical for the two different functions. This is just asking for 
trouble. 

> 
>> An optional "overwrite_if_exists=False" flag seems much nicer.
> 
> Aside from the argument name being too verbose, that violates the rule 
> of thumb "avoid constant bool flags" design principle.

Verbose is better than cryptic. Having the exact same name as something that 
does something else is pretty cryptic. 


> (Note that this is a *rule of thumb*, not a hard law: there are times 
> that it can and should be violated; its also not a well-known principle 
> and so lots of APIs violate it even when they shouldn't.)
> 
> If you have a function which takes a boolean argument to swap between 
> two different modes, and the usual calling pattern is to pass a 
> constant:
> 
>function(arg, x, y, flag=True)
> 
> rather than a flag which is not know until runtime, then it is (usually) 
> better to use two distinct functions rather than one function with a 
> parameter that swaps modes.

OK. Then a different name seems also to be in order. symlink/setsymlink seems a 
lot better to me for example. 

/ Anders 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

[Steven D'Aprano]

On Mon, May 13, 2019 at 04:38:08PM +0700, Tom Hale wrote:

> As suggested by Toshio Kuratomi at https://bugs.python.org/issue36656, I 
> am raising this here for inclusion in the shutil module.
> 
> Mimicking POSIX, os.symlink() will raise FileExistsError if the link 
> name to be created already exists.

Seems reasonable. I presume that's the current behaviour.


> A common use case is overwriting an existing file (often a symlink) with 
> a symlink. Naively, one would delete the file named link_name file if it 
> exists, then call symlink(). This "solution" is already 3 lines of code, 
> and without exception handling it introduces the race condition of a 
> file named link_name being created between unlink and symlink.
> 
> Depending on the functionality required, I suggest:
> 
> * os.symlink() - the new link name is expected to NOT exist
> * shutil.symlink() - the new symlink replaces an existing file

+1


> One issue I see with my suggested code is that the file at 
> temp_link_name could be changed before target is replaced with it. This 
> is mitigated by the randomness introduced by mktemp().

Arguing over the fine details of implementation should probably be done 
on the bug tracker, between the implementor and whoever is reviewing the 
code. As far as I'm concerned, the proposed interface is a good one:

- os.symlink for the low level function
- shutil.symlink for a high-level wrapper that handles the common case
  of wanting to overwrite an existing file.



-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

[Steven D'Aprano]

On Mon, May 13, 2019 at 12:31:08PM +0200, Anders Hovmöller wrote:
> 
> 
> > On 13 May 2019, at 11:38, Tom Hale  wrote:
> > 
> > * os.symlink() - the new link name is expected to NOT exist
> > * shutil.symlink() - the new symlink replaces an existing file
> 
> This seems error prone to me. There is nothing about those names that 
> hint at the difference in behavior. 

One of them is in the `os` module, and therefore we should expect that 
it will be a thin wrapper around the OS functionality.

The other is in the `shutil` module, and therefore we should expect that 
it will be a shell utility function of arbitrary complexity.

Being found in different modules should always be treated as a sign that 
the functions may be different. (If they're the same, why do we need 
both?)



> An optional "overwrite_if_exists=False" flag seems much nicer.

Aside from the argument name being too verbose, that violates the rule 
of thumb "avoid constant bool flags" design principle.

(Note that this is a *rule of thumb*, not a hard law: there are times 
that it can and should be violated; its also not a well-known principle 
and so lots of APIs violate it even when they shouldn't.)

If you have a function which takes a boolean argument to swap between 
two different modes, and the usual calling pattern is to pass a 
constant:

function(arg, x, y, flag=True)

rather than a flag which is not know until runtime, then it is (usually) 
better to use two distinct functions rather than one function with a 
parameter that swaps modes.

Another way to put it: in general, if you have two modes, you should 
have two functions.

For example:

- str.find and rfind, not str.find with a from_right parameter;

- bisect.insort_left and insort_right, not bisect.insort with a
  from_left parameter;

- statistics.stdev and pstdev rather than population parameter;

- os.setuid and setgid rather than a change_group parameter;

- zip and itertools.zip_longest, not zip with longest parameter.


This principle doesn't apply when the flag is typically not known until 
runtime. For example, this is unfortunate:

if get_some_flag(*args):
result = str.find(spam)
else:
result = str.rfind(spam)

but rare. If it were very common, then it would be justified to provide 
a single API with a mode flag switching behaviours.

The idea of "no constant bool arguments" is that if you typically 
know which mode you want at edit-time (that includes runtime in 
interactive environments), the two modes should be distinguished by 
function name rather than by a parameter.

I expect that, typically, users of this will know ahead of time whether 
they want to overwrite symlinks or not.




-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (meta) Broken link to gmane.org archive of python-ideas

[Tom Hale]

On 13/5/19 5:21 pm, Steven D'Aprano wrote:

If you don't like the mailman archives here

https://mail.python.org/pipermail/python-ideas/

you might try ActiveState:

https://code.activestate.com/lists/python-ideas/


Would https://bugs.python.org/ be an appropriate place for suggesting 
the updated links, or is there a specific channel for web updates?


--
Tom Hale
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Break multiple loop levels

[haael]

How about doing it pythonic way then, "everything is a hack on a symbol 
table".


Let normal loops remain normal loops.

Let's introduce a special construction:

> for x in iterator as loop_control_object:
> loop_body(loop_control_object)


The iterator in the loop would be wrapped inside another iterator that 
would yield everything as the original iterator, but it would also be 
controllable through loop_control_object.


Sample implementation:

> def make_control_object(iterator):
>control_object = ControlObject()
>control_object.running = True
>def breakable_iterator():
>for x in iterator:
>yield x
>if not control_object.running:
>return
>return breakable_iterator(), control_object

The control object could have methods that allow sophisticated loop 
control, such as breaking, continuing, injecting exceptions, counting 
iterations, skipping etc.


The keywords 'break loop_control_object' and 'continue 
loop_control_object' could be simply implemented as:


> loop_control_object.do_break()



On Sun, May 12, 2019, 5:36 PM Paul Moore > wrote:


On Sun, 12 May 2019 at 21:06, David Mertz mailto:me...@gnosis.cx>> wrote:
 > I thought of 'as' initially, and it reads well as English. But it
felt to me like the meaning was too different from the other
meanings of 'as' in Python. I might be persuaded otherwise.

If you think in terms of "named loops" rather than "labels", "as"
makes a lot more sense.


Every name created with 'as' currently is a regular Python object, with 
a memory address, and some methods, that can participate in comparison 
(at least equality), that has a type, can be passed as a function 
argument, etc. Actually, all that is true of EVERY name in Python other 
than keywords.


It seems like none of this would be true of a loop name/label. Hence my 
first thought that a different way of introducing the word is more 
clear... Maybe we can just prefix lines with labels instead, and add a 
'goto' command :-)


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (meta) Broken link to gmane.org archive of python-ideas

[Stefan Krah]

On Mon, May 13, 2019 at 08:21:59PM +1000, Steven D'Aprano wrote:
> Alas, Gmane is only a shadow of its former self. Most of the archives 
> are either lost or simply not restored.

It is also worth pointing put that the "new" communication modes like
Discourse, Slack or Gitter have no decent searchability or archives.


Stefan Krah



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

[Anders Hovmöller]

> On 13 May 2019, at 11:38, Tom Hale  wrote:
> 
> * os.symlink() - the new link name is expected to NOT exist
> * shutil.symlink() - the new symlink replaces an existing file

This seems error prone to me. There is nothing about those names that hint at 
the difference in behavior. An optional "overwrite_if_exists=False" flag seems 
much nicer.

/ Anders 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (meta) Broken link to gmane.org archive of python-ideas

[Steven D'Aprano]

On Mon, May 13, 2019 at 04:37:16PM +0700, Tom Hale wrote:
> Heya all,
> 
> I've joined this list and wanted to read the archives to get a flavour 
> of the community here.
> 
> I noticed that the link with text:
> 
>gmane.org archive of python-ideas

Alas, Gmane is only a shadow of its former self. Most of the archives 
are either lost or simply not restored.

http://home.gmane.org/category/news/

If you don't like the mailman archives here

https://mail.python.org/pipermail/python-ideas/

you might try ActiveState:

https://code.activestate.com/lists/python-ideas/



-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

[Tom Hale]

As suggested by Toshio Kuratomi at https://bugs.python.org/issue36656, I 
am raising this here for inclusion in the shutil module.


Mimicking POSIX, os.symlink() will raise FileExistsError if the link 
name to be created already exists.


A common use case is overwriting an existing file (often a symlink) with 
a symlink. Naively, one would delete the file named link_name file if it 
exists, then call symlink(). This "solution" is already 3 lines of code, 
and without exception handling it introduces the race condition of a 
file named link_name being created between unlink and symlink.


Depending on the functionality required, I suggest:

* os.symlink() - the new link name is expected to NOT exist
* shutil.symlink() - the new symlink replaces an existing file

Handling all possible race conditions (some detailed in issue36656) is 
non-trivial, however this is the best that I have come up with so far:


==

import os, tempfile

def symlink(target, link_name):
'''Create a symbolic link link_name pointing to target.
Overwrites link_name if it exists. '''

# os.replace() may fail if files are on different filesystems
link_dir = os.path.dirname(link_name)

# Link to a temporary filename that doesn't exist
while True:
temp_link_name = tempfile.mktemp(dir=link_dir)

# os.* functions mimic as closely as possible system functions
# The POSIX symlink() returns EEXIST if link_name already exists
# 
https://pubs.opengroup.org/onlinepubs/9699919799/functions/symlink.html

try:
os.symlink(target, temp_link_name)
break
except FileExistsError:
pass

# Replace link_name with temp_link_name
try:
# Pre-empt os.replace on a directory with a nicer message
if os.path.isdir(link_name):
raise IsADirectoryError(f"Cannot symlink over existing 
directory: '{link_name}'")

os.replace(temp_link_name, link_name)
except:
if os.path.islink(temp_link_name):
os.remove(temp_link_name)
raise

==

The documentation (https://docs.python.org/3/library/shutil.html) I 
suggest for this is:


shutil.symlink(target, link_name)
Create a symbolic link named link_name pointing to target, overwriting 
target if it exists. If link_name is a directory, IsADirectoryError is 
raised. To not overwrite target, use os.symlink()


==

It would be tempting to do:

while True:
try:
os.symlink(target, link_name)
break
except FileExistsError:
os.remove(link_name)

But this has a race condition when replacing a symlink should should 
*always* exist, eg:


/lib/critical.so -> /lib/critical.so.1.2

When upgrading by:

symlink('/lib/critical.so.2.0', '/lib/critical.so')

There is a point in time when /lib/critical.so doesn't exist.

==

One issue I see with my suggested code is that the file at 
temp_link_name could be changed before target is replaced with it. This 
is mitigated by the randomness introduced by mktemp().


While it is far less likely that a file is accessed with a random and 
unknown name than with an existing known name, I seek input on a 
solution if this is an unacceptable risk.



Prior art:

* https://bugs.python.org/issue36656 (already mentioned above)
* https://stackoverflow.com/a/55742015/5353461
* https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/ln.c

--
Tom Hale
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] (meta) Broken link to gmane.org archive of python-ideas

[Tom Hale]

Heya all,

I've joined this list and wanted to read the archives to get a flavour 
of the community here.


I noticed that the link with text:

   gmane.org archive of python-ideas

on the page:

https://www.python.org/community/lists/

gives a HTML page with the only text being:

Page Not Found

I couldn't easily work out what the correct link should be, but part of 
the solution seems to be httpS://


Perhaps someone here knows how to wrangle gmane.org?

--
Tom Hale
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Path conversion for f-strings

[Serge Matveenko]

On Mon, May 13, 2019 at 4:44 AM Batuhan Taskaya  wrote:
>
> Like repr and string the file system path is used alot and something like 
> path!p might be handy.
>
> >>> class MyClass:
> ... def __fspath__(self):
> ... return "/home/batuhan"
> ...
> >>> assert f"{mc!p}" == f"{os.fspath(mc)}"
>
> Also it saves us to unnecessarily import os for only fspath().

Hi! Honestly, I feel a lack of use case context here.

Just to mention, `pathlib.Path` like objects have there `__str__`
implement to return a string representation of the path, i.e.
`str(pathlib.Path("/home/batuhan"))` returns `"/home/batuhan"`
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/