[issue31926] compile error when converting selectmodule to AC due to missing #define-s

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 4f57409a2f7bdf8fb559cddc7c6533ca2c471c67 by Serhiy Storchaka (Tal 
Einat) in branch 'master':
bpo-31926: fix missing *_METHODDEF statements by argument clinic (#4230)
https://github.com/python/cpython/commit/4f57409a2f7bdf8fb559cddc7c6533ca2c471c67


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31930] IDLE: Pressing "Home" on Windows places cursor before ">>>"

[Cheryl Sabella]

Cheryl Sabella  added the comment:

On Ubuntu, HOME initially goes to between the text and >>>.  Pressing it again 
moves it before the >>> and then it toggles between the two.  So, it appears to 
be working as expected.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: A use-case for for...else with no break

[Serhiy Storchaka]

02.11.17 12:10, Steve D'Aprano пише:

Occasionally it is useful to loop over a bunch of stuff in the interactive
interpreter, printing them as you go on a single line:

for x in something():
 print(x, end='')

If you do that, the prompt overwrites your output, and you get a mess:


py> for x in "abcdefgh":
... print(x, end='')
...
py> efghpy>


What the interpreter or configuration do you use? The standard 
interpreter uses '>>> ' as a prompt.


--
https://mail.python.org/mailman/listinfo/python-list

[issue31932] setup.py cannot find vcversall.bat on MSWin 8.1 if installed in user AppData

[Hugh Fisher]

New submission from Hugh Fisher :

This was raised in issue #23246 but apparently not addressed at the time.

The Visual C for Python 2.7 tools on my MS Windows 8.1 system installed 
themselves under the invisible AppData directory because I did not do so as 
Administrator. Running setup.py to build a (2.7) native extension then fails 
because it can't find VCVERSALL.BAT.

Fix is a couple of extra lines in find_vcvarsall in msvc9compiler.py to search 
under AppData if it can't be found in the system location. I will submit a pull 
request.

--
components: Distutils
messages: 305470
nosy: dstufft, eric.araujo, laranzu
priority: normal
severity: normal
status: open
title: setup.py cannot find vcversall.bat on MSWin 8.1 if installed in user 
AppData
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: replacing `else` with `then` in `for` and `try`

[Jon Ribbens]

On 2017-11-03, Alexey Muranov  wrote:
> 'Then' describes what happens next indeed, unless some extraordinary 
> situation prevents it from happening, for example:
>
> try:
> go_to_the_bakery()
> then:
> buy_croissants(2)
> except BakeryClosed:
> go_to_the_grociery()
> buy_baguette(1)
> finally:
> come_back()
>
> I know this is a poor program example (why not to use a boolean return 
> value instead of an exception, etc.), and i know that currently in 
> Python `except` must precede `else`, it is just to illustrate the 
> choice of terms.

It looks like you're suggesting not just changing the 'else' keyword
to 'then', but changing the syntax completely. The above is certainly
not an improvement on the status quo, as it is completely
counter-intuitive that exceptions in the 'then' clause will not be
caught by the 'except' clauses.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Alexey Muranov]

On Thu, 2017-11-02 at 16:31 +, Jon Ribbens wrote:

> On 2017-11-02, Steve D'Aprano  wrote:

> > On Fri, 3 Nov 2017 12:39 am, Jon Ribbens wrote:

> > > Why would we want to make the language worse? It is fairly
> > > obvious
> > > what 'else' means,

> >
> > Yes, obvious and WRONG.

>
> Nope, obvious and right.
>


I suppose that to continue this way we'd need at some point define 
somehow the meaning of "obvious."





> > > whereas 'then' has an obvious meaning that is in
> > > fact the opposite of what it would actually do.

> >
> > Er... is today opposite day? Because 'then' describes precisely
> > what it
> > actually does.

>
> No, 'then' describes the opposite of what it does. The word 'then'
> implies something that always happens next, whereas 'else' conveys
> the correct meaning, which is something that happens if the course
> of the preceding piece of code did not go as expected.
>


Jon, i get from this that for you, when there is no exception in `try`, 
or no `break` in a loop, the things did not go as expected.  Either we 
need to agree that what is "expected" is subjective, or agree on some 
kind of formal or informal common meaning for it, because i would not 
have put it this way.


'Then' describes what happens next indeed, unless some extraordinary 
situation prevents it from happening, for example:


   try:
   go_to_the_bakery()
   then:
   buy_croissants(2)
   except BakeryClosed:
   go_to_the_grociery()
   buy_baguette(1)
   finally:
   come_back()

I know this is a poor program example (why not to use a boolean return 
value instead of an exception, etc.), and i know that currently in 
Python `except` must precede `else`, it is just to illustrate the 
choice of terms.


Best regards,

Alexey.

--
https://mail.python.org/mailman/listinfo/python-list

[issue20171] Derby #2: Convert 115 sites to Argument Clinic in Modules/_cursesmodule.c

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

PR 4251 converts most of functions in the _curses and _curses_panel modules 
(around 130 functions) to Argument Clinic. Many functions didn't have 
docstrings, the PR adds them. You can test new signatures and docstrings by the 
following commands:

./python -m pydoc curses
./python -m pydoc curses.window
./python -m pydoc curses.panel
./python -m pydoc curses.panel.panel

3 functions can't be converted. The initial versions of Argument Clinic 
supported such declarations, but now it doesn't support them. Will try to fix 
Argument Clinic in other issue. Yet 35 generated window methods have not been 
converted because I'm too tired by this patch. They will be converted by other 
patches.

I tried to make new and existing docstrings short and clear, but I think many 
docstrings still need editing.

--
versions: +Python 3.7 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18835] Add PyMem_AlignedAlloc()

[Stefan Krah]

Stefan Krah  added the comment:

> I'm not sure that the cost of the memory allocator itself defeats the gain of 
> aligned memory on algorithms. I expect data processing to be much more 
> expensive than the memory allocation, no?

I guess this issue isn't easy to focus due to the vast variety of use cases. So 
the is only about numpy/ndtypes:

What you write is true, but I'm simply getting cold feet w.r.t locking
myself into memset(). calloc() uses mmap() for large allocations, so I think 
one can happily allocate a large number of huge arrays without any
cost on Linux, as long as they're not accessed.

At least that's what my tests indicate.


Couple that with the fact that one has to use aligned_free() anyway,
and that posix_memalign() isn't that great, and the use case seems
less solid **for scientific computing**.


So I rather waste a couple of bytes per allocation and deal with
some Valgrind macros to get proper bounds checking.


Note that CPython still knows any allocation from ndtypes, because
ndt_callocfunc will be set to PyMem_Calloc() [1] and the custom
ndt_aligned_calloc() uses ndt_callocfunc.


[1] If #31912 is solved.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Thread safety issue (I think) with defaultdict

[Rhodri James]

On 02/11/17 20:24, Chris Angelico wrote:

Thank you. I've had this argument with many people, smart people (like
Steven), people who haven't grokked that all concurrency has costs -
that threads aren't magically more dangerous than other options.


I'm with Steven.  To be fair, the danger with threads is that most 
people don't understand thread-safety, and in particular don't 
understand either that they have a responsibility to ensure that shared 
data access is done properly or what the cost of that is.  I've seen far 
too much thread-based code over the years that would have been markedly 
less buggy and not much slower if it had been written sequentially.


--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Jon Ribbens]

On 2017-11-03, Steve D'Aprano  wrote:
> On Fri, 3 Nov 2017 03:31 am, Jon Ribbens wrote:
>> No, it's an obvious bug. You have a 'for...else' with no 'break'.
>> Like I said, that should probably be a syntax error.
>
> It should absolutely not be a syntax error. There's no reason for it
> to be a syntax error, except to satisfy some arrogant and foolish
> idea of purity.

It'd be nice if you could be a little less rude. It's not an "arrogant
and foolish idea of purity", it's to help people catch bugs in their
code, and to aid their understanding of the language.

> There are uncountable ways of writing code which is seemingly
> "pointless", and we don't make it a syntax error.

And there's uncountable ways of writing code which we *do* make a
syntax error. I'm not sure what your point is there.

>> No, 'then' describes the opposite of what it does. The word 'then'
>> implies something that always happens next, 
>
> Right, which is what happens with the for...else block.

No. Ok, so look. It's obvious that you and I have different mental
models of the situation here. You're thinking of 'for...else' as two
arbitrary clauses that run consecutively unless the whole thing is
aborted by a 'break', whereas I'm thinking of the 'for' clause as
being a search for a situation that matches a condition and the
'else' clause being what happens if the condition is not matched
(i.e. exactly the same as 'if...else').

Now there's nothing inherently *wrong* with your choice of mental
model, except it's leading you into confusion because my model means
the meaning of the 'else' keyword is intuitive and obvious, and yours
means it's counter-intuitive and confusing. Your suggestion is that
the fix is to change the language, my suggestion is to fix your model.
I'd suggest that changing your mind is easier than changing the
language ;-)

Also, my model has the advantage that if what the 'for' clause is
doing doesn't match the concept of 'searching for a match' then it's
obvious that you shouldn't be using 'for...else' in the first place.
Again, sure, the language doesn't currently strictly enforce the
requirement for a 'break', but nevertheless if you don't have one
you almost certainly have a bug.

>> Yes, I saw that. It's possible you are the only person in the world
>> ever to have done that. It would not make the interactive interpreter
>> 'worse' in the slightest for that silly trick to be lost.
>
> Just because you personally haven't used this technique doesn't make it
> a "silly trick".

It's an incredibly obscure work-around for a different problem,
i.e. an infelicity in the way the interactive prompt parses input
blocks. If the parsing is a genuine issue then the answer is to
fix that, not to look for hacks that almost never help anyway.

> And while I thank you for the complement that I am the cleverest and most
> insightful Python coder in the world,

I didn't say anything even remotely resembling that.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue21423] concurrent.futures.ThreadPoolExecutor/ProcessPoolExecutor should accept an initializer argument

[Antoine Pitrou]

Antoine Pitrou  added the comment:

By the way, the submitted PR follows Dan's argument about the initializer's 
argument: the actual call is `initializer(*initargs)`. If someone wants to know 
about the current thread or process, it's trivial to call 
`thread.current_thread()` or `multiprocessing.current_process()` (and I don't 
it's a bad idiom in itself :-)).

If you want to comment on the PR, I would recommend doing it quick, as I plan 
to merge in a day or two :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31890] Please define the flag METH_STACKLESS for Stackless Python

[STINNER Victor]

STINNER Victor  added the comment:

Anselm Kruis: Python 3.7 now has your METH_STACKLESS flag. Enjoy ;-)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31931] test_concurrent_futures: ProcessPoolSpawnExecutorTest.test_shutdown_race_issue12456() leaked dangling process on x86 Tiger 3.x

[STINNER Victor]

New submission from STINNER Victor :

http://buildbot.python.org/all/#/builders/30/builds/109/steps/4/logs/stdio

0:24:58 load avg: 2.01 [274/407/1] test_concurrent_futures failed (env changed) 
(279 sec)
(...)
test_shutdown_race_issue12456 
(test.test_concurrent_futures.ProcessPoolSpawnExecutorTest) ... 7.42s
Warning -- reap_children() reaped child process 3411
ok
(...)
Warning -- multiprocessing.process._dangling was modified by 
test_concurrent_futures
  Before: <_weakrefset.WeakSet object at 0x269fd54>
  After:  <_weakrefset.WeakSet object at 0x26a64cc>

--
components: Tests
keywords: buildbot
messages: 305468
nosy: haypo
priority: normal
severity: normal
status: open
title: test_concurrent_futures: 
ProcessPoolSpawnExecutorTest.test_shutdown_race_issue12456() leaked dangling 
process on x86 Tiger 3.x
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31911] Use malloc_usable_size() in pymalloc for realloc

[STINNER Victor]

Change by STINNER Victor :


--
title: Use malloc_usable_size() is pymalloc for realloc -> Use 
malloc_usable_size() in pymalloc for realloc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31308] forkserver process isn't re-launched if it died

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Davin, I think I'm going to merge the PR for this.  If you object it, it can 
still be reverted later.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31933] some Blake2 parameters are encoded backwards on big-endian platforms

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
pull_requests: +4211
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31308] forkserver process isn't re-launched if it died

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset fc6b348b12ad401cab0261b7b71a65c60a08c0a8 by Antoine Pitrou in 
branch 'master':
bpo-31308: If multiprocessing's forkserver dies, launch it again when necessary 
(#3246)
https://github.com/python/cpython/commit/fc6b348b12ad401cab0261b7b71a65c60a08c0a8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31308] forkserver process isn't re-launched if it died

[Antoine Pitrou]

Change by Antoine Pitrou :


--
keywords: +patch
pull_requests: +4214

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31931] test_concurrent_futures: ProcessPoolSpawnExecutorTest.test_shutdown_race_issue12456() leaked dangling process on x86 Tiger 3.x

[STINNER Victor]

STINNER Victor  added the comment:

More context:

test_correct_timeout_exception_msg 
(test.test_concurrent_futures.ProcessPoolSpawnAsCompletedTests) ... 2.66s ok
test_duplicate_futures 
(test.test_concurrent_futures.ProcessPoolSpawnAsCompletedTests) ... 4.64s ok
test_free_reference_yielded_future 
(test.test_concurrent_futures.ProcessPoolSpawnAsCompletedTests) ... 2.69s ok
test_no_timeout (test.test_concurrent_futures.ProcessPoolSpawnAsCompletedTests) 
... 2.80s ok
test_zero_timeout 
(test.test_concurrent_futures.ProcessPoolSpawnAsCompletedTests) ... 4.72s ok
test_free_reference (test.test_concurrent_futures.ProcessPoolSpawnExecutorTest) 
... 9.49s ok
test_killed_child (test.test_concurrent_futures.ProcessPoolSpawnExecutorTest) 
... 2.44s ok
test_map (test.test_concurrent_futures.ProcessPoolSpawnExecutorTest) ... 2.71s 
ok
test_map_chunksize (test.test_concurrent_futures.ProcessPoolSpawnExecutorTest) 
... 2.69s ok
test_map_exception (test.test_concurrent_futures.ProcessPoolSpawnExecutorTest) 
... 2.65s ok
test_map_timeout (test.test_concurrent_futures.ProcessPoolSpawnExecutorTest) 
... 8.65s ok
test_max_workers_negative 
(test.test_concurrent_futures.ProcessPoolSpawnExecutorTest) ... 2.64s ok
test_no_stale_references 
(test.test_concurrent_futures.ProcessPoolSpawnExecutorTest) ... 2.66s ok
test_ressources_gced_in_workers 
(test.test_concurrent_futures.ProcessPoolSpawnExecutorTest) ... 4.75s ok
test_shutdown_race_issue12456 
(test.test_concurrent_futures.ProcessPoolSpawnExecutorTest) ... 7.42s Warning 
-- reap_children() reaped child process 3411
ok
test_submit (test.test_concurrent_futures.ProcessPoolSpawnExecutorTest) ... 
2.97s ok
test_submit_keyword (test.test_concurrent_futures.ProcessPoolSpawnExecutorTest) 
... 3.03s ok
test_traceback (test.test_concurrent_futures.ProcessPoolSpawnExecutorTest) ... 
2.92s ok
test_context_manager_shutdown 
(test.test_concurrent_futures.ProcessPoolSpawnShutdownTest) ... 2.95s ok
test_del_shutdown (test.test_concurrent_futures.ProcessPoolSpawnShutdownTest) 
... 2.74s ok
test_hang_issue12364 
(test.test_concurrent_futures.ProcessPoolSpawnShutdownTest) ... 3.65s ok
test_interpreter_shutdown 
(test.test_concurrent_futures.ProcessPoolSpawnShutdownTest) ... 7.00s ok

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31933] some Blake2 parameters are encoded backwards on big-endian platforms

[Jack O'Connor]

New submission from Jack O'Connor :

See https://github.com/BLAKE2/libb2/issues/12.

All Blake2 params have to be encoded in little-endian byte order. For the two 
multi-byte integer params, leaf_length and node_offset, that means that 
assigning a native-endian integer to them appears to work on little-endian 
platforms, but gives the wrong result on big-endian. The current libb2 API 
doesn't make that very clear, and @sneves is working on new API functions in 
the GH issue above. In the meantime, we can work around the problem by 
explicitly assigning little-endian values to the parameter block.

--
messages: 305473
nosy: oconnor663
priority: normal
severity: normal
status: open
title: some Blake2 parameters are encoded backwards on big-endian platforms
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: A use-case for for...else with no break

[Steve D'Aprano]

On Fri, 3 Nov 2017 04:22 pm, Paul Rubin wrote:

> Steve D'Aprano  writes:
>> for x in something():
>> print(x, end='')
> 
> print(''.join(something()))


I hoped that people would recognise a simplified, toy example used only to
illustrate a technique, rather than an exact copy and paste of something I'm
actually doing.

Try this instead:

for s in sequence:
if isinstance(s, bytes):
try:
 s = s.decode.('utf-8')
except UnicodeDecodeError:
 s = s.decode.('latin-1')
print(s, end='')
else:
print()


Without writing a helper function, got a fancy one liner for that too?

Or another example, sometimes you really don't want to wait until the entire
sequence is processed before printing any output.

count = 0
for obj in sequence:
if count == 60:
print(' ', time.asctime())
count = 0
print('.', end='')
expensive_time_consuming_process_that_might_take_minutes_or_hours(obj)
count += 1
else:
del count
print()
print("finished at", time.asctime())


Now honestly, do you think the more complex examples illustrate the point I
was making, and the usefulness of for...else, better than the simple version?

Because I think they make them more complicated and hard to understand, and
distract from the point I am making.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31933] some Blake2 parameters are encoded backwards on big-endian platforms

[Christian Heimes]

Christian Heimes  added the comment:

Good work, thanks for your PR!

--
assignee:  -> christian.heimes
components: +Extension Modules
nosy: +christian.heimes
versions:  -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20171] Derby #2: Convert 115 sites to Argument Clinic in Modules/_cursesmodule.c

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +4213
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: replacing `else` with `then` in `for` and `try`

[Chris Angelico]

On Fri, Nov 3, 2017 at 8:48 PM, Alexey Muranov  wrote:
> 'Then' describes what happens next indeed, unless some extraordinary
> situation prevents it from happening, for example:
>
>try:
>go_to_the_bakery()
>then:
>buy_croissants(2)
>except BakeryClosed:
>go_to_the_grociery()
>buy_baguette(1)
>finally:
>come_back()
>
> I know this is a poor program example (why not to use a boolean return value
> instead of an exception, etc.), and i know that currently in Python `except`
> must precede `else`, it is just to illustrate the choice of terms.

What is the semantic difference between that code and the same without
the "then:"? The normal behaviour of both Python code and human
instruction sheets is to proceed to the next instruction if nothing
went wrong:

1. Go to the office.
2. Sit down at the desk.
3. Open the third drawer down at your right hand.
4. Take the secret instructions.
5. Read and follow the instructions.
-- if something goes wrong, take out your pistol and kill every zombie
you find --

If there's no desk in the office, you won't continue to the next
steps. But for the normal case, you don't have to say "then".

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31817] Compilation Error with Python 3.6.1/3.6.3 with Tkinter

[Ned Deily]

Ned Deily  added the comment:

Josh, sorry I don't have any further ideas off the top of my head and I likely 
won't have time for several weeks to try to reproduce this myself.  Have you 
tried renaming the _tkinter extension back, doing a "make install", and seeing 
whether it imports properly from the installed python?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31926] compile error when converting selectmodule to AC due to missing #define-s

[STINNER Victor]

STINNER Victor  added the comment:


New changeset f8b3f6b178e48773cd7298141cbaf408c6917e41 by Victor Stinner (Tal 
Einat) in branch '3.6':
[3.6] bpo-31926: fix missing *_METHODDEF statements by argument clinic 
(GH-4230) (#4253)
https://github.com/python/cpython/commit/f8b3f6b178e48773cd7298141cbaf408c6917e41


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20182] Derby #13: Convert 50 sites to Argument Clinic across 5 files

[STINNER Victor]

STINNER Victor  added the comment:

I really hate the title of these issues "Derby #13: Convert 50 sites to 
Argument Clinic across 5 files". I would prefer to have one issue per file. 
Would it possible to close this one and open a new issue once someone has a PR 
for one file?

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31935] subprocess.run() timeout not working with grandchildren and stdout=PIPE

[Martin Ritter]

New submission from Martin Ritter :

Hi,

I tried to use 

subprocess.run(..., stdout=subprocess.PIPE, timeout=N)

to run some test scripts with a given timeout. 

This works as expected with simple scripts. However if the script itself 
creates other children which write to stdout then `subprocess.run()` seems to 
wait for all of the children to finish. I've attached a minimal example.

I looked into subprocess.py and `subprocess.run()` calls 
`process.communicate()` again without timeout when handling the TimeoutExpired 
exception which then in turn waits for the pipes to be closed by all children.

If communicate() would check if the process is still alive while waiting for 
output and close the pipes once the process has finished the timeout feature 
should work as expected and descendants would get a SIGPIPE when writing to 
stdout/stderr.

--
components: Library (Lib)
files: test_killsub.py
messages: 305487
nosy: Martin Ritter
priority: normal
severity: normal
status: open
title: subprocess.run() timeout not working with grandchildren and stdout=PIPE
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file47251/test_killsub.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31920] pygettext ignores directories as inputfile argument

[Oleg Krasnikov]

Oleg Krasnikov  added the comment:

Thanks for quite sensible notes Serhiy. I've fixed all that in recent commit 
and added a regression test. Still not sure about "testing conventions" here 
cause this is my first PR to python repository, so please let me know if 
something is wrong and I'll fix that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31920] pygettext ignores directories as inputfile argument

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Thank you for your patch Oleg. 3.4 and 3.5 are in security fixes only mode now.

Good catch, the usage of os.walk() in pygettext.py is incorrect. But your 
change is not enough. In _visit_pyfiles() the name 'CVS' is removed from the 
names list. If names is a list of directories emitted by os.walk(), this would 
exclude the whole directory CVS from searching. But if it is a new list "dirs + 
files", this doesn't have any effect. And directories with the ".py" extension 
shouldn't be added to the list of Python files. Hence _visit_pyfiles should 
take two separate lists for directories and files. And since it no longer is a 
callback, it would be better to inline its code.

There are tests for pygettext in Lib/test/test_tools/test_i18n.py. It would be 
nice to add a new test for the fixed feature.

--
nosy: +serhiy.storchaka
stage: patch review -> needs patch
versions: +Python 2.7, Python 3.6, Python 3.7 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31308] forkserver process isn't re-launched if it died

[Antoine Pitrou]

Change by Antoine Pitrou :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31926] compile error when converting selectmodule to AC due to missing #define-s

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31310] semaphore tracker isn't protected against crashes

[Antoine Pitrou]

Change by Antoine Pitrou :


--
keywords: +patch
pull_requests: +4216

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31699] Deadlocks in `concurrent.futures.ProcessPoolExecutor` with pickling error

[Thomas Moreau]

Change by Thomas Moreau :


--
keywords: +patch
pull_requests: +4218
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: replacing `else` with `then` in `for` and `try`

[bartc]

On 03/11/2017 11:49, Jon Ribbens wrote:

On 2017-11-03, Steve D'Aprano  wrote:



Right, which is what happens with the for...else block.


No. Ok, so look. It's obvious that you and I have different mental
models of the situation here. You're thinking of 'for...else' as two
arbitrary clauses that run consecutively unless the whole thing is
aborted by a 'break', whereas I'm thinking of the 'for' clause as
being a search for a situation that matches a condition and the
'else' clause being what happens if the condition is not matched
(i.e. exactly the same as 'if...else').

Now there's nothing inherently *wrong* with your choice of mental
model, except it's leading you into confusion because my model means
the meaning of the 'else' keyword is intuitive and obvious, and yours
means it's counter-intuitive and confusing. Your suggestion is that
the fix is to change the language, my suggestion is to fix your model.
I'd suggest that changing your mind is easier than changing the
language ;-)


I don't think there will be a short keyword that will suit everyone. It 
may be necessary to use this to spell out exactly how it works:


  for i in r:
 pass ...
  after_normal_loop_termination_then:
 pass ...

(And now, there is the possibility of having an additional 'else' clause 
to cover abnormal termination via break. This time what 'else' does is 
more obvious.)


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

[issue31926] compile error when converting selectmodule to AC due to missing #define-s

[Tal Einat]

Change by Tal Einat :


--
pull_requests: +4215

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31310] semaphore tracker isn't protected against crashes

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset cbe1756e3ecefc0e24a5d0a4b8663db9b6d0cc52 by Antoine Pitrou in 
branch 'master':
bpo-31310: multiprocessing's semaphore tracker should be launched again if 
crashed (#3247)
https://github.com/python/cpython/commit/cbe1756e3ecefc0e24a5d0a4b8663db9b6d0cc52


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31310] semaphore tracker isn't protected against crashes

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset b5f09acf0a0219cec32b7eba3acdcb573fc74ab5 by Antoine Pitrou in 
branch '3.6':
[3.6] bpo-31310: multiprocessing's semaphore tracker should be launched again 
if crashed (GH-3247) (#4254)
https://github.com/python/cpython/commit/b5f09acf0a0219cec32b7eba3acdcb573fc74ab5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31310] semaphore tracker isn't protected against crashes

[Antoine Pitrou]

Change by Antoine Pitrou :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

python3 byte decode

[Ali Rıza KELEŞ]

Hi,

Yesterday, while working with redis, i encountered a strange case.

I want to ask why is the following `True`

```
"s" is b"s".decode()
```

while the followings are `False`?

```
"so" is b"so".decode()
"som" is b"som".decode()
"some" is b"some".decode()
```

Or vice versa?

I read that `is` compares same objects, not values. So my question is
why "s" and b"s".decode() are same objects, while the others aren't?

My python version is 3.6.3.

Thanks.

-- 
--
Ali Rıza Keleş
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31932] setup.py cannot find vcversall.bat on MSWin 8.1 if installed in user AppData

[Zachary Ware]

Change by Zachary Ware :


--
components: +Windows
nosy: +paul.moore, steve.dower, tim.golden, zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31308] forkserver process isn't re-launched if it died

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset 019c99f325287741d1e0eefeef2b75c8e00b884f by Antoine Pitrou in 
branch '3.6':
[3.6] bpo-31308: If multiprocessing's forkserver dies, launch it again when 
necessary (GH-3246) (#4252)
https://github.com/python/cpython/commit/019c99f325287741d1e0eefeef2b75c8e00b884f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20182] Derby #13: Convert 50 sites to Argument Clinic across 5 files

[Tal Einat]

Tal Einat  added the comment:

As the author of all of the updated patches, I wouldn't mind opening new issues 
separately for each of the remaining modules. Actually I would prefer it :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Thread safety issue (I think) with defaultdict

[Chris Angelico]

On Fri, Nov 3, 2017 at 10:26 PM, Rhodri James  wrote:
> On 02/11/17 20:24, Chris Angelico wrote:
>>
>> Thank you. I've had this argument with many people, smart people (like
>> Steven), people who haven't grokked that all concurrency has costs -
>> that threads aren't magically more dangerous than other options.
>
>
> I'm with Steven.  To be fair, the danger with threads is that most people
> don't understand thread-safety, and in particular don't understand either
> that they have a responsibility to ensure that shared data access is done
> properly or what the cost of that is.  I've seen far too much thread-based
> code over the years that would have been markedly less buggy and not much
> slower if it had been written sequentially.

Yes, but what you're seeing is that *concurrent* code is more
complicated than *sequential* code. Would the code in question have
been less buggy if it had used multiprocessing instead of
multithreading? What if it used explicit yield points? Multiprocessing
brings with it a whole lot of extra complications around moving data
around. Multithreading brings with it a whole lot of extra
complications around NOT moving data around. Yield points bring with
them the risk of locking another thread out unexpectedly (particularly
since certain system calls aren't async-friendly on certain OSes). All
three models have their pitfalls. It's not that threads are somehow
worse than every other model.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31926] compile error when converting selectmodule to AC due to missing #define-s

[STINNER Victor]

STINNER Victor  added the comment:

Thank you Tal Einat for your bug report and your fix. It's now merged into 3.6 
and master (future 3.7).

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31934] Failure to build out of source from a not clean source

[Xavier de Gaye]

Change by Xavier de Gaye :


--
keywords: +patch
pull_requests: +4217
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: python3 byte decode

[Chris Angelico]

On Fri, Nov 3, 2017 at 8:24 PM, Ali Rıza KELEŞ  wrote:
> Hi,
>
> Yesterday, while working with redis, i encountered a strange case.
>
> I want to ask why is the following `True`
>
> ```
> "s" is b"s".decode()
> ```
>
> while the followings are `False`?
>
> ```
> "so" is b"so".decode()
> "som" is b"som".decode()
> "some" is b"some".decode()
> ```
>
> Or vice versa?
>
> I read that `is` compares same objects, not values. So my question is
> why "s" and b"s".decode() are same objects, while the others aren't?
>
> My python version is 3.6.3.

You shouldn't be comparing string objects with 'is'. Sometimes two
equal strings will be identical, and sometimes they won't. All you're
seeing is that the interpreter happened to notice and/or cache this
particular lookup.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Thread safety issue (I think) with defaultdict

[Grant Edwards]

On 2017-11-03, Chris Angelico  wrote:
> On Sat, Nov 4, 2017 at 2:45 AM, Steve D'Aprano
> wrote:
>> So, all else being equal, which is likely to have more bugs?
>>
>>
>> 1. Multiprocessing code with very little coupling between processes; or
>>
>> 2. Threaded code with shared data and hence higher coupling between threads?
>>
>
> Obviously, option 1. But that's "all else being equal". How often can
> you actually have your processes that decoupled? And if you can write
> your code to be completely (or largely) decoupled, what's to stop you
> having your *threads* equally decoupled? You're assuming that "running
> in the same memoryspace" equates to "higher coupling", which is no
> more proven than any other assertion.

The big difference is that with threaded code you can have accidental
coupling.  With multiprocessing, code you have to explicitly work to
create coupling.

That said, I do a lot of threading coding (in both Python and C), and
have never found it particularly tricky.

It does require that you understand what you're doing and probably
doesn't work well if you're a stack-overflow, cargo-cult,
cut-and-paste programmer.  But then again, what does?

-- 
Grant Edwards   grant.b.edwardsYow! Maybe I should have
  at   asked for my Neutron Bomb
  gmail.comin PAISLEY --

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31936] "5. The import system" grammatical error

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

Re-reading the paragraph now, it also parsed weirdly for me.  See the linked PR 
for a suggested improvement, along with your change.

Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31936] "5. The import system" grammatical error

[Barry A. Warsaw]

Change by Barry A. Warsaw :


--
keywords: +patch
pull_requests: +4219
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Thread safety issue (I think) with defaultdict

[Steve D'Aprano]

On Sat, 4 Nov 2017 01:50 am, Chris Angelico wrote:

> On Fri, Nov 3, 2017 at 10:26 PM, Rhodri James  wrote:

>> I'm with Steven.  To be fair, the danger with threads is that most people
>> don't understand thread-safety, and in particular don't understand either
>> that they have a responsibility to ensure that shared data access is done
>> properly or what the cost of that is.  I've seen far too much thread-based
>> code over the years that would have been markedly less buggy and not much
>> slower if it had been written sequentially.
> 
> Yes, but what you're seeing is that *concurrent* code is more
> complicated than *sequential* code. Would the code in question have
> been less buggy if it had used multiprocessing instead of
> multithreading? 

Maybe.

There's no way to be sure unless you actually compare a threading
implementation with a processing implementation -- and they have to
be "equally good, for the style" implementations. No fair comparing the
multiprocessing equivalent of "Stooge Sort" with the threading equivalent
of "Quick Sort", and concluding that threading is better.

However, we can predict the likelihood of which will be less buggy by
reasoning in general principles. And the general principle is that shared
data tends, all else being equal, to lead to more bugs than no shared data.
The more data is shared, the more bugs, more or less.

I don't know if there are any hard scientific studies on this, but experience
and anecdote strongly suggests it is true. Programming is not yet fully
evidence-based.

For example, most of us accept "global variables considered harmful". With few
exceptions, the use of application-wide global variables to communicate
between functions is harmful and leads to problems. This isn't because of any
sort of mystical or magical malignity from global variables. It is because
the use of global variables adds coupling between otherwise distant parts of
the code, and that adds complexity, and the more complex code is, the more
likely we mere humans are to screw it up.

So, all else being equal, which is likely to have more bugs?


1. Multiprocessing code with very little coupling between processes; or

2. Threaded code with shared data and hence higher coupling between threads?


Obviously the *best* threaded code will have fewer bugs than the *worst*
multiprocessing code, but my heuristic is that, in general, the average
application using threading is likely to be more highly coupled, hence more
complicated, than the equivalent using multiprocessing.

(Async is too new, and to me, too confusing, for me to have an opinion on yet,
but I lean slightly towards the position that deterministic task-switching is
probably better than non-deterministic.)



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Thread safety issue (I think) with defaultdict

[Ian Kelly]

On Thu, Nov 2, 2017 at 10:27 AM, Israel Brewster  wrote:
>
>> On Nov 1, 2017, at 4:53 PM, Steve D'Aprano  
>> wrote:
>>
>> On Thu, 2 Nov 2017 05:53 am, Israel Brewster wrote:
>>
>> [...]
>>> So the end result is that the thread that "updates" the dictionary, and the
>>> thread that initially *populates* the dictionary are actually running in
>>> different processes.
>>
>> If they are in different processes, that would explain why the second
>> (non)thread sees an empty dict even after the first thread has populated it:
>>
>>
>> # from your previous post
>>> Length at get AC:  54 ID: 4524152200  Time: 2017-11-01 09:41:24.474788
>>> Length At update:  1 ID: 4524152200  Time: 2017-11-01 09:41:24.784399
>>> Length At update:  2 ID: 4524152200  Time: 2017-11-01 09:41:25.228853
>>
>>
>> You cannot rely on IDs being unique across different processes. Its an
>> unfortunately coincidence(!) that they end up with the same ID.
>
> I think it's more than a coincidence, given that it is 100% reproducible. 
> Plus, in an earlier debug test I was calling print() on the defaultdict 
> object, which gives output like "", where 
> presumably the 0x1066467f0 is a memory address (correct me if I am wrong in 
> that). In every case, that address was the same. So still a bit puzzling.

If the empty dict is created before the process is forked then I don't
think it's all that surprising.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Thread safety issue (I think) with defaultdict

[Rhodri James]

On 03/11/17 14:50, Chris Angelico wrote:

On Fri, Nov 3, 2017 at 10:26 PM, Rhodri James  wrote:

On 02/11/17 20:24, Chris Angelico wrote:


Thank you. I've had this argument with many people, smart people (like
Steven), people who haven't grokked that all concurrency has costs -
that threads aren't magically more dangerous than other options.



I'm with Steven.  To be fair, the danger with threads is that most people
don't understand thread-safety, and in particular don't understand either
that they have a responsibility to ensure that shared data access is done
properly or what the cost of that is.  I've seen far too much thread-based
code over the years that would have been markedly less buggy and not much
slower if it had been written sequentially.


Yes, but what you're seeing is that *concurrent* code is more
complicated than *sequential* code. Would the code in question have
been less buggy if it had used multiprocessing instead of
multithreading? What if it used explicit yield points?


My experience with situations where I can do a reasonable comparison is 
limited, but the answer appears to be "Yes".

 Multiprocessing

brings with it a whole lot of extra complications around moving data
around.


People generally understand how to move data around, and the mistakes 
are usually pretty obvious when they happen.  People may not understand 
how to move data around efficiently, but that's a separate argument.


 Multithreading brings with it a whole lot of extra

complications around NOT moving data around.


I think this involves more subtle bugs that are harder to spot.  People 
seem to find it harder to reason about atomicity and realising that 
widely separated pieces of code may interact unexpectedly.


 Yield points bring with

them the risk of locking another thread out unexpectedly (particularly
since certain system calls aren't async-friendly on certain OSes).


I've got to admit I find coroutines straightforward, but I did cut my 
teeth on a cooperative OS.  It certainly makes the atomicity issues 
easier to deal with.


 All

three models have their pitfalls.


Assuredly.  I just think threads are soggier and hard to light^W^W^W^W^W 
prone to subtler and more mysterious-looking bugs.


--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Chris Angelico]

On Sat, Nov 4, 2017 at 3:15 AM, Alexey Muranov  wrote:
> On Fri, 2017-11-03 at 22:03 +1100, Chris Angelico wrote:
>> On Fri, Nov 3, 2017 at 8:48 PM, Alexey Muranov > com> wrote:
>> > 'Then' describes what happens next indeed, unless some
>> > extraordinary
>> > situation prevents it from happening, for example:
>> >
>> > try:
>> > go_to_the_bakery()
>> > then:
>> > buy_croissants(2)
>> > except BakeryClosed:
>> > go_to_the_grociery()
>> > buy_baguette(1)
>> > finally:
>> > come_back()
>> >
>> > I know this is a poor program example (why not to use a boolean
>> > return value
>> > instead of an exception, etc.), and i know that currently in Python
>> > `except`
>> > must precede `else`, it is just to illustrate the choice of terms.
>>
>> What is the semantic difference between that code and the same
>> without the "then:"?
>
> Chris,
>
> the semantic difference is that their meanings/behaviours are not identical
> (i imply that `then` here does what `else` currently does).  I agree however
> that from practical viewpoint the difference will probably never be
> observable (unless the person enters the bakery and asks for croissants, but
> during this time the baker exits the bakery and closes it to go on
> vacation).

Okay, so what you're doing is taking the current "try/else" semantics,
and hoisting the "happy path" up to immediately underneath the part
guarded by the try. That's not unreasonable, but also not a huge
advantage. It's basically saying:

try:
do_stuff
except None: # let's pretend
print("all is well")
except ValueError:
print("do_stuff has the wrong value")
finally:
print("we're done")

But if the difference between this and the current layout (with "else"
coming *after* the except blocks) is significant to you, I would
recommend refactoring the try/finally into a context manager:

with car_trip() as trip:
try:
trip.goto(bakery)
buy_croissants(2)
except BakeryClosed:
trip.goto(grocery)
buy_baguette(1)

The context manager takes care of bringing us home unconditionally,
leaving us with cleaner logic.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31932] setup.py cannot find vcversall.bat on MSWin 8.1 if installed in user AppData

[Stefan Krah]

Change by Stefan Krah :


--
nosy: +jyrkih

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Nov 2, 2017, at 22:50, INADA Naoki  wrote:
> 
> When adding environment variable option, it should be documented in
> man page and `python -h`.

I thought about that, but the problem is that none of the -X options are 
documented in the —help output.  So do we only document the ones (e.g. 
importtime) that have an environment variable version, or do we now have to 
document them all?

I opted to document PYTHONPROFILEIMPORTTIME only in the cmdline documentation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: replacing `else` with `then` in `for` and `try`

[Alexey Muranov]

On Fri, 2017-11-03 at 22:03 +1100, Chris Angelico wrote:
> On Fri, Nov 3, 2017 at 8:48 PM, Alexey Muranov  com> wrote:
> > 'Then' describes what happens next indeed, unless some
> > extraordinary
> > situation prevents it from happening, for example:
> >
> > try:
> > go_to_the_bakery()
> > then:
> > buy_croissants(2)
> > except BakeryClosed:
> > go_to_the_grociery()
> > buy_baguette(1)
> > finally:
> > come_back()
> >
> > I know this is a poor program example (why not to use a boolean
> > return value
> > instead of an exception, etc.), and i know that currently in Python
> > `except`
> > must precede `else`, it is just to illustrate the choice of terms.
>
> What is the semantic difference between that code and the same
> without the "then:"?

Chris,

the semantic difference is that their meanings/behaviours are not 
identical (i imply that `then` here does what `else` currently does).  
I agree however that from practical viewpoint the difference will 
probably never be observable (unless the person enters the bakery and 
asks for croissants, but during this time the baker exits the bakery 
and closes it to go on vacation).


I can try to think of a better example if you give me any good use-case 
for `else` in `try`.  I have searched on-line, and on StackOverflow in 
particular, but didn't find anything better that this:


 * https://stackoverflow.com/a/6051978

People seem very shy when it comes to giving a real-life example of 
`else` in `try`.


Alexey.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Thread safety issue (I think) with defaultdict

[Chris Angelico]

On Sat, Nov 4, 2017 at 2:45 AM, Steve D'Aprano
 wrote:
> So, all else being equal, which is likely to have more bugs?
>
>
> 1. Multiprocessing code with very little coupling between processes; or
>
> 2. Threaded code with shared data and hence higher coupling between threads?
>

Obviously, option 1. But that's "all else being equal". How often can
you actually have your processes that decoupled? And if you can write
your code to be completely (or largely) decoupled, what's to stop you
having your *threads* equally decoupled? You're assuming that "running
in the same memoryspace" equates to "higher coupling", which is no
more proven than any other assertion. Ultimately, real-world code IS
going to have some measure of coupling (you could concoct a scenario
in which requests are handled 100% independent of each other, but even
with a web application, there's going to be SOME connection between
different requests), so all you do is move it around (eg in a web app
scenario, the most common solution is to do all coupling through a
database or equivalent).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Jon Ribbens]

On 2017-11-03, Chris Angelico  wrote:
> On Fri, Nov 3, 2017 at 10:49 PM, Jon Ribbens  
> wrote:
>>> It should absolutely not be a syntax error. There's no reason for it
>>> to be a syntax error, except to satisfy some arrogant and foolish
>>> idea of purity.
>>
>> It'd be nice if you could be a little less rude. It's not an "arrogant
>> and foolish idea of purity", it's to help people catch bugs in their
>> code, and to aid their understanding of the language.
>
> That wasn't rudeness.

Yes, it was.

>> No. Ok, so look. It's obvious that you and I have different mental
>> models of the situation here. You're thinking of 'for...else' as two
>> arbitrary clauses that run consecutively unless the whole thing is
>> aborted by a 'break', whereas I'm thinking of the 'for' clause as
>> being a search for a situation that matches a condition and the
>> 'else' clause being what happens if the condition is not matched
>> (i.e. exactly the same as 'if...else').
>>
>> Now there's nothing inherently *wrong* with your choice of mental
>> model, except it's leading you into confusion because my model means
>> the meaning of the 'else' keyword is intuitive and obvious, and yours
>> means it's counter-intuitive and confusing. Your suggestion is that
>> the fix is to change the language, my suggestion is to fix your model.
>> I'd suggest that changing your mind is easier than changing the
>> language ;-)
>
> If anything, I would say that Steven's model is at a lower abstraction
> layer than yours

Yes, absolutely.

> - though IMO your model is more of an example use-case than a
> description of what is actually happening.

It's a high-level way of thinking about it that avoids confusion and
leads to correct code. Unless you can come up with a *sensible*
use-case that it doesn't cover, it's also a comprehensive way of
thinking about it.

> TBH I prefer the "if 1:" trick to gather code into a block. But that
> requires pre-planning,

Or pressing up-arrow ;-)

> whereas slapping an "else:" after the loop can be done after the event.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Steve D'Aprano]

On Fri, 3 Nov 2017 10:49 pm, Jon Ribbens wrote:

> On 2017-11-03, Steve D'Aprano  wrote:
>> On Fri, 3 Nov 2017 03:31 am, Jon Ribbens wrote:
>>> No, it's an obvious bug. You have a 'for...else' with no 'break'.
>>> Like I said, that should probably be a syntax error.
>>
>> It should absolutely not be a syntax error. There's no reason for it
>> to be a syntax error, except to satisfy some arrogant and foolish
>> idea of purity.
> 
> It'd be nice if you could be a little less rude. 

How else should we respond to the idea that something that has been legal
syntax for over twenty years and is in no way an error should be treated as a
syntax error, just to satisfy the opinion that nobody should write for...else
unless they are performing a search and it contains a break?


>> There are uncountable ways of writing code which is seemingly
>> "pointless", and we don't make it a syntax error.
> 
> And there's uncountable ways of writing code which we *do* make a
> syntax error. I'm not sure what your point is there.

None of these are syntax errors:

if False: do_something()

for x in '': pass

if True: pass

and uncountable other pieces of code which are, from some perspective,
pointless. We don't make them syntax errors because *pointless* doesn't imply
is is an error.

You think that a for...else loop with no break is pointless and "silly", even
when I demonstrate an actual use for it. Too bad. Even if it is "silly", in
your opinion, that still does not make it an error.

Is my point clear now?


>>> No, 'then' describes the opposite of what it does. The word 'then'
>>> implies something that always happens next,
>>
>> Right, which is what happens with the for...else block.
> 
> No.

Yes. It is not a matter of opinion that the `else` clause executes after
execution falls out the bottom of the loop, i.e. the loop runs, THEN the
`else` block runs.

It is an objective fact that in the absence of something which causes
execution to jump outside of the loop altogether (a return, a raise, or a
break) the execution order is loop first, then `else` block.

Both of these two sentences makes sense in English:

"Execution reaches the end of the loop, and THEN the `else` block runs."

"Execution reaches the end of the loop, or ELSE the `else` block runs."

but only the first describes what Python's for...else statement actually does.

This is not just my mental model, it is the actual objective behaviour of the
for...else statement.


> Ok, so look. It's obvious that you and I have different mental 
> models of the situation here. You're thinking of 'for...else' as two
> arbitrary clauses that run consecutively 

Scrub out the "arbitrary", and that is exactly what they are. They aren't
arbitrary: the `else` is optional, but if present it must follow after the
loop, and never the other way around.

The code inside the blocks can, of course, be any legal code we like. There's
no rule that says "only code approved by Jon Ribbens' is allowed", and
there's no rule that says "only searches are allowed".


> unless the whole thing is aborted by a 'break', 

Or return, raise, os._exit, os.abort, or having the process killed by the OS.
For simplicity we can ignore the last three (as well as more exotic
mechanisms such as "pull the power supply out" or "drop a two-tonne weight on
the computer"), but return and raise are common ways of exiting a loop and
should not be ignored or forgotten.


> whereas I'm thinking of the 'for' clause as 
> being a search for a situation that matches a condition and the
> 'else' clause being what happens if the condition is not matched
> (i.e. exactly the same as 'if...else').

That model conflates one specific use-case for the for...else statement with
the semantics of the statement, rather like insisting that "print displays a
greeting to the world" merely because 'print("Hello, World")' is so common.

The for loop does not necessarily perform a search:


count = 1
for obj in sequence:
if count > MAX_OBJECTS:
print("too many objects, halting")
break
process(obj)
else:
print("finished")


According to your mental model, this code is... what? Illegal? Silly?
Impossible? A syntax error?

Obviously wrong and almost certainly buggy?

Even when there is a search, the sense of the search may be reversed from your
mental model:


for request in sequence:
if 'please' not in request:
break
process(request)
else:
print('done')
return
raise SysExit('failure to say please')


It doesn't matter if you can think of alternative ways of writing these code
snippets. They're legal, and they work, and your mental model doesn't cover
them, at all or easily. That means your mental model is at best incomplete.

This is what happens when you treat one use-case for a general purpose
statement as synonymous with the statement itself. `for...else` is not just
for searching, and the `else` clause doesn't necessarily run if the search is

Re: Thread safety issue (I think) with defaultdict

[Gene Heskett]

On Friday 03 November 2017 10:50:13 Chris Angelico wrote:

> On Fri, Nov 3, 2017 at 10:26 PM, Rhodri James  
wrote:
> > On 02/11/17 20:24, Chris Angelico wrote:
> >> Thank you. I've had this argument with many people, smart people
> >> (like Steven), people who haven't grokked that all concurrency has
> >> costs - that threads aren't magically more dangerous than other
> >> options.
> >
> > I'm with Steven.  To be fair, the danger with threads is that most
> > people don't understand thread-safety, and in particular don't
> > understand either that they have a responsibility to ensure that
> > shared data access is done properly or what the cost of that is. 
> > I've seen far too much thread-based code over the years that would
> > have been markedly less buggy and not much slower if it had been
> > written sequentially.
>
> Yes, but what you're seeing is that *concurrent* code is more
> complicated than *sequential* code. Would the code in question have
> been less buggy if it had used multiprocessing instead of
> multithreading? What if it used explicit yield points? Multiprocessing
> brings with it a whole lot of extra complications around moving data
> around. Multithreading brings with it a whole lot of extra
> complications around NOT moving data around. Yield points bring with
> them the risk of locking another thread out unexpectedly (particularly
> since certain system calls aren't async-friendly on certain OSes). All
> three models have their pitfalls. It's not that threads are somehow
> worse than every other model.
>
> ChrisA

I think that this discussion of threads and threading must be a different 
context than threading as I am using it in linuxcnc.

There, one assigns a function to run in a certain sequence in an assigned 
thread, which there can be several of. There, each thread is assigned a 
repetition rate, and the higher repetition rate stuff can always 
interrupt the slower threaded function in order to get the real time 
stuff done in a usable for the job time frame, and the high frequency 
thread can be as fast as every 25 microseconds on a good motherboard.  
Surprisingly, this seems to be far more board dependent than processor 
dependent, altho its pushing an AMD processor quite hard at 40 
microseconds, while the slower intel atoms can do 25 microseconds with 
about the same effort.

This is where a stepper motor is being stepped by software which didles 
pins on a parport. And it limits how fast you can move the motor 
compared to using an fpga card running at 200 MHz, not because of the 
step rate, but because of the latency of the mobo/cpu combination.

Jitter in step rate issuance is death to high performance with stepper 
motors because torque to do usefull work vanishes when the instant speed 
of the motor is wobbling with that timing jitter.

OTOH, hand controls of a machine using an encoder dial are nicely done in 
a thread running at 100 Hz, far more reliably that I can do it from the 
keyboard on a raspberry pi 3b. Why? The dials data goes into linuxcnc by 
way of a hardware fpga card that talks to the pi over an SPI buss, with 
the pi writing 32 bit packets at 41 megabaud, and reading the results at 
25 megabaud. It doesn't have to get thru that usb2 internal hub in the 
pi that all the other i/o has to go thru. Mouse and keyboard events get 
dropped on the floor, particularly dangerous when its a keyup event that 
gets dropped. The machine keeps moving until it crashes into something, 
often breaking drive parts or cutting tooling, all of which cost real 
money.

My point is that with an interpretor such as hal managing things, threads 
Just Work(TM).

It does of course take a specially built kernel to do that magic.
I'll get me coat.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue28706] msvc9compiler does not find a vcvarsall.bat of Visual C++ for Python 9.0

[Stefan Krah]

Change by Stefan Krah :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> setup.py cannot find vcversall.bat on MSWin 8.1 if installed in 
user AppData

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31936] "5. The import system" grammatical error

[Barry A. Warsaw]

Change by Barry A. Warsaw :


--
nosy: +barry
versions: +Python 3.7 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31936] "5. The import system" grammatical error

[Daniel U. Thibault]

New submission from Daniel U. Thibault :

https://docs.python.org/3/reference/import.html#importsystem
"Other mechanisms for invoking the import system (such as 
importlib.import_module()) may choose to subvert __import__() and use its own 
solution to implement import semantics."
should be
"Other mechanisms for invoking the import system (such as 
importlib.import_module()) may choose to subvert __import__() and use their own 
solution to implement import semantics."

--
assignee: docs@python
components: Documentation
messages: 305489
nosy: Daniel.U..Thibault, docs@python
priority: normal
severity: normal
status: open
title: "5. The import system" grammatical error
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

> I didn't think it's worth enough because import will be much slower than one 
> dict lookup.

I agree. The main value of my patch is handling possible (but very unlike) 
errors. Implementing negative value caching adds not many lines to the existing 
code and additional error handling.

But now, after adding the environment variable, do we still need the -X option? 
From a user side I don't see much difference between specifying an option and 
an environment variable, but the code for handling the environment variable is 
much simpler.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30057] signal.signal should check tripped signals

[Antoine Pitrou]

Change by Antoine Pitrou :


--
pull_requests: +4220
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31898] Add a `recommended-packages.txt` file

[Éric Araujo]

Change by Éric Araujo :


--
nosy: +eric.araujo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31924] Fix test_curses on NetBSD 8

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset a7723d8b09f516a2b75837a3527b8cc7bee89fad by Serhiy Storchaka in 
branch 'master':
bpo-31924: Fix test_curses on NetBSD 8. (#4228)
https://github.com/python/cpython/commit/a7723d8b09f516a2b75837a3527b8cc7bee89fad


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31924] Fix test_curses on NetBSD 8

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4222

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31924] Fix test_curses on NetBSD 8

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4221

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30057] signal.signal should check tripped signals

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset ea80ae04e2ec68c7e289048d3224a24b3c3fb107 by Antoine Pitrou (Miss 
Islington (bot)) in branch '3.6':
bpo-30057: Fix potential missed signal in signal.signal(). (GH-4258) (#4261)
https://github.com/python/cpython/commit/ea80ae04e2ec68c7e289048d3224a24b3c3fb107


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30057] signal.signal should check tripped signals

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4223

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30057] signal.signal should check tripped signals

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset f6f90ff079a22b79a58d47b6117cc8a8c7d366f3 by Antoine Pitrou in 
branch 'master':
bpo-30057: Fix potential missed signal in signal.signal(). (#4258)
https://github.com/python/cpython/commit/f6f90ff079a22b79a58d47b6117cc8a8c7d366f3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31933] some Blake2 parameters are encoded backwards on big-endian platforms

[Christian Heimes]

Christian Heimes  added the comment:


New changeset dcfb0e3c04f1b29a0d09bb0a81dcd5ee5a5fef1a by Christian Heimes 
(Jack O'Connor) in branch 'master':
bpo-31933: fix blake2 multi-byte params on big endian platforms (#4250)
https://github.com/python/cpython/commit/dcfb0e3c04f1b29a0d09bb0a81dcd5ee5a5fef1a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30057] signal.signal should check tripped signals

[Antoine Pitrou]

Change by Antoine Pitrou :


--
pull_requests: +4225

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Thread safety issue (I think) with defaultdict

[Rhodri James]

On 03/11/17 18:12, Israel Brewster wrote:

On Nov 3, 2017, at 7:11 AM, Rhodri James  wrote:

People generally understand how to move data around, and the mistakes are 
usually pretty obvious when they happen.


I think the existence of this thread indicates otherwise :-) This mistake was 
far from obvious, and clearly I didn't understand properly how to move data 
around *between processes*. Unless you are just saying I am ignorant or 
something? :-)


Ah, but from the point of view of this argument, you didn't make a 
mistake, you made a meta-mistake.  It wasn't that you didn't understand 
how to move data around between processes, it was that you didn't think 
you were moving between processes!  Whether or not you do understand 
remains to be seen :-)


--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Jon Ribbens]

On 2017-11-03, Steve D'Aprano  wrote:
> The for loop does not necessarily perform a search:
>
> count = 1
> for obj in sequence:
> if count > MAX_OBJECTS:
> print("too many objects, halting")
> break
> process(obj)
> else:
> print("finished")
>
> According to your mental model, this code is... what? Illegal? Silly?
> Impossible? A syntax error?

That conforms to my model. It's searching for the condition
'count > MAX_OBJECTS'.

> Even when there is a search, the sense of the search may be reversed
> from your mental model:
>
> for request in sequence:
> if 'please' not in request:
> break
> process(request)
> else:
> print('done')
> return
> raise SysExit('failure to say please')

That's searching for the condition "'please' not in request".

> It doesn't matter if you can think of alternative ways of writing these code
> snippets. They're legal, and they work, and your mental model doesn't cover
> them, at all or easily. That means your mental model is at best incomplete.

You've yet to come up with one that it doesn't cover.

> In other words, it was only when I realised that `else` would be better
> written as `then` that I understood the behaviour of the statement. If I read
> it as "else", then the natural interpretation is that the block runs only if
> the loop doesn't run:

You're repeating yourself.

> You'll probably dismiss them as "silly" because they don't meet your mental
> model, and insist that I would be wrong to use them.

As above, you're mistaken.

> Your response to code that doesn't match your mental model is to say
> that it is obviously wrong and probably buggy and should be made
> into a syntax error if possible.

No, it isn't. Try reading again what I actually wrote.

> And my mental model is to treat "else" in this concept as some foreign word,
> perhaps Dutch, a false-friend that actually means "next" but due to some
> awful coincidence happens to look exactly like the English word for "else".

And that's leading you into confusion, as you've demonstrated.

> Ah yes, because it is inconceivable that anyone might have thought of a use
> for for...else without a break.

It's not inconceivable, but nobody has thought of a sensible use so far
(by which I mean one that shows it's a useful feature).

> What matters is that only code that matches your model is allowed.

You do seem to enjoy telling other people what their opinions are.
Frankly, I don't know why I even bother having opinions when you're
here to tell me what they are. In future, I'll just come to you to
find out what I think.

>> It's an incredibly obscure work-around for a different problem,
>
> You mean a different problem to "searching"? Yes indeed it is.

No, I mean a problem that has nothing to do with 'for...else' clauses.
If your door is stuck then you might climb in through a window, but
that doesn't mean that the purpose of windows is to be entrances,
or that windows should be designed to make them better entrances,
or that the problem isn't the stuck door.

>>> And while I thank you for the complement that I am the cleverest and most
>>> insightful Python coder in the world,
>> 
>> I didn't say anything even remotely resembling that.
>
> That was sarcasm.

Yes dear.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Ian Kelly]

On Fri, Nov 3, 2017 at 8:32 AM, Chris Angelico  wrote:
> On Fri, Nov 3, 2017 at 10:49 PM, Jon Ribbens  
> wrote:
>> On 2017-11-03, Steve D'Aprano  wrote:
>>> On Fri, 3 Nov 2017 03:31 am, Jon Ribbens wrote:
 No, it's an obvious bug. You have a 'for...else' with no 'break'.
 Like I said, that should probably be a syntax error.
>>>
>>> It should absolutely not be a syntax error. There's no reason for it
>>> to be a syntax error, except to satisfy some arrogant and foolish
>>> idea of purity.
>>
>> It'd be nice if you could be a little less rude. It's not an "arrogant
>> and foolish idea of purity", it's to help people catch bugs in their
>> code, and to aid their understanding of the language.
>
> That wasn't rudeness. That was an accurate description of the
> situation. It is entirely possible for code to be opinionated,
> arrogant, foolish, and all those other adjectives that we normally
> appropriate to people.

Steve was clearly referring to the coder, not the code. Please stop
defending the use of incivility on this list.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue30057] signal.signal should check tripped signals

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Thanks for the report! I will push your patch soon.

--
versions: +Python 3.6, Python 3.7 -Python 2.7, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Thread safety issue (I think) with defaultdict

[Israel Brewster]

---
Israel Brewster
Systems Analyst II
Ravn Alaska
5245 Airport Industrial Rd
Fairbanks, AK 99709
(907) 450-7293
---




> On Nov 3, 2017, at 7:11 AM, Rhodri James  wrote:
> 
> On 03/11/17 14:50, Chris Angelico wrote:
>> On Fri, Nov 3, 2017 at 10:26 PM, Rhodri James  wrote:
>>> On 02/11/17 20:24, Chris Angelico wrote:
 
 Thank you. I've had this argument with many people, smart people (like
 Steven), people who haven't grokked that all concurrency has costs -
 that threads aren't magically more dangerous than other options.
>>> 
>>> 
>>> I'm with Steven.  To be fair, the danger with threads is that most people
>>> don't understand thread-safety, and in particular don't understand either
>>> that they have a responsibility to ensure that shared data access is done
>>> properly or what the cost of that is.  I've seen far too much thread-based
>>> code over the years that would have been markedly less buggy and not much
>>> slower if it had been written sequentially.
>> Yes, but what you're seeing is that *concurrent* code is more
>> complicated than *sequential* code. Would the code in question have
>> been less buggy if it had used multiprocessing instead of
>> multithreading? What if it used explicit yield points?
> 
> My experience with situations where I can do a reasonable comparison is 
> limited, but the answer appears to be "Yes".
> Multiprocessing
>> brings with it a whole lot of extra complications around moving data
>> around.
> 
> People generally understand how to move data around, and the mistakes are 
> usually pretty obvious when they happen.  

I think the existence of this thread indicates otherwise :-) This mistake was 
far from obvious, and clearly I didn't understand properly how to move data 
around *between processes*. Unless you are just saying I am ignorant or 
something? :-)

> People may not understand how to move data around efficiently, but that's a 
> separate argument.
> 
> Multithreading brings with it a whole lot of extra
>> complications around NOT moving data around.
> 
> I think this involves more subtle bugs that are harder to spot.  

Again, the existence of this thread indicates otherwise. This bug was quite 
subtile and hard to spot. It was only when I started looking at how many times 
a given piece of code was called (specifically, the part that handled data 
coming in for which there wasn't an entry in the dictionary) that I spotted the 
problem. If I hadn't had logging in place in that code block, I would have 
never realized it wasn't working as intended. You don't get much more subtile 
than that. And, furthermore, it only existed because I *wasn't* using threads. 
This bug simply doesn't exist in a threaded model, only in a multiprocessing 
model. Yes, the *explanation* of the bug is simple enough - each process "sees" 
a different value, since memory isn't shared - but the bug in my code was 
neither obvious or easy to spot, at least until you knew what was happening.

> People seem to find it harder to reason about atomicity and realising that 
> widely separated pieces of code may interact unexpectedly.
> 
> Yield points bring with
>> them the risk of locking another thread out unexpectedly (particularly
>> since certain system calls aren't async-friendly on certain OSes).
> 
> I've got to admit I find coroutines straightforward, but I did cut my teeth 
> on a cooperative OS.  It certainly makes the atomicity issues easier to deal 
> with.

I still can't claim to understand them. Threads? No problem. Obviously I'm 
still lacking some understanding of how data works in the multiprocessing 
model, however.

> 
> All
>> three models have their pitfalls.
> 
> Assuredly.  I just think threads are soggier and hard to light^W^W^W^W^W 
> prone to subtler and more mysterious-looking bugs.

And yet, this thread exists because of a subtle and mysterious-looking bug with 
multiple *processes* that doesn't exist with multiple *threads*. Thus the point 
- threads are no *worse* - just different - than any other concurrency model.

> 
> -- 
> Rhodri James *-* Kynesim Ltd
> -- 
> https://mail.python.org/mailman/listinfo/python-list 
> 
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31907] Clarify error message when attempting to call function via str.format()

[mickey695]

mickey695  added the comment:

I think it should be documented properly.

In roughly two weeks I will have some time to look into it. 
So I could probably document the current behaviour by the start of 
December(unless someone beats me to it)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31914] Document Pool.(star)map return type

[Дилян Палаузов]

Дилян Палаузов  added the comment:

If the idea is one day to switch the return type to an iterable/generator, 
returning from starmap() before all parallel executions have completed, this 
needs to be documented and the typeshed adjusted accordingly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30057] signal.signal should check tripped signals

[Antoine Pitrou]

Change by Antoine Pitrou :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31934] Failure to build out of source from a not clean source

[Brett Cannon]

Brett Cannon  added the comment:

What's "unclean" in this case? I mean you can always re-run Make after making 
changes and the build succeeds, so I'm not quite sure what leads to this state.

--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31933] some Blake2 parameters are encoded backwards on big-endian platforms

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4224

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: replacing `else` with `then` in `for` and `try`

[Michael Torrie]

On 11/03/2017 11:44 AM, Jon Ribbens wrote:
> And that's leading you into confusion, as you've demonstrated.

And indeed I've been led into considerable confusion about the else:
clause over the years. Every time I need to use it, I run a python shell
and try it out to remind myself how it works.  However now with your
mental model, Jon, I think I finally have it! Thank you.

I can't think of any normal circumstance where for/else is useful
without a break.  In fact if you have no break you may as well drop the
else entirely, because the block will always execute.  A linter should
definitely flag this.  One reason to leave it as valid syntax is maybe I
am coding up some skeleton code that I plan to fill in later.  So for
now the loop falls through, but at some future point I'll have finished
the searching code and added the break.  But if it were a syntax error I
don't think that'd be any big deal.

Anyway, thank you again, Jon.  Your explanation and model has greatly
cleared up my confusion, and I think if the docs reasoned along the same
lines it would clear up confusion in most programmers.  Certainly it
appears your line of reasoning is what Guido had in mind when he added
the else block.

Sorry Steven, I think Jon's right in this instance and your reasoning
isn't persuasive.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue30057] signal.signal should check tripped signals

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset c713837e91f39dc18740c74729cb7cebcf54fe6e by Antoine Pitrou in 
branch '2.7':
[2.7] bpo-30057: Fix potential missed signal in signal.signal(). (GH-4258) 
(#4263)
https://github.com/python/cpython/commit/c713837e91f39dc18740c74729cb7cebcf54fe6e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31934] Failure to build out of source from a not clean source

[Xavier de Gaye]

Xavier de Gaye  added the comment:

To build out of the 'src' source tree (a subdirectory of the current directory) 
in a 'build' subdirectory:

$ ls -ld src
drwxr-xr-x 21 xavier xavier 4.0K Nov  3 23:50 src/
$ mkdir build
$ cd build
$ $(cd ../src && pwd)/configure && make

This is useful when you build for multiple platform configurations. For example 
on Android one can build for multiple API versions and multiple architectures 
(x86_64, armv7, aarch64, ...).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30844] selectors: Add urgent data to read event

[Pim Klanke]

Pim Klanke  added the comment:

On 02-11-17 16:54, STINNER Victor wrote:
> STINNER Victor  added the comment:
>
> It seems like kqueue supports urgent data:
>
> "EV_OOBAND: Read filter on socket may set this flag to indicate the presence 
> of out of band data on the descriptor."
>
> Example: 
> https://github.com/daurnimator/cqueues/commit/52baaf1c25bc7e6f7cb4685cb05f4ed47a3f404a
Looks promising. I will have to look into this.
> Be careful, I also found:
>
> // Older versions of Mac OS X may not define EV_OOBAND.
> #if !defined(EV_OOBAND)
> # define EV_OOBAND EV_FLAG1
> #endif // !defined(EV_OOBAND)
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: A use-case for for...else with no break

[Steve D'Aprano]

On Fri, 3 Nov 2017 09:13 pm, Serhiy Storchaka wrote:

> What the interpreter or configuration do you use? The standard
> interpreter uses '>>> ' as a prompt.

I have this in my Python startup file:

if (sys.version_info[0] >= 3 and os.name == 'posix'
and os.environ['TERM'] in ['xterm', 'vt100']):
# Make the prompt bold in Python 3.
sys.ps1 = '\001\x1b[1m\002py> \001\x1b[0m\002'
sys.ps2 = '\001\x1b[1m\002... \001\x1b[0m\002'
else:
sys.ps1 = 'py> '



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31934] Failure to build out of source from a not clean source

[Xavier de Gaye]

New submission from Xavier de Gaye :

The error message when building out of a not clean source tree:

renaming build/scripts-3.7/2to3 to build/scripts-3.7/2to3-3.7
renaming build/scripts-3.7/pyvenv to build/scripts-3.7/pyvenv-3.7
gcc -pthread   -Xlinker -export-dynamic -o Programs/_testembed 
Programs/_testembed.o libpython3.7m.a -lpthread -ldl  -lutil   -lm
gcc: error: Programs/_testembed.o: No such file or directory
make[1]: *** [Makefile:707: Programs/_testembed] Error 1

--
components: Build
messages: 305486
nosy: xdegaye
priority: normal
severity: normal
status: open
title: Failure to build out of source from a not clean source
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: replacing `else` with `then` in `for` and `try`

[Chris Angelico]

On Fri, Nov 3, 2017 at 10:49 PM, Jon Ribbens  wrote:
> On 2017-11-03, Steve D'Aprano  wrote:
>> On Fri, 3 Nov 2017 03:31 am, Jon Ribbens wrote:
>>> No, it's an obvious bug. You have a 'for...else' with no 'break'.
>>> Like I said, that should probably be a syntax error.
>>
>> It should absolutely not be a syntax error. There's no reason for it
>> to be a syntax error, except to satisfy some arrogant and foolish
>> idea of purity.
>
> It'd be nice if you could be a little less rude. It's not an "arrogant
> and foolish idea of purity", it's to help people catch bugs in their
> code, and to aid their understanding of the language.

That wasn't rudeness. That was an accurate description of the
situation. It is entirely possible for code to be opinionated,
arrogant, foolish, and all those other adjectives that we normally
appropriate to people.

I don't think it would do much for comprehension. And if it's to catch
bugs, that is NOT the job of syntax - it's the job of linters. By all
means, have a linter that says "else clause after loop with no break";
but don't make it *illegal*.

>> There are uncountable ways of writing code which is seemingly
>> "pointless", and we don't make it a syntax error.
>
> And there's uncountable ways of writing code which we *do* make a
> syntax error. I'm not sure what your point is there.

The point is that syntax errors are for things that cannot possibly
make sense, not for things that are likely to be bugs.

There are a very few exceptions, and they're usually things that are
massive bug magnets, or where the semantics are subtly different. And
I can't think of any examples right now other than the way "async def"
forces a function to be a coroutine even if it has no "await"s in it;
note that it does *not* make a syntax error, it simply causes the
semantics to be correct anyway.

>>> No, 'then' describes the opposite of what it does. The word 'then'
>>> implies something that always happens next,
>>
>> Right, which is what happens with the for...else block.
>
> No. Ok, so look. It's obvious that you and I have different mental
> models of the situation here. You're thinking of 'for...else' as two
> arbitrary clauses that run consecutively unless the whole thing is
> aborted by a 'break', whereas I'm thinking of the 'for' clause as
> being a search for a situation that matches a condition and the
> 'else' clause being what happens if the condition is not matched
> (i.e. exactly the same as 'if...else').
>
> Now there's nothing inherently *wrong* with your choice of mental
> model, except it's leading you into confusion because my model means
> the meaning of the 'else' keyword is intuitive and obvious, and yours
> means it's counter-intuitive and confusing. Your suggestion is that
> the fix is to change the language, my suggestion is to fix your model.
> I'd suggest that changing your mind is easier than changing the
> language ;-)

If anything, I would say that Steven's model is at a lower abstraction
layer than yours - though IMO your model is more of an example
use-case than a description of what is actually happening.

>>> Yes, I saw that. It's possible you are the only person in the world
>>> ever to have done that. It would not make the interactive interpreter
>>> 'worse' in the slightest for that silly trick to be lost.
>>
>> Just because you personally haven't used this technique doesn't make it
>> a "silly trick".
>
> It's an incredibly obscure work-around for a different problem,
> i.e. an infelicity in the way the interactive prompt parses input
> blocks. If the parsing is a genuine issue then the answer is to
> fix that, not to look for hacks that almost never help anyway.

TBH I prefer the "if 1:" trick to gather code into a block. But that
requires pre-planning, whereas slapping an "else:" after the loop can
be done after the event.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31415] Add -X option to show import time

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Nov 3, 2017, at 14:41, Serhiy Storchaka  wrote:
> 
> Note that with environment variable you get more information.

Fun!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31934] Failure to build out of source from a not clean source

[Xavier de Gaye]

Xavier de Gaye  added the comment:

When you build out of the source tree and this source tree has been built 
previously (by running 'make' in the source tree), then you get the error msg 
printed in msg305486 when you try to build *out* of this not clean source tree. 
The solution is to run 'make clean' in the source tree first.

The reason that one must use a clean source tree is that otherwise 'make' finds 
an existing Programs/_testembed.o file in the source tree as the prerequisite 
to the Programs/_testembed target, but cannot build this target since 
Programs/_testembed.o does not exist locally.

It is not obvious from the error message to understand the reason for the build 
failure and one can waste quite some time finding the solution.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31415] Add -X option to show import time

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Nov 3, 2017, at 14:23, Terry J. Reedy  wrote:
> 
> Does it set the EV for the entire session (which one likely would not want), 
> or just the one command (which has no Windows equivalent that I know of)?  
> Please leave the easily remembered and typed option.

On *nix, this sets the environment variable for just this process.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31939] Support return annotation in signature for Argument Clinic

[STINNER Victor]

New submission from STINNER Victor :

Argument Clinic supports a few return types like NoneType, int, bool, etc. But 
the return type is omitted in the private docstring used to build the 
__text_signature__, finally used to build a Signature object in 
inspect.signature().


For example, os.dup() is declared in Modules/posixmodule.c with:

/*[clinic input]
os.dup -> int

fd: int
/

Return a duplicate of a file descriptor.
[clinic start generated code]*/


Currently, Argument Clinic generates:

PyDoc_STRVAR(os_dup__doc__,
"dup($module, fd, /)\n"
"--\n"
"\n"
"Return a duplicate of a file descriptor.");

The return type is omitted in the first line.


Finally, the return type is not documented in the signature:

haypo@selma$ ./python -c "import os,inspect; print(inspect.signature(os.dup))"
(fd, /)


I noticed this limitation while reviewing the PR 4265 which converts the select 
module to Argument Clinic. Previously, the return type like "-> None" or "-> 
int" was documented. With Argument Clinic, it's not documented nor generated in 
the signature, the information is lost.

--
messages: 305525
nosy: brett.cannon, haypo, larry, ncoghlan, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
title: Support return annotation in signature for Argument Clinic
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31938] Convert selectmodule.c to Argument Clinic

[STINNER Victor]

STINNER Victor  added the comment:

While reviewing PR 4265, I noticed that the return type like "-> None" or "-> 
int" is removed from the function definition, so the PR removes this 
information. But it seems to be a limitation of Argument Clinic: I opened the 
issue #31939.

IHMO it's not a blocker issue. We can add it back later, once the issue #31939 
is implemented.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31370] Remove support for threads-less builds

[Berker Peksag]

Change by Berker Peksag :


--
Removed message: https://bugs.python.org/msg302490

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31370] Remove support for threads-less builds

[Berker Peksag]

Change by Berker Peksag :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: right list for SIGABRT python binary question ?

[dieter]

Karsten Hilbert  writes:
> ...
> I have posted backtraces taken from the address being
> watched. Does that help any at all ?

Only in the case that the error is "local", i.e. detected
(quite) immediately.

You might be in this case as you have observed that the address
is stable after library preload. Thus, it might not be a heap
address but one associated with one of the libraries. Such
a memory block should never be "freed". The backtrace would allow
you to determine the library affected. Obtain its source. Recompile
with symbols and try to find out where this memory block comes from.

-- 
https://mail.python.org/mailman/listinfo/python-list