[issue28670] PEP 235: Implement on every POSIX system, and clean up the C code for `import'

[Michael Witten]

Michael Witten added the comment:

Thank you for the reply.

* As already stated, I believe that Python 3
  is not affected by this problem; certainly,
  version 3.5.2 does not seem to be affected,
  as per my ad hoc testing.

* Very many programs are targeted to Python 2.7,
  and probably will be for quite some time.

  Without my patch, `python' (2.7) itself breaks
  them as described; yet, with my patch, these
  programs will just work (without intervention).

  It seems to me that this patch isn't a change
  in a feature, but is rather the overdue
  implementation of promised behavior; the
  file system is what matters, not the host
  operating system, which has hitherto been
  conflated with the file system.
  
  That is, this patch is a bug fix.

* I'm not sure how a test could be written to
  handle this situation.
  
  I suppose that by default, such a test could
  be skipped with a warning; a user could be
  required to set up a relevant mount point,
  and then to signal the testing infrastructure
  to use it.

---

* With regard to the form of the patches, my
  intention is that they be introduced into
  the history severally, with the commit
  messages and other metadata as specified.
  
  It would probably make most sense for these
  commits to be the ancestors of a merge commit
  that provides the sole reference to this
  tracker issue.

  For instance, the most correct and useful
  application of the patch series is given
  by the following:

$ hg pull --branch 2.7 # This must pull in something new for the rest to 
work.
$ tip=$(hg log --limit 1 --template '{node}' -r tip)
$ hg update -c -r 59260b38f7cd4a2ae66928de5227798524006e64
$ hg import /path/to/pep-235-on-posix.patch
$ hg merge -r "$tip"
$ hg commit --message "Issue #28670: Implement PEP 235 on every POSIX 
system, and clean up the C code for \`import'"

  That produces a single commit whose message supplies
  the usual dearth of information, but whose ancestral
  history captures the rich details.

  It should be noted that `hg bundle' and `hg unbundle'
  could be used to import more directly the proper
  pre-merge history, albeit at the expense of the
  human-readability of the "patch" file.

  That is to say, I'm underwhelmed by the way you guys
  are managing this project's history. Despite all of its
  faults, Mercurial is still being underutilized.

* I could split the original patch file into 3 separate
  patch files, but what should I do with the metadata?
  If each separate patch were to contain its own metadata,
  would the code review tool still choke? I suspect it would.

  What's the point?

  The original file itself is perfectly readable, and you
  can simply review the diffs after applying them within
  your repo, anyway. Ironically, the very point of having
  multiple changesets is to make digesting the changes as
  easy as possible---not only now, but far into the future,
  when you're simply looking at the graph of the history!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28694] tkinter interface to fontchooser

[Terry J. Reedy]

Terry J. Reedy added the comment:

https://www.tcl.tk/man/tcl/TkCmd/fontchooser.htm
I agree that this should be wrapped, as is colorchooser.  There are a few 
details that may need to be thrashed out.  It is a nuisance that the native 
font dialog is modal on Windows and not on Mac (and ??? on *nix?).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28694] tkinter interface to fontchooser

[Zachary Ware]

Zachary Ware added the comment:

You've already done the proposing part by opening this issue ;).  To contribute 
you work, attach it here as a patch to the tkinter sources found in 
Lib/tkinter/ of a cpython source checkout.  You'll need to sign a contributor 
agreement before we can look at or accept it, the tracker should prompt you to 
do so when you attach your patch.  The devguide is also a great resource for 
questions you may have while preparing your patch.

Thanks for your interest in contributing!

--
nosy: +serhiy.storchaka, terry.reedy, zach.ware
stage:  -> needs patch
versions: +Python 3.7 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28666] Make test.support.rmtree() able to remove non-writable directories

[Kushal Das]

Kushal Das added the comment:

The patch looks good to me. This can be applied, and tests are running fine 
with the patch.

https://ci.centos.org/job/cPython-build-patch/25/console

--
nosy: +kushal.das

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28692] gettext: deprecate selecting plural form by fractional numbers

[Xiang Zhang]

Xiang Zhang added the comment:

Maybe you have forgotten to remove the debug print in the patch?

--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28694] tkinter interface to fontchooser

[Lance Ware]

New submission from Lance Ware:

Tcl/Tk 8.6 now has a fontchooser command. I have developed a tkinter interface 
to it, similar to colorchooser/askcolor. How does one go about contributing 
this or proposing for future enhancement to the tkinter module suite?

--
components: Tkinter
messages: 280820
nosy: Lance Ware
priority: normal
severity: normal
status: open
title: tkinter interface to fontchooser
type: enhancement
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28573] Python 3.6.0b3 64-bit has no sys._mercurial info

[Steve Dower]

Changes by Steve Dower :


--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28573] Python 3.6.0b3 64-bit has no sys._mercurial info

[Roundup Robot]

Roundup Robot added the comment:

New changeset d997e64130bd by Steve Dower in branch '3.6':
Issue #28573: Fixes issue with nested if blocks
https://hg.python.org/cpython/rev/d997e64130bd

New changeset 35f510158490 by Steve Dower in branch 'default':
Issue #28573: Fixes issue with nested if blocks
https://hg.python.org/cpython/rev/35f510158490

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28573] Python 3.6.0b3 64-bit has no sys._mercurial info

[Steve Dower]

Steve Dower added the comment:

I suspect this is because my PATH was getting too long within my build script 
because of setting up the VS build environment too many times (if PATH gets too 
long then new processes may ignore or truncate it).

The commit should cut this down to the point where we don't lose Mercurial off 
of PATH, but I'm running validation builds to be sure.

As this only applies to the Windows builds and we haven't seen it against 3.5 
yet, I'm dropping that from the versions list.

--
stage: test needed -> commit review
versions:  -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28573] Python 3.6.0b3 64-bit has no sys._mercurial info

[Roundup Robot]

Roundup Robot added the comment:

New changeset 25cb7df5b19d by Steve Dower in branch '3.6':
Issue #28573: Avoid setting up env too many times during build
https://hg.python.org/cpython/rev/25cb7df5b19d

New changeset ae8f525cef2a by Steve Dower in branch 'default':
Issue #28573: Avoid setting up env too many times during build
https://hg.python.org/cpython/rev/ae8f525cef2a

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28670] PEP 235: Implement on every POSIX system, and clean up the C code for `import'

[Brett Cannon]

Brett Cannon added the comment:

Because this would be a feature change in Python 2.7 it can't go in there. This 
could potentially go into 3.7 if it's deemed useful.

As for why the code review tool isn't picking up your uploads, it's because we 
work in patch files and not patch sets. See 
https://docs.python.org/devguide/patch.html on how to create a patch file that 
can be reviewed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28670] PEP 235: Implement on every POSIX system, and clean up the C code for `import'

[Brett Cannon]

Changes by Brett Cannon :


--
stage:  -> test needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Elliot Gorokhovsky]

Elliot Gorokhovsky added the comment:

On Mon, Nov 14, 2016 at 10:32 AM STINNER Victor 
wrote:

> You can use perf timeit --compare-to to check if the result is significant
> or not, and it displays you the "N.NNx faster" or "N.NNx slower" if it's
> significant.
>

Will do -- I'm writing this up as a paper since this is my science fair
project, so I'll redo the measurements that way and upload a pdf here.

> About benchmarks, I also would like to see a benchmark on the bad case,
> when specialization is not used. And not only on an empty list :-) For
> example, sort 1000 objects which implement compare operators and/or a sort
> function.
>

The worst case is the third benchmark from the top -- a list of floats with
a single, sad, solitary long at the end. That disables optimization because
keys_are_all_same_type gets set to 0 when assign_compare_function finds the
long (since key_type == _Type). That benchmark is the *absolute*
worst case for two reasons:

1. Float compares are really cheap, so the ratio doesn't get messed up by
the common term "time spent actually comparing floats" (since the total
time is "time spent on overhead + time spend actually comparing floats").
2. The list is of size 10. I guess I could've used size 3 or 4, but it
shouldn't be too far off... smaller lists give worse ratios because the
overhead is O(n) while sorting is O(nlogn).

So, again, the absolute worst possible case is the third benchmark, which
suffers a 10% slowdown. Certainly a reasonable price to pay considering how
rare that case is in practice, and considering the 40-75% gains we get on
the common cases.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28690] Loop in re (regular expression) processing

[Walter Farrell]

Walter Farrell added the comment:

Thanks, Gareth. That does work.

Interesting that regex does still seem to work linearly with the original
version, but your version seems cleaner.

On Mon, Nov 14, 2016 at 3:55 PM, Gareth Rees  wrote:

>
> Gareth Rees added the comment:
>
> This is a well-known gotcha with backtracking regexp implementations. The
> problem is that in the alternation "( +|'[^']*'|\"[^\"]*\"|[^>]+)" there
> are some characters (space, apostrophe, double quotes) that match multiple
> alternatives (for example a space matches both " +" and "[^>]+"). This
> causes the regexp engine to have to backtrack for each ambiguous character
> to try out the other alternatives, leading to runtime that's exponential in
> the number of ambiguous characters.
>
> Linear behaviour can be restored if you make the alternation unambiguous,
> like this: ( +|'[^']*'|\"[^\"]*\"|[^>'\"]+)
>
> --
> nosy: +Gareth.Rees
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28690] Loop in re (regular expression) processing

[Gareth Rees]

Gareth Rees added the comment:

This is a well-known gotcha with backtracking regexp implementations. The 
problem is that in the alternation "( +|'[^']*'|\"[^\"]*\"|[^>]+)" there are 
some characters (space, apostrophe, double quotes) that match multiple 
alternatives (for example a space matches both " +" and "[^>]+"). This causes 
the regexp engine to have to backtrack for each ambiguous character to try out 
the other alternatives, leading to runtime that's exponential in the number of 
ambiguous characters.

Linear behaviour can be restored if you make the alternation unambiguous, like 
this: ( +|'[^']*'|\"[^\"]*\"|[^>'\"]+)

--
nosy: +Gareth.Rees

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24117] Wrong range checking in GB18030 decoder.

[Mingye Wang]

Mingye Wang added the comment:

Just FYI, cp950 0xC6A1 (\uf6b1) is found in current WindowsBestFit: 
ftp://ftp.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WindowsBestFit/bestfit950.txt

--
nosy: +Artoria2e5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28693] No HKSCS support in Windows cp950

[Mingye Wang]

New submission from Mingye Wang:

Python's cp950 implementation lacks support for HKSCS ('big5hkscs'). This 
support, which maps HKSCS Big5-EUDC code points to Unicode PUA code points 
algorithmically, is found in Windows Vista+ as well as an update for XP.

An experiment session is shown below. I will use '2>>>' to denote a Win32 build 
of Python 2.7.10 running under a console window set to cp950 (via chcp), and 
'3>>>' to denote a Python 3.4.3 build running under Cygwin's UTF-8 mintty. 
HKSCS-2008's table is used  
http://www.ogcio.gov.hk/en/business/tech_promotion/ccli/terms/doc/hkscs-2008-big5-iso.txt
 for a list of HKSCS characters; note though, its non-PUA mappings are not 
found in Windows.

Let's start with the first character in that list.

3>>> u'\u43F0'
'䏰'
3>>> print(u'\uF266') # provisional PUA

3>>> u'\u43F0'.encode('cp950') # FAIL
3>>> u'\uF266'.encode('cp950') # FAIL
3>>> u'\u43F0'.encode('hkscs')
b'\x87@'
3>>> u'\uF266'.encode('hkscs') # FAIL`

These experiments above show how Python 3 handles HKSCS characters, and how 
U+43F0 should normally be encoded. Now let's switch to Windows console, which 
would be using Windows' decode-to-Unicode routine for cp950.

2>>> print b'\x87@'


Let's try to identify this character:

3>>> u''
'\uf266'

So indeed there is some sort of HKSCS going on. But note what Windows has is 
really not any kind of new HKSCS:

> Big5   ucs93  ucs00   ucs03 + 1-6
> 876B   9734   97349734
> 876C   F292   F292   27BEF
> 876D   5BDB   5BDB5BDB

2>>> print b'\x87\x6b,\x87\x6c,\x87\x6d'
,,
3>>> u',,'
'\uf291,\uf292,\uf293'

Just as for all other code pages, you can always find Microsoft's mapping at 
ftp://ftp.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WindowsBestFit/bestfit950.txt.
 If you are uncomfortable with adding a whole new table and wasting space (this 
is done for hkscs btw), use the algorithmic mapping at 
https://en.wikipedia.org/wiki/Code_page_950.

--
components: Unicode
messages: 280811
nosy: Artoria2e5, ezio.melotti, haypo
priority: normal
severity: normal
status: open
title: No HKSCS support in Windows cp950
type: behavior
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28692] gettext: deprecate selecting plural form by fractional numbers

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Sorry. Here is a patch.

--
keywords: +patch
Added file: 
http://bugs.python.org/file45485/gettext-non-int-plural-deprecate.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28686] py.exe ignored PATH when using python3 shebang

[Paul Moore]

Paul Moore added the comment:

I agree that the docs are a little confusing on this. Having said that, though, 
I'm not entirely sure the behaviour needs fixing. The scenario where there's a 
problem is:

1. User has written a script that needs Python 3 and won't run with Python 2.
2. User does not want to force a specific interpreter, or use the system Python.
3. User wants to use virtualenvs (or otherwise manipulate PATH to switch Python 
versions) and will sometimes have Python 2 and sometimes Python 3 active.
4. User expects to be able to run the script with Python 2 active and in that 
case use the system Python 3.

That's a pretty rare situation, IMO. Add to that the fact that it's really hard 
to detect when the python on PATH is Python 2 (you basically need to run python 
-V).

The discussion seems to be veering a little too much towards "do what I mean" 
to me, and I'd prefer we keep the behaviour simple.

I'm happy with the idea that "/usr/bin/env python3" should search, and then 
fall back to the system defined Python 3. The question is what should we search 
*for*. There seem to me to be only 2 options:

1. python - but this could select a Python 2 interpreter, so that seems wrong.
2. python3 - that's logical but has a number of quirks
   2a. There isn't a python3 executable supplied as standard, so this behaviour 
is useless without user customisation.
   2b. Do we want to force searching for additional extensions like .bat? What 
about .ps1 for Powershell users? What do we do for the GUI version?

Overall, I think option 2 has too many grey areas to be really acceptable, and 
I'm inclined to say YAGNI, and we simply leave "/usr/bin/env python3" undefined.

For the OP's use case, I'm not clear precisely why "/usr/bin/env python" 
wouldn't be a sufficiently good approach. If there's a specific scenario in his 
setup that means we really do *have* to have "/usr/bin/env python3" I'd be 
grateful if he could clarify. Or to put it another way, is this issue simply "I 
was surprised it didn't work as I expected" or is it "I cannot do what I want 
because this doesn't work"?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19717] resolve() fails when the path doesn't exist

[Steve Dower]

Changes by Steve Dower :


--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19717] resolve() fails when the path doesn't exist

[Steve Dower]

Changes by Steve Dower :


--
stage: commit review -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27854] Installed 2.7: IDLE Help disabled because help.html is missing

[Steve Dower]

Steve Dower added the comment:

I ran a build and the file is included now. I didn't test an install or the 
functionality from IDLE, so make sure you cover that with the RC (and since 
there won't be an RC with a release blocker open, I'll resolve this and we can 
reactivate if there are new issues).

--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28635] Update What's New for 3.6

[Roundup Robot]

Roundup Robot added the comment:

New changeset 993215342a95 by Yury Selivanov in branch '3.6':
Issue #28635: what's new in 3.6: add a few more notes on typing
https://hg.python.org/cpython/rev/993215342a95

New changeset c3b4eea73615 by Yury Selivanov in branch 'default':
Merge 3.6 (issue #28635)
https://hg.python.org/cpython/rev/c3b4eea73615

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28692] gettext: deprecate selecting plural form by fractional numbers

[Julien Palard]

Julien Palard added the comment:

Hi, did you forget to attach the patch?

--
nosy: +mdk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28691] python3.6 -Werror -c '"\c"' fails with an assertion error

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

This is my fault. Proposed patch adds tests for the "error" action.

--
components: +Interpreter Core
nosy: +serhiy.storchaka
stage:  -> patch review
type:  -> crash
Added file: 
http://bugs.python.org/file45484/test_eval_str_invalid_escape_error.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28692] gettext: deprecate selecting plural form by fractional numbers

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

GNU gettext library accepts only integer value for selecting plural form. 
Python gettext accepts arbitrary numbers. But gettext formulas are not purposed 
to support non-integer values and can return incorrect result. For example (in 
Ukrainian):

   "1 площа", but "1.5 площі", not "1.5 площ".
   "2 гектари", but "2.75 гектара", not "2.75 гектарів".
   "5 тонн", but "5.7 тонни", not "5.7 тонн".

Separate plural form should be used for fractional numbers. Even if fractional 
part happens to be zero, it is acceptable (e.g. "Time elapsed: 1.000 seconds" 
in English).

Proposed patch deprecates fractional numbers for selecting plural form in 
gettext.

--
components: Library (Lib)
messages: 280804
nosy: Tim.Graham, loewis, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: gettext: deprecate selecting plural form by fractional numbers
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28686] py.exe ignored PATH when using python3 shebang

[wdhwg001]

wdhwg001 added the comment:

Okay. But somehow I still think the current way of handling shebang is 
confusing. That makes a python3-only script unable to find a way to use both 
versioned shebang and virtualenv.

Maybe it could be changed to `Command is eligible for path search, or there is 
no version specification`.

Then if shebang is versioned, py.exe should try to search `PATH` for a 
versioned executable file(though it might not exist), and fallback to the 
installed list.

This could be less confusing I think.

And more importantly, virtualenv cannot and shouldn't create registry keys or 
even hijack py.exe to fix this issue. This change provides a better way.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Yes, you are right, the stacklevel should be 4. Thank you for your report and 
testing Tim.

Committed the patch without any deprecation. Will add a deprecation in separate 
issue.

--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[STINNER Victor]

STINNER Victor added the comment:

Maybe we should investigate more optimizations on specialized lists.

PyPy uses a more compact structure for lists of integers for example. Something 
like compact strings, PEP 393, of Python 3.3, but for lists.

http://doc.pypy.org/en/latest/interpreter-optimizations.html#list-optimizations

But we are limited by the C API, so we cannot change deeply the C structure 
without breaking backward compatibility.


> # (difference is within std dev)

You can use perf timeit --compare-to to check if the result is significant or 
not, and it displays you the "N.NNx faster" or "N.NNx slower" if it's 
significant.

About benchmarks, I also would like to see a benchmark on the bad case, when 
specialization is not used. And not only on an empty list :-) For example, sort 
1000 objects which implement compare operators and/or a sort function.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Roundup Robot]

Roundup Robot added the comment:

New changeset 5b6cde995b3a by Serhiy Storchaka in branch '3.3':
Issue #28563: Make plural form selection more lenient and accepting
https://hg.python.org/cpython/rev/5b6cde995b3a

New changeset 6ca91a14a555 by Serhiy Storchaka in branch '2.7':
Issue #28563: Make plural form selection more lenient and accepting
https://hg.python.org/cpython/rev/6ca91a14a555

New changeset f78a05cda5aa by Serhiy Storchaka in branch '3.4':
Issue #28563: Make plural form selection more lenient and accepting
https://hg.python.org/cpython/rev/f78a05cda5aa

New changeset 6a2754055ff8 by Serhiy Storchaka in branch '3.5':
Issue #28563: Make plural form selection more lenient and accepting
https://hg.python.org/cpython/rev/6a2754055ff8

New changeset 4c201c65ce5d by Serhiy Storchaka in branch '3.6':
Issue #28563: Make plural form selection more lenient and accepting
https://hg.python.org/cpython/rev/4c201c65ce5d

New changeset d0efb8532589 by Serhiy Storchaka in branch 'default':
Issue #28563: Make plural form selection more lenient and accepting
https://hg.python.org/cpython/rev/d0efb8532589

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Elliot Gorokhovsky]

Elliot Gorokhovsky added the comment:

Sure, if it compiles without that def, I'll remove it from the patch. I
added it because I did all the development in an extension module, which
also included Python.h, but for some reason it gave me a "function
implicitly defined" error for using Py_ABS. Even though I had included
Python.h. Weird, right? So I assumed I had to leave it in the patch as
well. Thanks for pointing this out!

On Mon, Nov 14, 2016 at 6:09 AM Julien Palard 
wrote:

>
> Julien Palard added the comment:
>
> Hi Elliot, nice spot!
>
> Why are you redefining Py_ABS, which looks already defined in `pymacro.h`
> included itself by `Python.h`? I'm not fan of undefining it later, it may
> surprise someone later expecting it to be there.
>
> I tried to compile without your definition of Py_ABS, just in case I
> missed something in the includes, and it works.
>
> --
> nosy: +mdk
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28688] Warning -- warnings.filters was modified by test_warnings

[Xiang Zhang]

Xiang Zhang added the comment:

test___all__ gets the same behaviour.

./python -Werror -m test test___all__
Run tests sequentially
0:00:00 [1/1] test___all__
Warning -- warnings.filters was modified by test___all__
test___all__ failed (env changed)

1 test altered the execution environment:
test___all__

Total duration: 1 sec
Tests result: SUCCESS

And on my PC in company I sometimes get the behaviour for test_distutils.

I originally thought this is not a problem.

--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28691] python3.6 -Werror -c '"\c"' fails with an assertion error

[STINNER Victor]

STINNER Victor added the comment:

> By the way, an unit test is missing since the bug was not catched by the test 
> suite!

Lib/test/test_string_literals.py contains an unit test but it changes the 
warnings filter, so -Werror option is ignored.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28691] python3.6 -Werror -c '"\c"' fails with an assertion error

[STINNER Victor]

STINNER Victor added the comment:

Attached ast.patch fixes the issue: replace the DeprecationWarning with a 
SyntaxError.

Is it the expected behaviour? Is it worth it to chain the two exceptions?

--
keywords: +patch
Added file: http://bugs.python.org/file45483/ast.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28569] mock.attach_mock should work with any return value of patch()

[Andrey Fedorov]

Andrey Fedorov added the comment:

Here's the full extension of the example from documentation that doesn't
seem to handle classes and functions consistently:

  import mock

  patches = {
  'requests_get': 'requests.get',
  'mymodule_Class1': 'mymodule.Class1'
  }

  root_mock = mock.Mock()
  for name, path in patches.items():
  m = mock.patch(path, autospec=True)
  root_mock.attach_mock(m.start(), name)

  import requests
  import mymodule

  mymodule.Class1('foo')
  requests.get('bar')

  print root_mock.mock_calls
  # [call.mymodule_Class1('foo')]

Does this working as expected make sense, or is there some reason this is
an undesirable API to behave consistently regardless of what's being
patched?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28691] python3.6 -Werror -c '"\c"' fails with an assertion error

[STINNER Victor]

STINNER Victor added the comment:

By the way, an unit test is missing since the bug was not catched by the test 
suite!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28691] python3.6 -Werror -c '"\c"' fails with an assertion error

[STINNER Victor]

New submission from STINNER Victor:

The issue #28128 (changeset 259745f9a1e4) introduced a DeprecationWarning on 
invalid Unicode sequence like "\c". When Python is run with -Werror, Python 
crashs with an assertion error:

$ ./python -Werror -c '"\c"'
python: Objects/abstract.c:2232: PyObject_Call: Assertion `!PyErr_Occurred()' 
failed.
Aborted (core dumped)

Gdb traceback:

(gdb) where
#0  0x7711f6f5 in __GI_raise (sig=sig@entry=6) at 
../sysdeps/unix/sysv/linux/raise.c:54
#1  0x771212fa in __GI_abort () at abort.c:89
#2  0x77117f97 in __assert_fail_base (fmt=, 
assertion=assertion@entry=0x69b577 "!PyErr_Occurred()", 
file=file@entry=0x69a922 "Objects/abstract.c", line=line@entry=2232, 
function=function@entry=0x69b9c8 <__PRETTY_FUNCTION__.11084> 
"PyObject_Call") at assert.c:92
#3  0x77118042 in __GI___assert_fail (assertion=0x69b577 
"!PyErr_Occurred()", file=0x69a922 "Objects/abstract.c", line=2232, 
function=0x69b9c8 <__PRETTY_FUNCTION__.11084> "PyObject_Call")
at assert.c:101
#4  0x0045afe8 in PyObject_Call (func=, 
args=(2, 'No such file or directory', ''), kwargs=0x0) at 
Objects/abstract.c:2232
#5  0x005e77de in PyErr_SetFromErrnoWithFilenameObjects (exc=, filenameObject='', filenameObject2=0x0) at 
Python/errors.c:559
#6  0x005e7631 in PyErr_SetFromErrnoWithFilenameObject (exc=, filenameObject='') at Python/errors.c:471
#7  0x0043e839 in _Py_fopen_obj (path='', mode=0x6c86e7 "rb") at 
Python/fileutils.c:1138
#8  0x005e8e54 in PyErr_ProgramTextObject (filename='', 
lineno=1) at Python/errors.c:1179
#9  0x00598e90 in ast_error (c=0x7fffd8b0, n=0x702d31f0, 
errmsg=0x702c1a80 "invalid escape sequence \\c") at Python/ast.c:673
#10 0x005a3ae8 in warn_invalid_escape_sequence (c=0x7fffd8b0, 
n=0x702d31f0, first_invalid_escape_char=99 'c') at Python/ast.c:4134
#11 0x005a4123 in decode_unicode_with_escapes (c=0x7fffd8b0, 
n=0x702d31f0, s=0x703157f0 "\\c\313\313\313\313\313\313\313\313\313", 
, len=2)
at Python/ast.c:4202
#12 0x005a63b8 in parsestr (c=0x7fffd8b0, n=0x702d31f0, 
bytesmode=0x7fffd29c, rawmode=0x7fffd298, result=0x7fffd290, 
fstr=0x7fffd288, fstrlen=0x7fffd280)
at Python/ast.c:5114
#13 0x005a64cb in parsestrplus (c=0x7fffd8b0, n=0x702d31a8) at 
Python/ast.c:5145
#14 0x0059d17b in ast_for_atom (c=0x7fffd8b0, n=0x702d31a8) at 
Python/ast.c:2112
#15 0x0059e4d9 in ast_for_atom_expr (c=0x7fffd8b0, 
n=0x702d3160) at Python/ast.c:2467
#16 0x0059e649 in ast_for_power (c=0x7fffd8b0, n=0x702d3118) at 
Python/ast.c:2504
#17 0x0059ef95 in ast_for_expr (c=0x7fffd8b0, n=0x702d3118) at 
Python/ast.c:2692
#18 0x0059f8d2 in ast_for_testlist (c=0x7fffd8b0, n=0x70351d30) 
at Python/ast.c:2883
#19 0x0059f992 in ast_for_expr_stmt (c=0x7fffd8b0, 
n=0x70351ce8) at Python/ast.c:2906
#20 0x005a3475 in ast_for_stmt (c=0x7fffd8b0, n=0x70351ce8) at 
Python/ast.c:3983
#21 0x005997e8 in PyAST_FromNodeObject (n=0x70351c10, 
flags=0x7fffdcf0, filename='', arena=0x702ec040) at 
Python/ast.c:839
#22 0x0042a146 in PyParser_ASTFromFileObject (fp=0x774a78c0 
<_IO_2_1_stdin_>, filename='', enc=0x77e574f8 "UTF-8", start=256, 
ps1=0x702cdd18 ">>> ", 
ps2=0x702cdde8 "... ", flags=0x7fffdcf0, errcode=0x7fffda4c, 
arena=0x702ec040) at Python/pythonrun.c:1169
#23 0x00426b00 in PyRun_InteractiveOneObject (fp=0x774a78c0 
<_IO_2_1_stdin_>, filename='', flags=0x7fffdcf0) at 
Python/pythonrun.c:212
#24 0x00426698 in PyRun_InteractiveLoopFlags (fp=0x774a78c0 
<_IO_2_1_stdin_>, filename_str=0x697276 "", flags=0x7fffdcf0) at 
Python/pythonrun.c:112
#25 0x00426491 in PyRun_AnyFileExFlags (fp=0x774a78c0 
<_IO_2_1_stdin_>, filename=0x697276 "", closeit=0, flags=0x7fffdcf0) 
at Python/pythonrun.c:74
#26 0x004421cd in run_file (fp=0x774a78c0 <_IO_2_1_stdin_>, 
filename=0x0, p_cf=0x7fffdcf0) at Modules/main.c:319
#27 0x00443086 in Py_Main (argc=2, argv=0x9d2010) at Modules/main.c:779
#28 0x0041d2c9 in main (argc=2, argv=0x7fffdf28) at 
./Programs/python.c:69

--
messages: 280793
nosy: ebarry, eric.smith, haypo, ned.deily
priority: release blocker
severity: normal
status: open
title: python3.6 -Werror -c '"\c"' fails with an assertion error
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Tim Graham]

Tim Graham added the comment:

Yes, that fixes the second test. Current warning (with stacklevel=3):

/home/tim/code/cpython/Lib/gettext.py:454: DeprecationWarning: Plural value 
must be an integer, got 1.29
  tmsg = self._catalog[(msgid1, self.plural(n))]

Possibly the stacklevel should instead be 4:

/home/tim/code/django/django/utils/translation/trans_real.py:373: 
DeprecationWarning: Plural value must be an integer, got 1.29
  return getattr(t, translation_function)(singular, plural, number)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

What about following patch?

--
Added file: http://bugs.python.org/file45482/gettext-non-integer-plural-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28690] Loop in re (regular expression) processing

[Walter Farrell]

New submission from Walter Farrell:

Given:
  pattern = r"(^|[^\\])<(pm [^ ]+( +|'[^']*'|\"[^\"]*\"|[^>]+)+)>"
  s = "Bain, F. W. " added to the end of s, it returns quickly with a match. 
Without the ">" it should fail, but instead seems to loop.

(If I use the regex module instead of re, it fails properly and quickly.)

Python 3.5.1, Windows 10:
Python 3.5.1 (v3.5.1:37a07cee5969, Dec  6 2015, 01:54:25) [MSC v.1900 64 bit 
(AMD64)] on win32

--
components: Library (Lib), Regular Expressions
messages: 280790
nosy: Walter Farrell, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: Loop in re (regular expression) processing
type: resource usage
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Tim Graham]

Tim Graham added the comment:

Thanks, that does fix that first test. There is one more that still fails:

$ python -Wall tests/runtests.py 
humanize_tests.tests.HumanizeTests.test_i18n_intword
Testing against Django installed in '/home/tim/code/django/django' with up to 3 
processes
Creating test database for alias 'default'...
Creating test database for alias 'other'...
/home/tim/code/cpython/Lib/gettext.py:454: DeprecationWarning: Plural value 
must be an integer, got 1.2
  tmsg = self._catalog[(msgid1, self.plural(n))]
/home/tim/code/cpython/Lib/gettext.py:454: DeprecationWarning: Plural value 
must be an integer, got 1.2
  tmsg = self._catalog[(msgid1, self.plural(n))]
F
==
FAIL: test_i18n_intword (humanize_tests.tests.HumanizeTests)
--
Traceback (most recent call last):
  File "/home/tim/code/django/tests/humanize_tests/tests.py", line 133, in 
test_i18n_intword
self.humanize_tester(test_list, result_list, 'intword')
  File "/home/tim/code/django/tests/humanize_tests/tests.py", line 39, in 
humanize_tester
msg="%s test failed, produced '%s', should've produced '%s'" % (method, 
rendered, result))
AssertionError: '1,2 Million' != '1,2 Millionen' : intword test failed, 
produced '1,2 Million', should've produced '1,2 Millionen'

I think the relevant code is 
https://github.com/django/django/blob/cbae4d31847d75d889815bfe7c04af035f45e28d/django/contrib/humanize/templatetags/humanize.py#L60-L63.
 I'm not too familiar with this code, but I'll try to get a better explanation 
if it's not clear to you.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28689] OpenSSL 1.1.0c test failures

[Christian Heimes]

Christian Heimes added the comment:

OpenSSL upstream bug: https://github.com/openssl/openssl/issues/1919

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28662] catch also PermissionError in tests when spawning a non existent program

[Roundup Robot]

Roundup Robot added the comment:

New changeset 1b2b2cb8f962 by Xavier de Gaye in branch '3.6':
Issue #28662: Catch PermissionError in tests when spawning a non existent 
program
https://hg.python.org/cpython/rev/1b2b2cb8f962

New changeset c3f7d81d9050 by Xavier de Gaye in branch 'default':
Issue #28662: Merge 3.6
https://hg.python.org/cpython/rev/c3f7d81d9050

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28662] catch also PermissionError in tests when spawning a non existent program

[Xavier de Gaye]

Changes by Xavier de Gaye :


--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28689] OpenSSL 1.1.0c test failures

[Christian Heimes]

Christian Heimes added the comment:

https://github.com/openssl/openssl/commit/122580ef71e4e5f355a1a104c9bfb36feee43759

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28689] OpenSSL 1.1.0c test failures

[Christian Heimes]

Christian Heimes added the comment:

A git bisect between OpenSSL_1_1_0b (good) and OpenSSL_1_1_0c (bad) revealed 
the breaking commit:

$ git bisect good
122580ef71e4e5f355a1a104c9bfb36feee43759 is the first bad commit
commit 122580ef71e4e5f355a1a104c9bfb36feee43759
Author: Matt Caswell 
Date:   Fri Oct 21 13:25:19 2016 +0100

A zero return from BIO_read()/BIO_write() could be retryable

A zero return from BIO_read()/BIO_write() could mean that an IO operation
is retryable. A zero return from SSL_read()/SSL_write() means that the
connection has been closed down (either cleanly or not). Therefore we
should not propagate a zero return value from BIO_read()/BIO_write() back
up the stack to SSL_read()/SSL_write(). This could result in a retryable
failure being treated as fatal.

Reviewed-by: Richard Levitte 
(cherry picked from commit 4880672a9b41a09a0984b55e219f02a2de7ab75e)

:04 04 8097bc37a0a2a3c1e6a8879ad49ee773001d8d52 
8083927cb2eb28a71baa8b90b07b0962016d74b3 M  ssl

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Proposed patch makes gettext accepting non-integer numbers. But 
DeprecationWarning is emitted if this number have fractional part (because in 
this case the result of current code can be different from the result of old 
code). I think Django tests should be passed without errors and warnings.

In future versions (probably in 3.7) this warning will be replaced with 
TypeError or ValueError, and new warning will be added for all non-integer 
numbers.

--
stage: resolved -> patch review
status: closed -> open
Added file: http://bugs.python.org/file45481/gettext-non-integer-plural.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24339] iso6937 encoding missing

[STINNER Victor]

STINNER Victor added the comment:

Ok. I'm not waiting for a simpler patch reusing existing charmap
functions to see the complexity of the codec ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28688] Warning -- warnings.filters was modified by test_warnings

[STINNER Victor]

STINNER Victor added the comment:

> How you got this warning? I can't reproduce.

Sorry, I forgot to mention that the warning only occurs if you run Python with 
-Werror:

@selma$ ./python -Werror -m test test_warnings
Run tests sequentially
0:00:00 [1/1] test_warnings
Warning -- warnings.filters was modified by test_warnings
test_warnings failed (env changed)

1 test altered the execution environment:
test_warnings

Total duration: 2 sec
Tests result: SUCCESS


Moreover, the warning goes away if you don't run tests in verbose mode!?

haypo@selma$ ./python -Werror -m test -v test_warnings
...
1 test OK.

Total duration: 2 sec
Tests result: SUCCESS

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Xiang Zhang]

Xiang Zhang added the comment:

GNU gettext only allows the plural value(n) to be an integer(unsigned long 
int). Django deliberately converts the value to long in filesizeformat. Any 
reason not to use int?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Tim Graham]

Tim Graham added the comment:

Hi, this broke a couple tests with Django because it's passing number as a 
float rather than an integer. For example:

==
ERROR: test_localized_formats 
(template_tests.filter_tests.test_filesizeformat.FunctionTests)
--
Traceback (most recent call last):
  File 
"/home/tim/code/django/tests/template_tests/filter_tests/test_filesizeformat.py",
 line 28, in test_localized_formats
self.assertEqual(filesizeformat(1023), '1023\xa0Bytes')
  File "/home/tim/code/django/django/template/defaultfilters.py", line 895, in 
filesizeformat
value = ungettext("%(size)d byte", "%(size)d bytes", bytes_) % {'size': 
bytes_}
  File "/home/tim/code/django/django/utils/translation/__init__.py", line 91, 
in ungettext
return _trans.ungettext(singular, plural, number)
  File "/home/tim/code/django/django/utils/translation/trans_real.py", line 
385, in ngettext
return do_ntranslate(singular, plural, number, 'ngettext')
  File "/home/tim/code/django/django/utils/translation/trans_real.py", line 
372, in do_ntranslate
return getattr(t, translation_function)(singular, plural, number)
  File "/home/tim/code/cpython/Lib/gettext.py", line 441, in ngettext
tmsg = self._catalog[(msgid1, self.plural(n))]
  File "", line 4, in func
ValueError: Plural value must be an integer, got 1023.0.

Can you advise if this patch could be adapted or if Django should adapt?

By the way, I used this patch to get a more useful error message:

diff --git a/Lib/gettext.py b/Lib/gettext.py
index 7032efa..2076a3f 100644
--- a/Lib/gettext.py
+++ b/Lib/gettext.py
@@ -185,7 +185,7 @@ def c2py(plural):
 exec('''if True:
 def func(n):
 if not isinstance(n, int):
-raise ValueError('Plural value must be an integer.')
+raise ValueError('Plural value must be an integer, got %%s.
 return int(%s)
 ''' % result, ns)
 return ns['func']

--
nosy: +Tim.Graham

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24339] iso6937 encoding missing

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> My rule is more to only added encodings used (in practice) as locale 
> encodings.

Just for reference: issue19459, issue21081, issue22679, issue20087.

> @Serhiy: Do you think that the encoding is popular enough to pay the
price of its maintainance?

Yes, it seems to me that the encoding still in use. I found questions about 
decoding from ISO 6937 and implementations for different programming languages.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28689] OpenSSL 1.1.0c test failures

[Christian Heimes]

Christian Heimes added the comment:

test_server_accept (test.test_ssl.ThreadedTests) ... Exception in thread 
Thread-348:
Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/threading.py", line 916, in 
_bootstrap_inner
self.run()
  File "/home/heimes/dev/python/cpython/Lib/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
  File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 3044, in 
serve
remote.recv(1)
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 987, in recv
return self.read(buflen)
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 865, in read
return self._sslobj.read(len, buffer)
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 627, in read
v = self._sslobj.read(len)
OSError: [Errno 0] Error

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28689] OpenSSL 1.1.0c test failures

[Chi Hsuan Yen]

Changes by Chi Hsuan Yen :


--
nosy: +Chi Hsuan Yen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28688] Warning -- warnings.filters was modified by test_warnings

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

How you got this warning? I can't reproduce.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28689] OpenSSL 1.1.0c test failures

[Christian Heimes]

New submission from Christian Heimes:

OpenSSL 1.1.0c broke a bunch of tests. The same tests are passing fine with 
OpenSSL 1.1.0 to 1.1.0b. It looks like a problem with EOF / connection close 
error. I'm seeing similar problems in MIT KRB5's OpenSSL plugin, too.

==
ERROR: test_ciphers (test.test_ssl.SimpleBackgroundTests)
--
Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 1658, in 
test_ciphers
s.connect(self.server_addr)
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1093, in connect
self._real_connect(addr, False)
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1084, in _real_connect
self.do_handshake()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1061, in do_handshake
self._sslobj.do_handshake()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 683, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer

==
ERROR: test_connect (test.test_ssl.SimpleBackgroundTests)
--
Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 1483, in 
test_connect
s.connect(self.server_addr)
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1093, in connect
self._real_connect(addr, False)
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1084, in _real_connect
self.do_handshake()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1061, in do_handshake
self._sslobj.do_handshake()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 683, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer

==
ERROR: test_connect_cadata (test.test_ssl.SimpleBackgroundTests)
--
Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 1600, in 
test_connect_cadata
s.connect(self.server_addr)
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1093, in connect
self._real_connect(addr, False)
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1084, in _real_connect
self.do_handshake()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1061, in do_handshake
self._sslobj.do_handshake()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 683, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer

==
ERROR: test_connect_capath (test.test_ssl.SimpleBackgroundTests)
--
Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 1579, in 
test_connect_capath
s.connect(self.server_addr)
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1093, in connect
self._real_connect(addr, False)
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1080, in _real_connect
socket.connect(self, addr)
ConnectionRefusedError: [Errno 111] Connection refused

==
ERROR: test_connect_with_context (test.test_ssl.SimpleBackgroundTests)
--
Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 1541, in 
test_connect_with_context
s.connect(self.server_addr)
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1093, in connect
self._real_connect(addr, False)
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1084, in _real_connect
self.do_handshake()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1061, in do_handshake
self._sslobj.do_handshake()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 683, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer

==
ERROR: test_get_server_certificate (test.test_ssl.SimpleBackgroundTests)
--
Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 1645, in 
test_get_server_certificate
_test_get_server_certificate(self, *self.server_addr, cert=SIGNING_CA)
  File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 1830, in 
_test_get_server_certificate
pem = ssl.get_server_certificate((host, port), ca_certs=cert)
  File 

[issue28631] [2.7/3.5/3.6 Regression] crash using ctypes

[Matthias Klose]

Matthias Klose added the comment:

closing

--
resolution:  -> not a bug
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28631] [2.7/3.5/3.6 Regression] crash using ctypes

[Stuart Prescott]

Stuart Prescott added the comment:

Having added the suggested restype (plus a lot of others and also quite a few 
extra argtypes), we now have code that works across a few different versions of 
Python once again.

Thanks for the hint, eryksun and also thanks to doko for proxying the report 
and narrowing down the version range in the first place.

--
nosy: +themill

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24339] iso6937 encoding missing

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 14.11.2016 13:03, STINNER Victor wrote:
> 
> STINNER Victor added the comment:
> 
> iso6937.py:
> 
>> # from utf-8 to iso6937
>> def iso6937_encode(input,errors,encoding_map):
> 
> Wait, is this code for Python 3? Decode from UTF-8 and encode to ISO-6937 in 
> the same function seems strange to me.

The patch shows the file as UTF-8. In reality, it is decoding from
Unicode strings.

> I expected that the codec only implements two functions: encode text 
> (unicode) to ISO-6937 (bytes), decode bytes from ISO-6937 to text.
> 
> Since the encoding is non trivial (multibyte), if we decide to add it, I 
> suggest to require unit tests. I would like to see unit tests on multibyte 
> strings, to check how the error handler is handled.

+1

> In general, I would prefer to not embed too many codecs in Python, it has a 
> little cost to maintain these codecs.
> 
> My rule is more to only added encodings used (in practice) as locale 
> encodings.

This encoding is used in EPG data of various DVB television
formats. As such it is in active use (even though it is very old).

I think "active use" is a better approach to restricting
ourselves to only locale encodings, since the latter are
slowly converging towards UTF-8 :-)

BTW: Once a charmap style codec is written, there is little
change, so the maintenance is minimal. Codecs which include
more active logic such as this one are different, of course,
and therefore may potentially add more maintenance burden.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Julien Palard]

Julien Palard added the comment:

Hi Elliot, nice spot!

Why are you redefining Py_ABS, which looks already defined in `pymacro.h` 
included itself by `Python.h`? I'm not fan of undefining it later, it may 
surprise someone later expecting it to be there.

I tried to compile without your definition of Py_ABS, just in case I missed 
something in the includes, and it works.

--
nosy: +mdk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24339] iso6937 encoding missing

[Julien Palard]

Julien Palard added the comment:

@Serhiy @haypo: Popular enough or not, it may start as a lib on pypi, we'll see 
its usage from here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24339] iso6937 encoding missing

[STINNER Victor]

STINNER Victor added the comment:

@Serhiy: Do you think that the encoding is popular enough to pay the
price of its maintainance?

It's already possible to register manually a new encoding in an
application. It was even already possible in Python 2.7 (and older).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28684] [asyncio] bind() on a unix socket raises PermissionError on Android for a non-root user

[STINNER Victor]

STINNER Victor added the comment:

I suggest to write a decorator in Lib/asyncio/test_utils.py to skip AF_UNIX 
tests on Android if os.getuid() != 0.

Another approach is to catch the PermissionError and re-raises an 
unittest.SkipTest. Begin by modifying unix_socket_path() context manager in 
Lib/asyncio/test_utils.py.

I prefer the approach catching PermissionError: it allows to run unit tests on 
a flavor of Android which doesn't raise PermissionError, and it handles 
PermissionError on other platforms.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23839] Clear caches after every test

[STINNER Victor]

STINNER Victor added the comment:

Martin Panter:
"""
When I run the tests with -Werror (or any other -W option), I now get

2 tests altered the execution environment:
test___all__ test_warnings
"""

I confirm: I opened issue #28688 to track this issue (this issue is now closed).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28688] Warning -- warnings.filters was modified by test_warnings

[STINNER Victor]

New submission from STINNER Victor:

The issue #23839 modified the test runner to always clear caches before running 
tests. As a side effect, test_warnings started to complain with:

Warning -- warnings.filters was modified by test_warnings

The issue comes from the following function of test_warnings/__init__.py:

def setUpModule():
py_warnings.onceregistry.clear()
c_warnings.onceregistry.clear()

I suggest to rewrite this function as a setUp/tearDown method in BaseTest and 
*restores* the old value of these dictionaries.

I guess that the bug affects all Python versions, even if only Python 3.7 logs 
a warning.

--
components: Tests
messages: 280767
nosy: haypo, martin.panter, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Warning -- warnings.filters was modified by test_warnings
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28686] py.exe ignored PATH when using python3 shebang

[Eryk Sun]

Eryk Sun added the comment:

Virtual commands that are parsed to start with "python" are restricted to the 
list of installed versions of Python -- unless the virtual command allows 
searching and the Python command has no version specification (e.g.`#! 
/usr/bin/env python`). "python3" is versioned, so the launcher will not search 
for it using the default SearchPath path. 

For reference, see maybe_handle_shebang() [1], lines 1265-69:

command += 6;   /* skip past "python" */
if (search && ((*command == L'\0') || isspace(*command))) {
/* Command is eligible for path search, and there
 * is no version specification.
 */

The default SearchPath path includes the py.exe application directory, the 
process working directory, system directories, and the %Path% directories. This 
default search path can be made 'safe' via SetSearchPathMode or a registry 
setting, but doing so just swaps the order to search system directories before 
the working directory. It doesn't remove the working directory, unlike the safe 
search used by CreateProcess when %NoDefaultCurrentDirectoryInExePath% is 
defined. 

I think the implementation of find_on_path() could be improved. It should 
always try appending .EXE, .COM, .BAT, and .CMD if the bare command isn't 
found, not just if the command doesn't contain a ".". It's not unheard of to 
have a name with mulitple dots in it, e.g. "spam4.2.exe". Also, looping over 
all of the extensions listed in %PathExt% is of doubtful value. We don't call 
ShellExecuteEx to execute arbitrary file types, such as .VBS, .VBE, .JS, .JSE, 
.WSF, .WSH, and .MSC.

[1]: https://hg.python.org/cpython/file/v3.6.0b3/PC/launcher.c#l1112

--
nosy: +eryksun
versions: +Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24339] iso6937 encoding missing

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I think the encoder can just use codecs.charmap_encode(). The decoder seems 
could be simpler too.

Would be nice to generate the ISO 6937 encoding file from external data (e.g. 
from glibc localedata) like they are generated for other encodings. Take 
Tools/unicode/ files as a pattern.

Tests are required.

A number of lists of encodings should be updated: Doc/library/codecs.rst, 
Lib/encodings/aliases.py, Lib/locale.py, Lib/test/test_unicode.py, 
Lib/test/test_codecs.py, Lib/test/test_xml_etree.py.

--
stage:  -> needs patch
versions: +Python 3.7 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28618] Decorate hot functions using __attribute__((hot)) to optimize Python

[STINNER Victor]

STINNER Victor added the comment:

INADA Naoki added the comment:
> How about marking lookdict_unicode and lookdict_unicode_nodummy as hot?

I don't understand well the effect of the hot attribute, so I suggest
to run benchmarks and check that it has a non negligible effect on
benchmarks ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22495] merge large parts of test_binop.py and test_fractions.py

[STINNER Victor]

STINNER Victor added the comment:

> This requires quite a bit of work though for a relatively minor improvement 
> so do you think it's worth the effort ?

If you factorize code of unit tests, I expect better tests and more tests, so 
yes, it's valuable.

But since this issue is old (no activity last 2 years), I close the issue.

Please come back with a reviewable patch (reopen the issue, or open a new one, 
it's up to you), so it will be easier to discuss. The patch doesn't have to be 
complete, you can start by factorizing a single function just to discuss the 
principle and then complete the patch.

Note: I added our fractions experts to the issue while closing it :-)

--
nosy: +haypo, mark.dickinson, rhettinger
resolution:  -> out of date
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28629] Emit ResourceWarning when implicitly terminating a suspended frame?

[Nick Coghlan]

Nick Coghlan added the comment:

Oops, I forgot to mention one other potential cost-minimisation strategy for a 
`co_cleanuptab` field: only populate it with setup/cleanup opcode pairs that 
include a yield, yield from, or await operation between them. 

The potential benefit I can see to *not* doing that is that if the information 
is always available (even on synchronous frames), then greenlet based 
frameworks like gevent may be able to make use of it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24339] iso6937 encoding missing

[STINNER Victor]

STINNER Victor added the comment:

iso6937.py:

> # from utf-8 to iso6937
> def iso6937_encode(input,errors,encoding_map):

Wait, is this code for Python 3? Decode from UTF-8 and encode to ISO-6937 in 
the same function seems strange to me.

I expected that the codec only implements two functions: encode text (unicode) 
to ISO-6937 (bytes), decode bytes from ISO-6937 to text.

Since the encoding is non trivial (multibyte), if we decide to add it, I 
suggest to require unit tests. I would like to see unit tests on multibyte 
strings, to check how the error handler is handled.

--

In general, I would prefer to not embed too many codecs in Python, it has a 
little cost to maintain these codecs.

My rule is more to only added encodings used (in practice) as locale encodings.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[STINNER Victor]

STINNER Victor added the comment:

The initial bug (performance regression in python_startup) is fixed. Thanks 
everyone for the nice discussion. I close the issue.

Please open new issues if you want to continue efforts on making CPython faster 
;-)

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24339] iso6937 encoding missing

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

Just as reference, here's the wikipedia page for the encoding:

https://en.wikipedia.org/wiki/ISO/IEC_6937

and this is the ISO document (as preview):

http://webstore.iec.ch/preview/info_isoiec6937%7Bed3.0%7Den.pdf

(from the German wikipedia page).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[STINNER Victor]

STINNER Victor added the comment:

Ethan Furman: "Does this mean we can put enum back in re?"

I checked manually: adding "import enum" to Lib/re.py has no more impact on 
"python_startup" benchmark.


Guido van Rossum: "Honestly now the basic startup time is under control I am 
fine with restoring the re enum. We can optimize that later."

I concur with Guido, so I reapplied Ethan's changeset. Enjoy readable str() on 
re flags ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28569] mock.attach_mock should work with any return value of patch()

[Michael Foord]

Michael Foord added the comment:

Oh, I see what you mean - an attribute that doesn't exist on the original. With 
autospec that should throw an exception (AttributeError) I think, yes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28569] mock.attach_mock should work with any return value of patch()

[Syed Suhail Ahmed]

Syed Suhail Ahmed added the comment:

But since autospec is set as True, then shouldnt attach_mock throw an
exception when called with an attribute that doesn't exist?

On Mon, Nov 14, 2016 at 5:12 PM, Michael Foord 
wrote:

>
> Michael Foord added the comment:
>
> It should be perfectly valid to use attach_mock with an attribute that
> doesn't already exist. Part of it's purpose is to create new attributes.
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28569] mock.attach_mock should work with any return value of patch()

[Michael Foord]

Michael Foord added the comment:

It should be perfectly valid to use attach_mock with an attribute that doesn't 
already exist. Part of it's purpose is to create new attributes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28082] re: convert re flags to (much friendlier) IntFlag constants

[Roundup Robot]

Roundup Robot added the comment:

New changeset 5fd69d4a93e0 by Victor Stinner in branch '3.6':
Issue #28637: Reapply changeset 223731925d06
https://hg.python.org/cpython/rev/5fd69d4a93e0

New changeset be66786e95de by Victor Stinner in branch '3.6':
Issue #28082: Add basic unit tests on re enums
https://hg.python.org/cpython/rev/be66786e95de

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[Roundup Robot]

Roundup Robot added the comment:

New changeset 5fd69d4a93e0 by Victor Stinner in branch '3.6':
Issue #28637: Reapply changeset 223731925d06
https://hg.python.org/cpython/rev/5fd69d4a93e0

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28629] Emit ResourceWarning when implicitly terminating a suspended frame?

[Nick Coghlan]

Nick Coghlan added the comment:

Martin raises an interesting point - the cases where ResourceWarning is likely 
to be appropriate are going to be those where the frame has been suspended 
inside a with statement or a try/finally statement.

However, we don't currently track those at runtime or at compile time in any 
way - they're implicit in the state of the frame stack.

That doesn't mean we *couldn't* track them though, and I think co_lnotab 
provides a hint as to how we could do it with minimal runtime overhead: add a 
new co_cleanup data table to code objects that provides efficiently packed 
storage for a sequence of "setup/cleanup" bytecode offset 2-tuples.

All code objects would gain an additional pointer, and those that used with and 
try/finally would get fractionally larger than that, but frame objects would 
remain the same size they are today.

Glossing over the complexities of large bytecode offsets (ala co_lnotab), 
consider:

>>> def g_with():
... with ...:
... pass
... 
>>> dis.dis(g_with)
  2   0 LOAD_CONST   1 (Ellipsis)
  3 SETUP_WITH   5 (to 11)
  6 POP_TOP

  3   7 POP_BLOCK
  8 LOAD_CONST   0 (None)
>>   11 WITH_CLEANUP_START
 12 WITH_CLEANUP_FINISH
 13 END_FINALLY
 14 LOAD_CONST   0 (None)
 17 RETURN_VALUE

Here, co_cleanup would include a single 2-tuple with '0x03' as an absolute 
offset (for the setup) and '0x08' as a relative offset (for the cleanup). If 
'(f_lasti - 0x03) < 0x08' then __del__ on a generator-iterator or coroutine 
would raise ResourceWarning.

>>> def g_finally():
... try:
... pass
... finally:
... pass
... 
>>> dis.dis(g_finally)
  2   0 SETUP_FINALLY4 (to 7)

  3   3 POP_BLOCK
  4 LOAD_CONST   0 (None)

  5 >>7 END_FINALLY
  8 LOAD_CONST   0 (None)
 11 RETURN_VALUE

Here, co_cleanup would include a single 2-tuple with '0x00' as an absolute 
offset (for the setup) and '0x07' as a relative offset (for the cleanup). If 
'(f_lasti - 0x00) < 0x07' then __del__ on a generator-iterator or coroutine 
would raise ResourceWarning.

The key is that *only* __del__ would get slower (not explicit close() calls), 
and it would only get slower in cases where the frame was still suspended *and* 
co_cleanup was populated.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24459] Mention PYTHONFAULTHANDLER in the man page

[Berker Peksag]

Berker Peksag added the comment:

Thanks for the updated patch, Joshua. I will review and commit it this week.

--
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28673] When using pyro4 with more than 15 threads, python 2.7.12 cores frequently

[INADA Naoki]

INADA Naoki added the comment:

Traceback tells:

#22 0x005a52db in PyInterpreterState_Clear (interp=0x95ac80) at 
../Python/pystate.c:111
Py_CLEAR(interp->sysdict);

While cleaning up interp->sysdict...

#19 0x004d86c6 in tb_dealloc.46270 (tb=0x7f0aa1278f38) at 
../Python/traceback.c:28
#20 0x004d87e1 in tb_dealloc.46270 (tb=0x7f0aa1278ea8) at 
../Python/traceback.c:27

Removing some traceback from sysdict. Maybe, sys.exc_traceback.

#17 0x004c56e7 in subtype_dealloc.25650 (

self=) at remote 
0x7f0a9e924190>, objectsById={'root': 
, _started_at=, 
cookie='cookie_9de0e8d93ed84452a1ce8cc92268979e', _com
 plete=<_Condition(_Condition__lock=...(truncated)) at 
../Objects/typeobject.c:1035
#18 0x00534b93 in frame_dealloc.14921 (f=Frame 0x7f0a9f6d49d8, for file 
/etc/remoting/remoting_agent.zip/Pyro4/socketserver/threadpool.py, line 39, in 
run ())
at ../Objects/frameobject.c:458

Removing frame object owned by the traceback.

...

#7  PyEval_EvalFrameEx (f=f@entry=Frame 0x7f0a4b50, for file 
/etc/remoting/remoting_agent.zip/Pyro4/socketutil.py, line 453, in __del__ 
(self=),
throwflag=throwflag@entry=0) at ../Python/ceval.c:2987
#4  PyEval_EvalFrameEx (f=f@entry=Frame 0x7f0a9e929b60, for file 
/etc/remoting/remoting_agent.zip/Pyro4/socketutil.py, line 463, in close 
(self=),
throwflag=throwflag@entry=0) at ../Python/ceval.c:3251

Calling SocketConnection.__del__.  It calls SocketConnection.close.

#3  0x0055acc0 in set_exc_info (tb=, value=exceptions.AttributeError("'NoneType' object has no 
attribute 'SHUT_RDWR'",),
type=, tstate=0x95ad10) at ../Python/ceval.c:3736

SocketConnection.close raise AttributeError.

#2  PySys_SetObject (name=name@entry=0x61d02a "exc_type", v=v@entry=) at ../Python/sysmodule.c:83

Trying to register the exception to sys.

#0  PyDict_SetItem (op=op@entry=0x0, key=key@entry='exc_type', 
value=value@entry=) at ../Objects/dictobject.c:832

But interp->sysdict is NULL already.

--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27050] Demote run() below the high level APIs in subprocess docs

[Nick Coghlan]

Nick Coghlan added the comment:

Indeed, further discussion showed that what they were after was something more 
along the lines of sarge.Capture: 
http://sarge.readthedocs.io/en/latest/overview.html#main-features

That is, the ability to start the subprocess running in the background, and 
access the output line-by-line, but with the precise mechanics of how that 
works being a hidden implementation detail (just as concurrent.futures hides 
the details of the inter-process communication for function calls).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28618] Decorate hot functions using __attribute__((hot)) to optimize Python

[INADA Naoki]

INADA Naoki added the comment:

How about marking lookdict_unicode and lookdict_unicode_nodummy as hot?

--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28687] Python 2.7.12 windows x64 installer fails after previous bad uninstall

[Igor Skochinsky]

New submission from Igor Skochinsky:

It's somewhat my fault but I think it's not such an uncommon scenario.
I had installed 2.7.10 x64 some time ago into C:\Python27_64 but then had to 
delete it to save space. Rather foolishly instead of using the uninstaller, I 
just trashed the directory and now I have a problem: neither the Control Panel 
uninstall item, nor the fresh 2.7.12 installer work. The latter fails when it 
tries to use the nonexisting Python to remove pip:

MSI (s) (04:B0) [11:00:09:881]: Product: Python 2.7.10 (64-bit) -- Error 1721. 
There is a problem with this Windows Installer package. A program required for 
this install to complete could not be run. Contact your support personnel or 
package vendor. Action: RemovePip, location: C:\Python27_64\python.exe, 
command: -B -m ensurepip._uninstall 

In the UI just the first message is shown, without the failed commandline, so 
it's not at all clear what went wrong.

I think at the very least the user should be informed about the failed command 
so they can clean up the old installer's remains, or maybe the installation 
should be allowed to proceed anyway, replacing the bad paths in the registry.

In the end, I had to use "Msizap TWA {E2B51919-207A-43EB-AE78-733F9C6797C3}" 
after which I could install 2.7.12 into a separate directory

--
components: Installation
messages: 280747
nosy: Igor.Skochinsky
priority: normal
severity: normal
status: open
title: Python 2.7.12 windows x64 installer fails after previous bad uninstall
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21449] Replace _PyUnicode_CompareWithId with _PyUnicode_CompareWithIdEqual

[Xiang Zhang]

Xiang Zhang added the comment:

_PyUnicode_FromId could fail due to memoryerror or bad encoded data. They 
should be treated differently like PyUnicode_READY.

Could we treat it like PyDict_GetItem? If memoryerror happens it's highly 
possible other parts will fail too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28686] py.exe ignored PATH when using python3 shebang

[wdhwg001]

wdhwg001 added the comment:

And during the entire testing procedure, `C:\Users\wdhwg001\study\venv\Scripts` 
was manually added to user `PATH` and system `PATH`, but `py.exe` cannot find 
the hardlinked `python3.exe` in there.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28686] py.exe ignored PATH when using python3 shebang

[wdhwg001]

wdhwg001 added the comment:

But actually after I created a `python3.bat` into venv/Scripts, the shebang `#! 
/usr/bin/env python3` still does not take the `python3.bat`.

Then I created a hardlink from `python.exe` to `python3.exe`, the shebang still 
does not work.

I didn't check the code of py.exe, but it just behave like it may not search 
the PATH of current session correctly.

More details were updated on github:
https://github.com/pypa/virtualenv/issues/979

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28686] py.exe ignored PATH when using python3 shebang

[Paul Moore]

Paul Moore added the comment:

1. I don't think searching . should be included - on Unix /usr/bin/env searches 
PATH, and I believe we should do the same here.

2. The PATH search does happen (from my reading of the code) but it looks for a 
"python3" command, which isn't available. Again this is the same behaviour as 
Unix, and so defensible, but given that Windows doesn't provide the versioned 
executables, it's less useful there.

The biggest problem is that with "#!/usr/bin/env python3" the user clearly 
expects Python 3, and without versioned executables, we can't guarantee that on 
Windows for a PATH search. Whether not supporting this usage is worse than 
supporting it without a guarantee that you'll get Python 3, I'm not sure.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21449] Replace _PyUnicode_CompareWithId with _PyUnicode_CompareWithIdEqual

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

There are yet few subtle details.

1. _PyUnicode_FromId() uses UTF-8 for decoding from C string, but 
PyUnicode_CompareWithASCIIString() uses Latin1. Two ways of comparison can 
return different results. Currently all identifiers are ASCII, thus perhaps we 
can ignore this issue for a time. Perhaps the simplest solution is to make 
PyUnicode_FromId() using ASCII or Latin1.

2. PyUnicode_READY() can fail either because Unicode object is misformed or due 
to MemoryError. The former case is unavoidable error and returning false is 
good. But the latter can be temporary error and we should add a fallback, 
compare wchar_t * representation of non-ready Unicode object with char * 
representation of identifier.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24339] iso6937 encoding missing

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

Another comment about coding style: please use \u hex code representations 
for the decoding map. The stdlib source code is normally kept ASCII compatible 
and, for codecs, the Unicode code point numbers make it easier to check the 
mappings for correctness.

Thanks.

PS: You will also have to sign a contributor agreement: 
https://www.python.org/psf/contrib/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28686] py.exe ignored PATH when using python3 shebang

[wdhwg001]

New submission from wdhwg001:

https://github.com/pypa/virtualenv/issues/979

TL;DR

py.exe does not search PATH when using `#! /usr/bin/env python3`.

And the right steps I think should be:

1. Search ./
2. Search PATH
3. Fallback to system installed Python 3

But currently py.exe seems only do Step 3 once the shebang has substring of 
`python3`.

--
components: Windows
messages: 280740
nosy: paul.moore, steve.dower, tim.golden, wdhwg001, zach.ware
priority: normal
severity: normal
status: open
title: py.exe ignored PATH when using python3 shebang
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28023] python-gdb.py must be updated for the new Python 3.6 compact dict

[INADA Naoki]

INADA Naoki added the comment:

@haypo, would you review this patch?

--
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28629] Emit ResourceWarning when implicitly terminating a suspended frame?

[Martin Panter]

Martin Panter added the comment:

BTW my understanding is that currently, if the garbage collector deletes a 
generator, it effectively calls its close() method. When close() is called, it 
throws GeneratorExit into the suspended generator. This is meant to cause all 
the “with” and “try” statements to clean up.

So, yes, explicitly calling close() should immediately release the resources, 
call obj.__exit__(), etc. The idea of the ResourceWarning is to tell the 
programmer they forgot to call close().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com