[issue18844] allow weights in random.choice

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

What is wrong with generators?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] ceval: use Wordcode, 16-bit bytecode

[Brett Cannon]

Brett Cannon added the comment:

Thanks for the benchmark results, Demur, but I think the  benchmarks Victor was 
talking about hg.Python.org/benchmarks

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] ceval: use Wordcode, 16-bit bytecode

[Demur Rumed]

Demur Rumed added the comment:

To clarify format of extended arg listings: 1st column is the number of 
instances of EXTENDED_ARG being emitted, 2nd column is length of bytecode, 
followed by filename

The previous numbers were of modules, which generally are run-once and listing 
many constants. I've attached a modification where instead I iterated over the 
code objects inside co_consts from compiling the *.py file. Trunk currently 
only emits EXTENDED_ARGs for classes (Pdb, & then the rest in Lib/typing.py) so 
I've omitted it

--
Added file: http://bugs.python.org/file42325/exarg_in_funcs.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] ceval: use Wordcode, 16-bit bytecode

[Demur Rumed]

Demur Rumed added the comment:

I've attached some benchmarking results as requested

There is 1 failing test which doesn't fail in master for test_trace; the unit 
test for #9936

--
Added file: http://bugs.python.org/file42324/wcpybm.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26668] Remove Lib/test/test_importlib/regrtest.py?

[Brett Cannon]

Changes by Brett Cannon :


--
assignee:  -> brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26668] Remove Lib/test/test_importlib/regrtest.py?

[Brett Cannon]

Brett Cannon added the comment:

So that file wasn't to make sure  importlib was used by the test suite but  to 
actually make sure importlib.__import__ was heavily exercised (it's not the 
same as builtins.__import__ which is an accelerated C version). But thanks to 
test_importlib.util that shouldn't be an issue as long as all the modules in 
test_importlib.import_ use util.test_both(..., __import__=util.__import__).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26639] Tools/i18n/pygettext.py: replace deprecated imp module with importlib

[STINNER Victor]

STINNER Victor added the comment:

See also the issue #20021 '"modernize" makeopcodetargets.py"' (now closed).

--
nosy: +berker.peksag, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26666] File object hook to modify select(ors) event mask

[STINNER Victor]

STINNER Victor added the comment:

> The whole point of a transparent SOCKS module is that it provides a function 
> that's a *drop-in replacement* for socket.socket().

Fully transparent asynchronous I/O is an old dream of developers. You may try 
eventlet if you like monkey-patching.

It's a deliberate choice in the asyncio design to make asynchonous I/O explicit 
in the code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18844] allow weights in random.choice

[Raymond Hettinger]

Raymond Hettinger added the comment:

Thanks for the patch.   Can I get you to fill out a contributor agreement?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26668] Remove Lib/test/test_importlib/regrtest.py?

[STINNER Victor]

New submission from STINNER Victor:

Docstring of the file:

"""Run Python's standard test suite using importlib.__import__.

Tests known to fail because of assumptions that importlib (properly)
invalidates are automatically skipped if the entire test suite is run.
Otherwise all command-line options valid for test.regrtest are also valid for
this script.

"""

Since Python 3 now uses importlib by default for __import__, I think that the 
file became useless and can be removed.

--
messages: 262641
nosy: brett.cannon, eric.snow, haypo
priority: normal
severity: normal
status: open
title: Remove Lib/test/test_importlib/regrtest.py?
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26295] Random failures when running test suite in parallel (-m test -j0) caused by test_regrtest

[Roundup Robot]

Roundup Robot added the comment:

New changeset f7ddd72b70ea by Victor Stinner in branch 'default':
Issue #26295: When using "python3 -m test --testdir=TESTDIR", regrtest doesn't
https://hg.python.org/cpython/rev/f7ddd72b70ea

New changeset c83349c129c9 by Victor Stinner in branch 'default':
Issue #26295: test_regrtest now uses a temporary directory
https://hg.python.org/cpython/rev/c83349c129c9

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26662] configure/Makefile doesn't check if "python" command works, needed to build Objects/typeslots.inc

[Martin Panter]

Martin Panter added the comment:

Maybe the configure.ac code for generating opcode.h is a better alternative. It 
looks like it prints a warning but would continue with the build:

if test "$PYTHON" = not-found; then
OPCODEHGEN="@echo python: $PYTHON! cannot run 
Tools/scripts/generate_opcode_h.py"
else
OPCODEHGEN="$PYTHON"
fi

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26641] doctest doesn't support packages

[STINNER Victor]

STINNER Victor added the comment:

I pushed a different fix using module.__path__ since my use case is the test 
module splitted into at least two directories.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26641] doctest doesn't support packages

[Roundup Robot]

Roundup Robot added the comment:

New changeset 32539353eb95 by Victor Stinner in branch 'default':
doctest now supports packages
https://hg.python.org/cpython/rev/32539353eb95

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26662] configure/Makefile doesn't check if "python" command works, needed to build Objects/typeslots.inc

[STINNER Victor]

STINNER Victor added the comment:

> Typeslots.inc is listed in the .hgtouch file, so you might be able to work 
> around by running “make touch” before building.

My initial issue is that "import ssl" fails with : 
invalid slot offset. This error is really strange and Google doesn't know it.

I probably missed something when I compiled Python, but I would prefer that 
Python build system detects such issues for me, rather than having to use 
workarounds.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26666] File object hook to modify select(ors) event mask

[Martin Panter]

Martin Panter added the comment:

Python’s select() call operates at a minimal OS level. I don’t completely 
understand your __preselect__() etc proposal, but it does sound like you are 
trying to invent another higher-level “asynchronous” framework like asyncio.

I think you are wrong about the proxy connection being made in the socksocket() 
constructor. In general, there could be both write and read events depending on 
the stage of the proxy connection setup. E.g. with a HTTP proxy, there would be 
write events to establish the underlying TCP connection, and also to send the 
CONNECT request, and then a read event to receive the proxy’s response.

Perhaps the SSL socket wrapper objects face a similar situation to your 
problem. In non-blocking mode, SSLWantRead/WriteError exceptions are raised 
that indicate which select() event is required.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26662] configure/Makefile doesn't check if "python" command works, needed to build Objects/typeslots.inc

[Martin Panter]

Martin Panter added the comment:

Typeslots.inc is listed in the .hgtouch file, so you might be able to work 
around by running “make touch” before building.

I noticed there are other boostrap scripts that require a “python” command via 
various mechanisms, e.g. Python/makeopcodetargets.py is run via “#! 
/usr/bin/env python” by default. It would at least be nice to make everything 
uniform, i.e. always use a /usr/bin/env hashbang, always use $(PYTHON), or 
always use this smarter external Python command.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26649] Fail update installation: 'utf-8' codec can't decode

[STINNER Victor]

Changes by STINNER Victor :


--
resolution:  -> third party
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26659] slice() leaks memory when part of a cycle

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26666] File object hook to modify select(ors) event mask

[Zack Weinberg]

Zack Weinberg added the comment:

> Obvious question: why not working on the asyncio support in this library?

The whole point of a transparent SOCKS module is that it provides a function 
that's a *drop-in replacement* for socket.socket().  An asyncio-based SOCKS 
module would have a completely different API.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26662] configure/Makefile doesn't check if "python" command works, needed to build Objects/typeslots.inc

[STINNER Victor]

Changes by STINNER Victor :


--
components: +Build

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26661] python fails to locate system libffi

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26662] configure/Makefile doesn't check if "python" command works, needed to build Objects/typeslots.inc

[STINNER Victor]

Changes by STINNER Victor :


--
title: FreeBSD: "invalid slot offset" error on _ssl, _curses_panel and 
_testmultiphase extensions -> configure/Makefile doesn't check if "python" 
command works, needed to build Objects/typeslots.inc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25911] Regression: os.walk now using os.scandir() breaks bytes filenames on windows

[STINNER Victor]

STINNER Victor added the comment:

The issue looks to be fixed in Python 3.5 (future version 3.5.2) and 3.6, I 
close the issue.

I repeat myself: On Python 3, don't store filenames as bytes, it's a bad 
practice. Use unicode, it works on all platforms and gives access to the full 
Unicode Character Set!

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26654] asyncio is not inspecting keyword arguments of functools.partial

[STINNER Victor]

STINNER Victor added the comment:

> Oh, wow. I didn't even know we special-case partial() there.

I wrote the code doing that. I did it to get more readable and shorter logs 
since asyncio produces a lot of logs.

Example:

Exception in callback print("Hello", "World!")() at ...:4

without special case:

Exception in functools.partial(, 'Hello', 
'World!') at ...:4

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26624] Windows hangs in call to CRT setlocale()

[STINNER Victor]

STINNER Victor added the comment:

> The deadlock is between a CRT thread initialization lock and Windows's 
> process-wide module initialization lock. So if anything is being loaded on 
> demand by the watchdog thread, it's probably locked too (and that could be 
> within the CRT or the OS, so not easy to identify).

Basically, on timeout, the faulthandler thread uses the write() syscall and 
reads memory in the Python process. It's designed to be super safe. The code 
only uses async-signal safe functions, since the same code is used in signal 
handlers.

https://hg.python.org/cpython/file/688c6b16228e/Modules/faulthandler.c#l540

But the watchdog is trigerred by PyThread_acquire_lock_timed() which indirectly 
calls WaitForSingleObjectEx() on Windows. In expect that the wait times out 
before the lock is released on this buildbot.

Maybe Windows provides a safer function to implement a watchdog? Basically, the 
required function is a wait that can be interrupted.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26666] File object hook to modify select(ors) event mask

[STINNER Victor]

STINNER Victor added the comment:

> I'm attempting to enhance a transparent-SOCKS module 
> (https://github.com/Anorov/PySocks) to support non-blocking connect().

Obvious question: why not working on the asyncio support in this library?

--
components: +asyncio
nosy: +gvanrossum, haypo, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26488] hashlib command line interface

[STINNER Victor]

STINNER Victor added the comment:

> I think we should add a dependency in issue 14156 for this issue.

Maybe you can start using strings for filename parameters and open files 
manually, and later consider switching back to FileType once it's fixed?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26488] hashlib command line interface

[STINNER Victor]

STINNER Victor added the comment:

About the compatibility with existing tools, I recall a discussion when the 
tarfile module got a CLI. First I expected a clone of the UNIX tar command, but 
it was decided to design a new *simpler* CLI.

---
$ python3 -m tarfile
usage: tarfile.py [-h] [-v] [-l  | -e  [ ...] |
  -c  [ ...] | -t ]

A simple command line interface for tarfile module.

optional arguments:
  -h, --helpshow this help message and exit
  -v, --verbose Verbose output
  -l , --list 
Show listing of a tarfile
  -e  [ ...], --extract  [ ...]
Extract tarfile into target dir
  -c  [ ...], --create  [ ...]
Create tarfile from sources
  -t , --test 
Test if a tarfile is valid
---


A common trap of the md5sum CLI is that users write "echo string|md5sum" which 
adds a newline to string. For an unknown reason, my french manual page of the 
md5sum command has a -s STRING/--string=STRING argument, but not my effective 
md5sum program. Maybe we should consider adding such option to avoid the trap?


Do you want to implement a function to compare computed hash to a file which 
contains the expected hash? Check for file integrity, md5sum -c 
FILE/--check=FILE. Example:
--
$ md5sum test_socket_with.patch > check
$ cat check 
cfc1d69e76c827c32af4f28f50714a5e  test_socket_with.patch

$ md5sum -c check
test_socket_with.patch: OK

$ vim test_socket_with.patch 


$ md5sum -c check
test_socket_with.patch: FAILED
md5sum: WARNING: 1 computed checksum did NOT match
--


I worked hard to release the GIL when a hash is released. It would be super 
cool (a killer feature?) to automatically spawn threads to compute the hash. 
For example, use N threads where N is the number of CPU (os.cpu_count() or 1). 
Last time I wrote my md5sum.py, it was much faster than the UNIX md5sum tool 
since it uses all my CPU cores. You should just ensure that output is written 
in the correct order.


Raymond wrote:
> 1) Neither the md5 or shasum command-line tools offer control over the 
> blocksize.  I suggest that option be dropped from the command-line API giving 
> a nice simplification and usability improvement.

I agree. You should compute it per file using os.stat().st_blksize:

   https://docs.python.org/dev/library/os.html#os.stat_result.st_blksize

The io module uses st_blksize if it is greater than 1, or 8 * 1024 bytes.

(By the way, it looks like shutil.copyfile() doesn't use st_blksize.)

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] ceval: use Wordcode, 16-bit bytecode

[STINNER Victor]

Changes by STINNER Victor :


--
title: Wordcode -> ceval: use Wordcode, 16-bit bytecode

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18844] allow weights in random.choice

[Steven Basart]

Steven Basart added the comment:

The entire function of weighted choice. I removed the generator and replaced it 
by adding an optional argument to specify an amount by which you want to call 
this function.

--
Added file: http://bugs.python.org/file42323/weighted_choice_v3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] Wordcode

[Demur Rumed]

Changes by Demur Rumed :


--
nosy: +abarnert

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18844] allow weights in random.choice

[Steven Basart]

Steven Basart added the comment:

Reopen this idea but removing the generator from weighted choice.

--
nosy: +progressive-o
Added file: http://bugs.python.org/file42322/weighted_choice_v3.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] Wordcode

[STINNER Victor]

STINNER Victor added the comment:

> The change is to have all instructions take an argument. This removes the 
> branch on each instruction on whether to load oparg. (...)

Oh ok, I like that :-) I had the same idea.

Your patch contains unrelated changes, you should revert them to have a change 
simpler to review.

Removing HAVE_ARGUMENT from opcode.h/dis.py doesn't seem like a good idea. IMHO 
it's stil useful for dis to show a more compact bytcode. For example, I expect 
"DUP_TOP", not "DUP_TOP 0", or worse "DUP_TOP 5".

For backward compatibility, I also suggest to keep HAS_ARG() even if it must 
not be used to decode instructions anymore.

The following obvious change is to use a pointer aligned to 16-bits for co_code 
to be able to use 16-bit instructions rather than two 8-bit instructions to 
retrieve the opcode and then the argument in ceval.c (see the issue #25823). I 
suggest to implement that later to keep the change as simple as possible.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26219] implement per-opcode cache in ceval

[STINNER Victor]

STINNER Victor added the comment:

See the issue #26647 which may make the implementation of this cache simpler.

See also my message about inline caching:
https://bugs.python.org/issue26647#msg262622

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] Wordcode

[STINNER Victor]

STINNER Victor added the comment:

> This also means that if I want to create something like a tracer that tracks 
> some information for each instruction, I can allocate an array of codesize/2 
> bytes, then index off of half the instruction index. This isn't currently 
> done in peephole.c, nor does this include halving jump opargs

There is something called "inline caching": put the cache between instructions, 
in the same memory block. Example of paper on CPython:

"Efficient Inline Caching without Dynamic Translation" by Stefan Brunthaler 
(2009)
https://www.sba-research.org/wp-content/uploads/publications/sac10.pdf

Yury's approach is a standard lookup table: offset => cache. In the issue 
#26219, he even used two tables: co->co_opt_opcodemap is an array mapping an 
instruction offset to the offset in the cache, then the second offset is used 
to retrieve cache data from a second array. You have 3 structures (co_code, 
co_opt_opcodemap, co_opt), whereas inline caching propose to only use one flat 
structure (a single array).

The paper promises "improved data locality and instruction decoding effciency".

but "The new combined data-structure requires significantly more space—two 
native machine words for each instruction byte. To compensate for the 
additional space requirements, we use a profiling infrastructure to decide when 
to switch to this new instruction encoding at run time."

Memory footprint and detection of hot code is handled in the issue #26219.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26667] Update importlib to accept pathlib.Path objects

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
dependencies: +Support Path objects in the posix module

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26606] logging.baseConfig is missing the encoding parameter

[Vinay Sajip]

Vinay Sajip added the comment:

But you can open a stream using the encoding you want and pass it as the 
stream= parameter to basicConfig(). Why does that not work for you?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26664] find a bug in activate.fish of venv of cpython3.6

[Vinay Sajip]

Vinay Sajip added the comment:

The change you mention was in response to Issue #26348. The patch supplied 
there introduced the $ characters you say aren't required. I'm not a fish user, 
but it would seem that "$__VENV_PROMPT__" would resolve to the *value* of the 
environment variable __VENV_PROMPT__, whereas "__VENV_PROMPT__" would just 
resolve to the literal string "__VENV_PROMPT__" which doesn't seem like what is 
wanted. Can you explain the reasoning by which you assert that the $ isn't 
wanted?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10682] With '*args' or even bare '*' in def/call argument list, trailing comma causes SyntaxError

[Mike Lissner]

Mike Lissner added the comment:

This is an old issue, but where I run into it frequently is when I use the 
format function and string interpolation. For example, this throws a 
SyntaxError:

"The name of the person is {name_first} {name_last}".format(
**my_obj.__dict__,
)

Because strings tend to be fairly long, it's pretty common that the arguments 
to format end up on their own line. 

I was always taught to use trailing commas in Python, and I'm fanatical about 
ensuring they're there. It's a smart part of the language that saves you from 
many bugs and much typing when copy/pasting/tweaking. 

This is the first time I've ever run into an implementation bug in CPython, and 
at least from the post on StackOverflow, this looks like the parser isn't 
obeying the grammar: 
https://stackoverflow.com/questions/16950394/python-why-is-this-invalid-syntax

--
nosy: +Mike.Lissner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26667] Update importlib to accept pathlib.Path objects

[Brett Cannon]

New submission from Brett Cannon:

We should go through importlib and update the various APIs to accept 
pathlib.Path objects and not just str.

--
messages: 262618
nosy: brett.cannon
priority: normal
severity: normal
status: open
title: Update importlib to accept pathlib.Path objects
type: enhancement
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26632] __all__ decorator

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

On Mar 28, 2016, at 04:26 PM, Serhiy Storchaka wrote:

>There is a helper in test.support that helps to add a test for __all__, and
>we slowly add these test for all modules. So this is not large issue for the
>stdlib.
>
>New module level names are added not very often. Keeping __all__ in sync is
>not the largest problem in the maintaining.

stdlib use is not really the point of this proposal.  It's for all those 3rd
party projects that use __all__.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18987] distutils.utils.get_platform() for 32-bit Python on a 64-bit machine

[Paul Morelle]

Changes by Paul Morelle :


--
nosy: +madprog

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] Wordcode

[Demur Rumed]

Demur Rumed added the comment:

I'll dig up benchmark results when I get home, but I'd be interested to get 
results on a less wannabe RISC CPU

The change is to have all instructions take an argument. This removes the 
branch on each instruction on whether to load oparg. It then also aligns 
instructions to always be 2 bytes rather than 1 or 3 by having arguments only 
take up 1 byte. In the case that an argument to an instruction is greater than 
255, it can chain EXTENDED_ARG up to 3 times. In practice this rarely occurs, 
mostly only for jumps, & abarnert measured stdlib to be ~5% smaller

The rationale is that this offers 3 benefits: Smaller code size, simpler 
instruction iteration/indexing (One may now scan backwards, as peephole.c does 
in this patch), which between the two results in a small perf gain (the only 
way for perf to be negatively impacted is by an increase in EXTENDED_ARGs, when 
I post benchmarking I'll also post a count of how many more EXTENDED_ARGs are 
emitted)

This also means that if I want to create something like a tracer that tracks 
some information for each instruction, I can allocate an array of codesize/2 
bytes, then index off of half the instruction index. This isn't currently done 
in peephole.c, nor does this include halving jump opargs

I've looked up the 'recent work to cache attribute/global lookup' issue I 
mentioned: http://bugs.python.org/issue26219
I believe that patch would benefit from this one, but it'd be better to get 
Yury's opinion that belief

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26666] File object hook to modify select(ors) event mask

[Zack Weinberg]

New submission from Zack Weinberg:

This is pretty esoteric, please bear with me.  I'm attempting to enhance a 
transparent-SOCKS module (https://github.com/Anorov/PySocks) to support 
non-blocking connect().  This means, you should be able to do this:

socks.set_default_proxy(socks.SOCKS5, "localhost")
s = socks.socksocket(socket.AF_INET, socket.SOCK_STREAM)
s.setblocking(False)
err = s.connect_ex(address)
if err == errno.EINPROGRESS:
   select.select([], [s], []) # returns when connection completes

Note the position of s in the select() arguments: the documented behavior of 
non-blocking connect(), at the operating system level, is that the socket will 
become *writable* when the connection resolves (whether successfully or not).  
However, in this case, under the hood, the socket is *already connected* -- to 
the proxy server -- after socksocket() returns.  When connect_ex() returns 
EINPROGRESS, the thing we're really waiting for is a SOCKS-protocol reply 
message, i.e. the socket needs to become *readable* before the application can 
continue.  An application that used the above code (with a hypothetical version 
of PySocks where this was supported) would get woken up immediately, since the 
initial SOCKS client->server message doesn't even come close to filling up the 
TCP send buffer.

There's no practical way to hide this divergence with the current library.  
What would be needed, I think, is a pair of new special methods on filelikes, 
which rewrite the set of events to listen for and the set of events to report.  
Hypothetical, for my use case:

def __preselect__(self, events):
if not self._connecting: return events
return selectors.EVENT_READ

def __postselect__(self, events):
if not self._connecting: return events
return selectors.EVENT_WRITE

(I'm using the high-level selectors.EVENT_* constants for illustration.  This 
needs to happen in the low-level select-module methods, because callers can't 
be expected to use selectors.)

There are a bunch of awkward corner cases to worry about with this, and I'm not 
even sure the feature is worth it, but I thought I'd write up the problem and 
see what other people think.

--
components: IO
messages: 262612
nosy: zwol
priority: normal
severity: normal
status: open
title: File object hook to modify select(ors) event mask
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26488] hashlib command line interface

[Raymond Hettinger]

Raymond Hettinger added the comment:

Overall, I'm +1 on the idea and think the patch looks good.  There are two 
issues to look at:

1) Neither the md5 or shasum command-line tools offer control over the 
blocksize.  I suggest that option be dropped from the command-line API giving a 
nice simplification and usability improvement.

The stdin option has issues:

$ md5
abc
0bee89b07a248e27c83fc3d5951213c1

$ python3.6 -m hashlib md5
abc
^D
^CTraceback (most recent call last):
  File "/Users/raymond/cpython/Lib/runpy.py", line 184, in _run_module_as_main
"__main__", mod_spec)
  File "/Users/raymond/cpython/Lib/runpy.py", line 85, in _run_code
exec(code, run_globals)
  File "/Users/raymond/cpython/Lib/hashlib.py", line 246, in 
main()
  File "/Users/raymond/cpython/Lib/hashlib.py", line 238, in main
data = args.file.read(args.block_size)
KeyboardInterrupt

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26488] hashlib command line interface

[SilentGhost]

SilentGhost added the comment:

I've left some comment on rietveld yesterday, not sure if you got the e-mail.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26488] hashlib command line interface

[Raymond Hettinger]

Raymond Hettinger added the comment:

Here is a quick check versus other command-line tools:

$ md5 xml_prolog.diff 
MD5 (xml_prolog.diff) = c30e1fe70651d5a472efe9598db70771
$ python3.6 -m hashlib md5 xml_prolog.diff 
c30e1fe70651d5a472efe9598db70771
$ shasum -a 1 xml_prolog.diff 
db5fd60f3002ed270fe4469c7f343aba515d9b1a  xml_prolog.diff
$ python3.6 -m hashlib sha1 xml_prolog.diff 
db5fd60f3002ed270fe4469c7f343aba515d9b1a
$ shasum -a 224 xml_prolog.diff 
d0d18eb3e4d71269a610308e51e49d8b4650134bf2757dd22d69430b  xml_prolog.diff
$ python3.6 -m hashlib sha224 xml_prolog.diff 
d0d18eb3e4d71269a610308e51e49d8b4650134bf2757dd22d69430b
$ shasum -a 256 xml_prolog.diff 
0bbd834589d5fc9e26e32e5c665de4a2fbd93ea3bb4688ea25ef1139cd152b09  
xml_prolog.diff
$ python3.6 -m hashlib sha256 xml_prolog.diff 
0bbd834589d5fc9e26e32e5c665de4a2fbd93ea3bb4688ea25ef1139cd152b09
$ shasum -a 384 xml_prolog.diff 
f627b6c2bdc95e1af00da2a4b5d3897284127d0e820963f60758e34e5cfeb64cb529bfb306789f73c91d58d7594a3f3b
  xml_prolog.diff
$ python3.6 -m hashlib sha384 xml_prolog.diff 
f627b6c2bdc95e1af00da2a4b5d3897284127d0e820963f60758e34e5cfeb64cb529bfb306789f73c91d58d7594a3f3b
$ shasum -a 512 xml_prolog.diff 
c93babfef5a25bb569e4fd5c6edbb6d5e1de92044576b68522abcd9d9a356eca6b791df97a08e5c75cd090f00d5f2f73d940b901796b190681d370e5da74e4d8
  xml_prolog.diff
$ python3.6 -m hashlib sha512 xml_prolog.diff 
c93babfef5a25bb569e4fd5c6edbb6d5e1de92044576b68522abcd9d9a356eca6b791df97a08e5c75cd090f00d5f2f73d940b901796b190681d370e5da74e4d8

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26488] hashlib command line interface

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
assignee:  -> rhettinger
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26624] Windows hangs in call to CRT setlocale()

[Steve Dower]

Steve Dower added the comment:

The deadlock is between a CRT thread initialization lock and Windows's 
process-wide module initialization lock. So if anything is being loaded on 
demand by the watchdog thread, it's probably locked too (and that could be 
within the CRT or the OS, so not easy to identify).

I'm still keen to get some version numbers off the build machine, mainly 
ucrtbased.dll. I'd add code to the build process to just dump them, but I can't 
get to that for at least a week right now, so if Jeremy can provide it we can 
rule in/out some possibilities.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26665] pip is not bootstrapped by default

[Axel Luttgens]

New submission from Axel Luttgens:

Relevant page:

https://docs.python.org/2/library/ensurepip.html

One may read:

In most cases, end users of Python shouldn’t need to invoke
this module directly (as pip should be bootstrapped by default),
but it may be needed if installing pip was skipped when
installing Python (or when creating a virtual environment) or
after explicitly uninstalling pip.

As far as I correctly read the configure file that comes with the 2.7.11 
tarball, the default is to behave as if --with-ensurepip=no had been passed.

As a result, that sentence fragment:

(as pip should be bootstrapped by default)

may appear somewhat ambiguous for those compiling Python themselves.

Perhaps could this one avoid some head scratching:

(as packagers are expected to have bootstrapped pip)

HTH,
Axel

--
assignee: docs@python
components: Documentation
messages: 262610
nosy: Axel, docs@python
priority: normal
severity: normal
status: open
title: pip is not bootstrapped by default
type: enhancement
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26664] find a bug in activate.fish of venv of cpython3.6

[SilentGhost]

Changes by SilentGhost :


--
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26664] find a bug in activate.fish of venv of cpython3.6

[鄭景文]

Changes by 鄭景文 :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26664] find a bug in activate.fish of venv of cpython3.6

[鄭景文]

Changes by 鄭景文 :


--
components: +Library (Lib)
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26664] find a bug in activate.fish of venv of cpython3.6

[鄭景文]

New submission from 鄭景文:

I use venv module to make a virtual environment in cpython3.6.
After it, I try to activate it with fish, but it give me an error, like the 
screenshot. 
I open the activate.fish and remove the $ sign in line 58 and 59, then the 
activate file works. 
I open the activate.fish of py3.5 venv, there isn't a $ sign in line 58 and 59, 
too. 
However, there is a bug issue which add the $ in front of the prompt. 
I'm not sure how this happen, but I think there should not be a $ there.

--
files: screenshot.png
messages: 262609
nosy: 鄭景文
priority: normal
severity: normal
status: open
title: find a bug in activate.fish of venv of cpython3.6
Added file: http://bugs.python.org/file42321/screenshot.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26654] asyncio is not inspecting keyword arguments of functools.partial

[iceboy]

iceboy added the comment:

Created a PR https://github.com/python/asyncio/pull/328. Please review. Thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25911] Regression: os.walk now using os.scandir() breaks bytes filenames on windows

[STINNER Victor]

STINNER Victor added the comment:

Since all test_os now pass again on 3.x buildbots (after a few months of red 
buildbots), I backported the _DummyDirEntry fixes to 3.5.

David:
> The 3.5 branch failure about the mis-matched tuple values from the walk is a 
> separate issue, (...)

I'm sorry, I don't understand you. The test_os failure was the same on Python 
3.5 and 3.6:


- ('@test_4312_tmp\\TEST1\\SUB2', ['broken_link', 'link'], ['tmp3'])
? --
+ ('@test_4312_tmp\\TEST1\\SUB2', ['link'], ['broken_link', 'tmp3'])
? ++

This error should now be fixed on Python 3.5 and 3.6.

Is there a remaining open issue related to os.scandir() (bytes or Unicode)?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25911] Regression: os.walk now using os.scandir() breaks bytes filenames on windows

[STINNER Victor]

STINNER Victor added the comment:

> BTW, with respect to changeset 2b25fa7e3b7a, I'm not sure it has quite the 
> right semantics for is_dir, at least not if it's supposed to parallel 
> os.path.isdir().  I believe that should return True for a symbolic link to a 
> directory, which it doesn't look like this change would, if is_symlink 
> happened to have been called first.  It's possible the semantics of how 
> _DummyDirEntry is used precludes that scenario, but it seems a little 
> fragile.  I'd probably just use lstat() for is_symlink, but otherwise not 
> touch is_dir.

I think that you misunderstood the code. The "use cache lstat" path is only 
taken if the file is *not* symbolic link.

I tested manually _DummyDirEntry on a symbolic link to a directory: _lstat is 
filled by the constructor, is_dir() *and* is_symlink() returns True. Both are 
not exclusive, is_dir() works as follow_symlinks=True, whereas is_symlink() 
works as follow_symlinks=False.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25911] Regression: os.walk now using os.scandir() breaks bytes filenames on windows

[Roundup Robot]

Roundup Robot added the comment:

New changeset c38ac7ab8d9a by Victor Stinner in branch '3.5':
Issue #25911: Backport os._DummyDirEntry fixes
https://hg.python.org/cpython/rev/c38ac7ab8d9a

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26643] regrtest: rework libregrtest.save_env submodule

[Roundup Robot]

Roundup Robot added the comment:

New changeset c5944a76697b by Victor Stinner in branch '3.5':
Issue #26643: Add missing shutil resources to regrtest.py
https://hg.python.org/cpython/rev/c5944a76697b

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26624] Windows hangs in call to CRT setlocale()

[STINNER Victor]

STINNER Victor added the comment:

Jeremy: Did you update your buildbot? It looks like the deadlock occurred again:
-
http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/7478/steps/test/logs/stdio

(...)
0:14:50 [398/400] test_concurrent_futures (101 sec) -- running: test_strptime 
(40 sec)
0:14:52 [399/400] test_ssl -- running: test_strptime (42 sec)

command timed out: 1200 seconds without output, attempting to kill
program finished with exit code 1
elapsedTime=2096.738000
-

By the way, I don't understand why the faulthandler watch dog thread is unable 
to kill the main process and dump a traceback. I recently added a 
"faulthandler.dump_traceback_later" call Lib/test/libregrtest/runtest_mp.py. It 
uses a timeout of 15 min, it looks like buildbot kills the process after a 
timeout of 20 minutes "without output".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26662] FreeBSD: "invalid slot offset" error on _ssl, _curses_panel and _testmultiphase extensions

[STINNER Victor]

STINNER Victor added the comment:

Oh ok, I understood the issue.
---   
[haypo@freebsd ~/prog/python/default]$ rm Objects/typeslots.inc 
[haypo@freebsd ~/prog/python/default]$ make
python ./Objects/typeslots.py < ./Include/typeslots.h > Objects/typeslots.inc
/bin/sh: python: not found
*** Error code 127

Stop.
make: stopped in /home/haypo/prog/python/default
---

I have "python2" and "python3" but no "python" on FreeBSD CURRENT. The 
configure script is smart enough to select "python3" as the ASDL generator, but 
it uses $(PYTHON) ("python") to build Objects/typeslots.inc.

We should use a smarter code to select the external "python" program to build 
Objects/typeslots.inc.

Moreover, when "make Objects/typeslots.inc" fails, it creates an empty 
Objects/typeslots.inc file:
---
Objects/typeobject.o: Objects/typeslots.inc
Objects/typeslots.inc: $(srcdir)/Include/typeslots.h 
$(srcdir)/Objects/typeslots.py
$(PYTHON) $(srcdir)/Objects/typeslots.py < 
$(srcdir)/Include/typeslots.h > Objects/typeslots.inc
---

The Objects/typeslots.inc should be written directly by Objects/typeslots.py.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26663] asyncio _UnixWritePipeTransport._close abandons unflushed writes

[Robert Smallshire]

Changes by Robert Smallshire :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26663] asyncio _UnixWritePipeTransport._close abandons unflushed writes

[Robert Smallshire]

New submission from Robert Smallshire:

Note: This report is based on a close reading of the asyncio code, rather than 
having experienced defective behaviour in test or production.

The documentation for BaseTransport.close() states: "Close the transport. If 
the transport has a buffer for outgoing data, buffered data will be flushed 
asynchronously."

The _UnixWritePipeTransport._close method, which is called by close() contains 
this code in Python 3.5.1

def _close(self, exc=None):
self._closing = True
if self._buffer:
self._loop.remove_writer(self._fileno)
self._buffer.clear()
self._loop.remove_reader(self._fileno)
self._loop.call_soon(self._call_connection_lost, exc)

In this context, _buffer is a list of bytes objects comprising 
yet-to-be-written data. Note that close() removes the writer if _buffer is 
*not* empty, so the buffered data will never be written.  I believe this 
conditional should be inverted, so the writer is removed only if the buffer is 
*empty*. So:

if not self._buffer:
self._loop.remove_writer(self._fileno)

Arguably though, a more robust and easier to read test would be to call 
get_write_buffer_size(), like this:

if self.get_write_buffer_size() == 0:
self._loop.remove_writer(self._fileno)

Note that in the event that the writer is not removed by _close(), the writer 
will
remove itself when the buffer does eventually become empty.

Assuming my analysis is correct, and a fix is made to close(), then abort() 
will also need to be modified to *not* flush buffered writes, as the 
documentation for abort() states "Close the transport immediately, without 
waiting for pending operations to complete. Buffered data will be lost."

--
components: asyncio
messages: 262601
nosy: Robert Smallshire, gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio _UnixWritePipeTransport._close abandons unflushed writes
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26662] FreeBSD: "invalid slot offset" error on _ssl, _curses_panel and _testmultiphase extensions

[STINNER Victor]

New submission from STINNER Victor:

I built Python 3.6 (default) on FreeBSD using:

./configure --cache-file=config_debug.cache --with-pydebug CFLAGS=-O0
make

I'm using FreeBSD current, uname -a:

FreeBSD freebsd 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r296485: Tue Mar  8 
07:04:36 UTC 2016 r...@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC 
 amd64

It looks like "cc" is clang LLVM compiler 3.8.0.

Extract of the make output:
---
[haypo@freebsd ~/prog/python/default]$ make
running build
running build_ext
INFO: Can't locate Tcl/Tk libs and/or headers
*** WARNING: importing extension "_testmultiphase" failed with : invalid slot offset
*** WARNING: importing extension "_ssl" failed with : 
invalid slot offset
*** WARNING: importing extension "_curses_panel" failed with : invalid slot offset
---

--
messages: 262600
nosy: haypo
priority: normal
severity: normal
status: open
title: FreeBSD: "invalid slot offset" error on _ssl, _curses_panel and 
_testmultiphase extensions
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25911] Regression: os.walk now using os.scandir() breaks bytes filenames on windows

[David Bolen]

David Bolen added the comment:

I'm including some comments here from an email thread I had with Victor about 
this issue on the Win8 buildbot, which led to his recent changeset 2b25fa7e3b7a.

The Win8 3.x failure (the 5 != 4 length error) was due to the revision of 
os._DummyDirEntry (introduced I believe in eb91d0387d59) using stat() rather 
than lstat() to check for a symbolic link, which couldn't work, which in turn 
failed to exclude the symbolic directory link from the walk in the test.

The 3.5 branch failure about the mis-matched tuple values from the walk is a 
separate issue, where the broken directory link is still counted as a directory 
(rather than a file).  It seems due to os.path.isdir() - in the older 
os._DummyDirEntry implementation - returning True for the broken link in the 
test, rather than False as on Unix.  That's because on Unix the underlying 
os.stat call fails, but on Windows isdir is replaced with nt._isdir which 
avoids os.stat but is just a shim for.  The success on Windows sort of makes 
sense since the target_is_directory parameter to os.symlink() was used to 
create the broken link in the test, so Windows knows it's a directory, even if 
the link target is missing.

If the bytes walk implementation needs to treat broken links like files on 
Windows (matching the non-bytes version), then it can't depend on isdir() 
failing.  The non-bytes version (using scandir()) works even on Windows since 
internally scandir appears to build up mode bits out of os.stat/os.lstat calls 
and never uses isdir.

Back-porting the 3.x implementation of DummyDirEntry would be one quick way to 
fix the 3.5 issue as it also avoids isdir().

BTW, with respect to changeset 2b25fa7e3b7a, I'm not sure it has quite the 
right semantics for is_dir, at least not if it's supposed to parallel 
os.path.isdir().  I believe that should return True for a symbolic link to a 
directory, which it doesn't look like this change would, if is_symlink happened 
to have been called first.  It's possible the semantics of how _DummyDirEntry 
is used precludes that scenario, but it seems a little fragile.  I'd probably 
just use lstat() for is_symlink, but otherwise not touch is_dir.

Finally, as to why this only showed up on the Win8 buildbot - it turns out 
that's the only machine where the tests could create a symbolic link and thus 
encounter the scenario - they were prevented on Win7 and Win10 under the normal 
buildbot user due to lack of privileges (which was a surprise to me).  That's 
also what was happening on Victor's local Win8 VM.  So the failing conditions 
were just being skipped.  The Win8 buildbot was itself sort of pure luck, as 
early on I must have set it up to run under an elevated (administrative) 
command prompt, probably due to trying to handle some other issue.  I've now 
done the same thing for Win10 so it began seeing the same failures in the last 
few tests.

--
nosy: +db3l

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26661] python fails to locate system libffi

[Robert Kuska]

New submission from Robert Kuska:

Python fails to locate system version of ffi when building. We delete bundled 
version of ffi and build process fail with following error:

error: [Errno 2] No such file or directory: 
'/builddir/build/BUILD/Python-3.5.1/Modules/_ctypes/libffi'

This is caused by setup.py@1993 checking only for '#define LIBFFI_H' within 
ffi.h.

Content of /usr/include/ffi.h:

> $ cat /usr/include/ffi.h
/* This file is here to prevent a file conflict on multiarch systems. */
#ifdef ffi_wrapper_h
#error "Do not define ffi_wrapper_h!"
#endif
#define ffi_wrapper_h

#if defined(__i386__)
#include "ffi-i386.h"
#elif defined(__powerpc64__)
#include "ffi-ppc64.h"
#elif defined(__powerpc__)
#include "ffi-ppc.h"
#elif defined(__s390x__)
#include "ffi-s390x.h"
#elif defined(__s390__)
#include "ffi-s390.h"
#elif defined(__x86_64__)
#include "ffi-x86_64.h"
#else
#error "The libffi-devel package is not usable with the architecture."
#endif

#undef ffi_wrapper_h

It is important to note that this content is Fedora specific. As a workaround 
we (Fedora) use attached patch.

Would it be plausible to accept this patch in upstream?

--
components: Build
files: 00184-ctypes-should-build-with-libffi-multilib-wrapper.patch
keywords: patch
messages: 262598
nosy: rkuska
priority: normal
severity: normal
status: open
title: python fails to locate system libffi
type: compile error
Added file: 
http://bugs.python.org/file42320/00184-ctypes-should-build-with-libffi-multilib-wrapper.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25805] Failure in test_pkgutil run from command-line

[SilentGhost]

Changes by SilentGhost :


--
nosy: +r.david.murray
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] Wordcode

[STINNER Victor]

STINNER Victor added the comment:

Sorry, I don't have the context. Can you please explain your change? What did 
you do? What is the rationale? Do you expect better performances? If yes, 
please run the Python benchmark suite and post results here. What is the new 
format of bytecode? etc.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26657] Directory traversal with http.server and SimpleHTTPServer on windows

[Xiang Zhang]

Xiang Zhang added the comment:

Url handling in http.server is not perfect and there have already been some 
issues talking about it, i.e, issue5714, issue14567.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26657] Directory traversal with http.server and SimpleHTTPServer on windows

[Thomas]

Thomas added the comment:

Martin Panter: Regarding the warning, you appear to be correct.
However, reading the source of http.server again made me notice
_url_collapse_path(path)
which seems to have some overlap with translate_path. Also it
crashes with an IndexError if path contains '..'.

Also, yes, python 2.7's SimpleHTTPServer is affected as well.

Discarding weird paths instead of trying to repair them would change semantics, 
but from a user perspective, it would be easier to understand what is going on, 
so I'd agree with that change.

Further, I agree that it would be nice if there was some library function to 
safely handle path operations.
The function you proposed in https://bugs.python.org/issue21109#msg216675 and 
https://bitbucket.org/vadmium/pyrescene/src/34264f6/rescene/utility.py#cl-217 
leaves handling path separators to the user. Maybe that should be handled as 
well?
The function withstood my fuzzing tests on windows, so it might be correct.
There is probably a good reason for disallowing paths that contain /dev/null 
but I don't know why. Could you add a word or two of documentation to explain?

A really high-level solution would be to do away with all the strings and 
handle paths properly as the structure that they represent instead of trying to 
fake all kinds of things with strings, but that is probably beyond the scope of 
this issue.

--
versions: +Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25911] Regression: os.walk now using os.scandir() breaks bytes filenames on windows

[Roundup Robot]

Roundup Robot added the comment:

New changeset 2b25fa7e3b7a by Victor Stinner in branch 'default':
Fix os._DummyDirEntry.is_symlink()
https://hg.python.org/cpython/rev/2b25fa7e3b7a

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26659] slice() leaks memory when part of a cycle

[SilentGhost]

Changes by SilentGhost :


--
nosy: +benjamin.peterson, georg.brandl, pitrou, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26660] tempfile.TemporaryDirectory() cleanup exception on Windows if readonly files created

[SilentGhost]

Changes by SilentGhost :


--
nosy: +georg.brandl
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26655] pathlib glob case sensitivity issue on Windows

[Udo Eberhardt]

Udo Eberhardt added the comment:

So this is a trade-off between consistent behavior and efficiency. My point of 
view is that glob is for enumerating matching files and it should consistently 
return the real file names. Typically glob will be called with a pattern like 
'*.txt' and it will have to iterate names anyway, right? In the special case 
that it is called with a literal name it could do the same to produce 
consistent results. A user who wants to check (more efficiently) if a literal 
name exists, can use Path.exists().

The statement in the doc could be: 
Note: To find the literal names in the file system, glob always enumerates 
files and directories. To check more efficiently whether a specific file 
exists, use exists().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26638] Avoid warnings about missing CLI options when building documentation

[Martin Panter]

Martin Panter added the comment:

FTR the warnings that I am fixing were apparently enabled in Sphinx 1.3.4, 
reverted in 1.3.6, and added in 1.4.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26633] multiprocessing behavior combining daemon with non-daemon children inconsistent with threading

[Robert Cope]

Changes by Robert Cope :


--
nosy: +rpcope1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26658] test_os fails when run on Windows ramdisk

[Eryk Sun]

Eryk Sun added the comment:

> I'm inclined to believe that this is a bug in the ImDisk device driver

Junctions, and other filesystem reparse points, are implemented by volume 
devices, not disk devices. NTFS and ReFS support reparse points, but FAT, 
FAT32, and exFAT do not. Does the current filesystem claim to support reparse 
points? Here's how to check this:

import ctypes
kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)

FILE_SUPPORTS_REPARSE_POINTS = 0x0080

def volume_supports_reparse_points(root_path):
flags = ctypes.c_uint()
if not kernel32.GetVolumeInformationW(root_path,
  None, 0, None, None,
  ctypes.byref(flags),
  None , 0):
raise ctypes.WinError(ctypes.get_last_error())
return bool(flags.value & FILE_SUPPORTS_REPARSE_POINTS)

For example:

>>> volume_supports_reparse_points('C:\\')
True

Win32JunctionTests creates a junction to the "Lib/test" directory in the 
current directory. It should create a temporary target directory instead of 
using "Lib/test". 

I installed ImDisk Virtual Disk 2.0.9 and was able to create a valid junction 
on an NTFS RAM disk. What are the steps required to reproduce the problem?

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25829] Mixing multiprocessing pool and subprocess may create zombie process, and cause program to hang.

[Robert Cope]

Changes by Robert Cope :


--
nosy: +rpcope1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16500] Add an 'atfork' module

[Robert Cope]

Changes by Robert Cope :


--
nosy: +rpcope1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com