[issue35748] urlparse library detecting wrong hostname leads to open redirect vulnerability

[Neeraj Sonaniya]

New submission from Neeraj Sonaniya :

Summary:
It have been identified that `urlparse` under `urllib.parse` module is 
detecting wrong hostname which could leads to a security issue known as Open 
redirect vulnerability.

Steps to reproduce the issue:

Following code will help you in reproducing the issue:

```
from urllib.parse import urlparse
x= 'http://www.google.com\@xxx.com'
y = urlparse(x)
print(y.hostname)
```

Output:
xxx.com

The hostname from above URL which is actually rendered by browser is : 
'https://www.google.com'.

In following browsers tested: (hostname detected as: https://www.google.com)

```
1. Chromium - Version 72.0.3626.7  - Developer Build
2. Firefox - 60.4.0esr (64-bit)
3. Internet Explorer - 11.0.9600.17843
4. Safari - Version 12.0.2 (14606.3.4)
```

--
components: Library (Lib)
files: Screenshot from 2019-01-16 12-47-22.png
messages: 333750
nosy: nsonaniya2010, orsenthil
priority: normal
severity: normal
status: open
title: urlparse library detecting wrong hostname leads to open redirect 
vulnerability
type: security
versions: Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file48058/Screenshot from 2019-01-16 
12-47-22.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: python package management confusion

[dieter]

dcs3spp via Python-list  writes:
> ...
> So to manage the development of private packages, e.g. wheels, I would have 
> to use my own private repository (something like devpi or a an alternative 
> cloud pypi subscription service) to store each private dependency that I have 
> written.

No, you do not need something like "devpi" (or similar).
Instead, you can set up a "virtualenv" (there is a Python package
which can build "virtualenv"s), and use "setuptool"'s "develop"
"setup" command to "install" links to the package sources you
currently have under development. "develop" will automatically
install external requirements via "pypi".

> ...
> However, if I wanted to take a step further and run a CI build using cloud 
> services(e.g. in a private gitlab.com repository) for a package that uses the 
> private packages, then presumably there is no access to the devpi repository 
> on my local system? So, alternatively when developing private Python packages 
> I either use requirements.txt or pay subscription for a private pypi cloud 
> repository and configure pip, setup.cfg on gitlab.com CI to reference it in 
> config files. When the CI build completes it pushes the package to the 
> private pypi repository. 

I assume that you will be able to build an appropriate "virtualenv"
in a CI build setup.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: get the terminal's size

[Karen Shaeffer]

That will tell you the terminal size at the time Python was started.


If the terminal size has changed while Python was running, those

environment variables will be wrong.  You need to use the TIOCGWINSZ

ioctl call:


http://www.delorie.com/djgpp/doc/libc/libc_495.html


And to detect the size changes (so you know _when_ you need to do the

above), you need to attach a signal handler for the WINCH signal.


Hi,

I'm running a python 3 interpreter on linux. I'm actually ssh'd into the
terminal

on a headless server. And so my terminal is my local laptop terminal
window, with

the python interpreter running on the remote linux box terminal,
communicating

over an ssh connection.


$ python3

Python 3.6.7 (default, Oct 22 2018, 11:32:17)

[GCC 8.2.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import shutil

>>> print(f"{shutil.get_terminal_size()}\n")

os.terminal_size(columns=118, lines=63)


>>> print(f"{shutil.get_terminal_size()}\n")

os.terminal_size(columns=133, lines=63)


>>> print(f"{shutil.get_terminal_size()}\n")

os.terminal_size(columns=118, lines=65)


>>> print(f"{shutil.get_terminal_size()}\n")

os.terminal_size(columns=118, lines=63)



With the python interpreter running on the remote terminal, I have resized

the terminal window on my local laptop several times. And each time, the
remote

python interpreter knows about the change, correctly printing the new size.
I

have done nothing with environment variables. I have not used a signal
handler

for the WINCH signal. It just works.


Karen.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue35537] use os.posix_spawn in subprocess

[Kyle Evans]

Kyle Evans  added the comment:

> * On FreeBSD, if setting posix_spawn() "attributes" or execute posix_spawn() 
> "file actions" fails, posix_spawn() succeed but the child process exits 
> immediately with exit code 127 without trying to call execv(). If execv() 
> fails, posix_spawn() succeed, but the child process exit with exit code 127.

Hi,

As a disclaimer, I'm a FreeBSD developer interested in making sure we're doing 
the right thing here. =)

May I ask what the above assessment is based on, and specifically what we need 
to address?

As far as I can tell, our implementation is as POSIX describes -- errors 
processing the file actions or attrs triggers a 127 exit [1][2] which get 
bubbled up via the return value to posix_spawn [3]. exec failures capture errno 
at [4] and bubble the error up to the return value of posix_spawn as well via 
[3]. POSIX explicitly does not require an implementation to use errno for this, 
only return values, and we seem to have gone the route of not using errno to 
match OpenSolaris behavior.

What do I need to do to reproduce the results for deriving the results seen in 
the above quote, so that I can fix us and we can also see this improvement?

I threw together a minimal C reproducer for posix-spawn on -current (this 
particular bit being unchanged since FreeBSD 11.x times) and was returned 
ENOENT for a bad exec and otherwise given a pid for successful exec with a 
return of 0.

[1] 
https://svnweb.freebsd.org/base/head/lib/libc/gen/posix_spawn.c?view=markup#l214
[2] 
https://svnweb.freebsd.org/base/head/lib/libc/gen/posix_spawn.c?view=markup#l219
[3] 
https://svnweb.freebsd.org/base/head/lib/libc/gen/posix_spawn.c?view=markup#l232
[4] 
https://svnweb.freebsd.org/base/head/lib/libc/gen/posix_spawn.c?view=markup#l225

--
nosy: +kevans

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34782] Pdb crashes when code is executed in a mapping that does not define `__contains__`

[Henry Chen]

Change by Henry Chen :


--
nosy: +scotchka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35730] IDLE: Fix squeezer test_reload.

[Terry J. Reedy]

Change by Terry J. Reedy :


--
pull_requests:  -11182

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35730] IDLE: Fix squeezer test_reload.

[Terry J. Reedy]

Change by Terry J. Reedy :


--
pull_requests:  -11183

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: sampling from frequency distribution / histogram without replacement

[duncan smith]

On 15/01/2019 17:59, Ian Hobson wrote:
> Hi,
> 
> If I understand your problem you can do it in two passes through the
> population.
> 

The thing is that I start with the population histogram and I want to
generate a sample histogram. The population itself is too large to deal
with each population member individually.

> First, however, lets work through taking a sample of 2 from 7 to
> demonstrate the method.
> 
> Take the first element with a probability of 2/7. (Note 1).
> If you took it, you only want 1 more, so the probability drops to 1/6.
> If you didn't take it you want 2 from 6, so probability goes to 2/6.
> Take the next in the population with probability 1/6 or 2/6 as appropriate.
> Continue in similar manner until the probability
> drops to 0 (when you have your whole sample). When the
> denominator drops to zero the population is expired.
> 

Yes, based on the chain rule.

> Your first pass has to categorise the population and create your
> histogram, (index N) of frequencies Y(N).
> 
> Then divide up the sample size you wish to take into the histogram,
> giving array X(N) of sample sizes. X(N) need not be integer.
> 
> Then pass through the population again, for each entry:
>    Compute the N it falls in the histogram.
>    Take this entry as a sample with a probability of X(N)/Y(N).  Note 2.
>    If the element was taken, decrement X(N).
>    Decrement Y(N).
>    step to next element.
> 

Ah, I'm not quota sampling. I want a simple random sample without
replacement. I just happen to have the data in the form of categories
and frequencies, and that's the form of output that I want.

> Note 1 - In most languages you can generate a pseudo-random number
> with a uniform distribution from 0 to Y(N)-1. Take the element if it is
> in range 0 to floor(X(N))-1.
> 
> Note 2 - X(N) need not be integer, but you can't actually take a sample
> of 6.5 out of 1000. You will either run out of population having taken
> 6, or, if you take 7, the probability will go negative, and no more
> should be taken (treat as zero). The number taken in slot N will be
> floor(X(N)) or ceiling(X(N)). The average over many tries will however
> be X(N).

> Sorry I did not come back to you sooner. It took a while to drag the
> method out of my memory from some 35 years ago when I was working on an
> audit package. 

Well I'd already forgotten that I'd coded up something for srs without
replacement only a few years ago. In fact I coded up a few algorithms
(that I can't take credit for) that allowed weighted sampling with
replacement, and at least one that didn't require a priori knowledge of
the population size (a single pass algorithm). The problem is that they
also (mostly) require scanning the whole population.

That was where I learned two things you may be interested
> in.

> 1) Auditors significantly under sample. Our Auditors actually took
> samples that were between 10% and 25% of what was necessary to support
> their claims.
> 

It's not just auditors :-(. The journals are full of claims based on
positive results from low powered tests or from "null fields". i.e. A
very high proportion are likely to be false positives (like 99% when it
comes to foodstuffs and the risks of various diseases). A while ago a
mate of mine (Prof. of statistics in Oz) told me about a student who
engineered a statistically significant result by copying and pasting her
data to double her sample size. That's no worse than some of the stuff
I've come across in the (usually medical) journals.

> 2) Very very few standard pseudo-random number generators are actually
> any good.
> 
> Regards
> 
> Ian

[snip]

BTW, the approach I'm currently using is also based on the chain rule.
Generate the number of sample units for the first category by sampling
from a (bivariate) hypergeometric. The number of sample units for the
second category (conditional on the number sampled for the first) is
another hypergeometric. Iterate until the full sample is obtained. It
helps to order the categories from largest to smallest. But I think I'll
get better performance by recursive partitioning (when I have the time
to try it). Cheers.

Duncan
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue35692] pathlib.Path.exists() on non-existent drive raises WinError instead of returning False

[Eryk Sun]

Eryk Sun  added the comment:

> There's no reason a non-existing drive should fail differently than 
> a non-existing parent directory.

The drive exists (or should) if we're getting ERROR_NOT_READY (21). It's likely 
a removable media device, such as an optical disc or card reader, and there's 
no media in the device. 

If a logical drive isn't defined at all, we should get ERROR_PATH_NOT_FOUND 
(from the NT status value STATUS_OBJECT_PATH_NOT_FOUND). This gets mapped to 
the errno value ENOENT, which is already handled. For example:

>>> os.stat('Q:/')
Traceback (most recent call last):
  File "", line 1, in 
FileNotFoundError: [WinError 3] The system cannot find the path specified: 
'Q:/'

>>> pathlib.Path('Q:/whatever/blah.txt').exists()
False

Similarly if a UNC 'drive' doesn't exist, we should get ERROR_BAD_NET_NAME 
(from NT STATUS_BAD_NETWORK_NAME), which is also mapped to ENOENT:

>>> os.stat('//some/where')
Traceback (most recent call last):
  File "", line 1, in 
FileNotFoundError: [WinError 67] The network name cannot be found: 
'//some/where'

>>> pathlib.Path('//some/where/whatever/blah.txt').exists()
False

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[Ned Deily]

Ned Deily  added the comment:


New changeset 216a4d83c3b72f4fdcd81b588dc3f42cc461739a by Ned Deily (Miss 
Islington (bot)) in branch '3.6':
bpo-35746: Fix segfault in ssl's cert parser (GH-11569) (GH-11573)
https://github.com/python/cpython/commit/216a4d83c3b72f4fdcd81b588dc3f42cc461739a


--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35306] OSError [WinError 123] when testing if pathlib.Path('*') (asterisks) exists

[Eryk Sun]

Eryk Sun  added the comment:

> (sidenote: what os.path operation does Path.resolve() match? 
> Path('nonexistent').resolve() returns a relative path on Python 
> 3.7.1, whereas Path().resolve() returns an absolute path.)

pathlib should resolve 'nonexistent' in Windows. It works as expected in Unix:

>>> os.getcwd()
'/etc'
>>> os.fspath(Path('nonexistent').resolve())
'/etc/nonexistent'

A PR to implement ntpath.realpath is in development for issue 14094. The 
proposed implementation calls ntpath.abspath at the start, unless it's an 
extended path (i.e. prefixed by \\?\). Unlike Unix, Windows normalizes a path 
in user mode as a text operation before passing it to the kernel and file 
system. This means there's no problem if abspath removes a reparse point (e.g. 
symlink or mountpoint) when it resolves a ".." component.

> The code paths should be audited to check that EINVAL can't mean something 
> else.

We'd have to use the Windows error code (e.g. ERROR_INVALID_NAME) if it has to 
be specific. EINVAL is the default errno value. In particular, EINVAL includes 
some low-level device failures such as ERROR_IO_DEVICE and errors for 
operations that a device doesn't implement, which are commonly 
ERROR_INVALID_PARAMETER, ERROR_INVALID_FUNCTION, and ERROR_NOT_SUPPORTED. 

Also, a few device and files-system errors are mapped to EACCES (e.g. 
ERROR_NOT_READY and ERROR_SECTOR_NOT_FOUND). If we include EACCES, then files 
that exist but are inaccessible (e.g. the user isn't allowed to list the parent 
 directory) will be reported as not existing instead of raising an error. It's 
what os.path.exists does, but I guess pathlib wants to be more nuanced.

When using C runtime I/O (e.g. open, read, write), it can help to get the last 
Windows error code, _doserrno [1]. Its value gets set when errno is set by 
mapping an OS error. The last NT status value may also help in some cases. It 
gets set whenever an NT status code is mapped to a Windows error via 
RtlNtStatusToDosError (usually followed immediately by RtlSetLastWin32Error). 
It would be nice if OSError always included these two values, maybe as 
"last_winerror" (differentiated from "winerror") and "last_ntstatus".

For example, here's a case of trying to open a file on a CD drive that has no 
disk in it.

import ctypes

doserrno = ctypes.WinDLL('ucrtbase').__doserrno
doserrno.restype = ctypes.POINTER(ctypes.c_ulong)
doserrno.errcheck = lambda r, f, a: r[0]

get_last_nt_status = ctypes.WinDLL('ntdll').RtlGetLastNtStatus
get_last_nt_status.restype = ctypes.c_ulong

def test():
try:
open('D:\\test.txt')
except:
winerror, ntstatus = doserrno(), get_last_nt_status()
print('Windows error:', winerror)
print('NT status:', format(ntstatus, '#010x'))
raise

>>> test()
Windows error: 21
NT status: 0xc013
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 3, in test
PermissionError: [Errno 13] Permission denied: 'D:\\test.txt'

Windows error 21 is ERROR_NOT_READY, so we're already much better informed than 
EACCES (13). NT status 0xC013 is STATUS_NO_MEDIA_IN_DEVICE.

[1]: 
https://docs.microsoft.com/en-us/cpp/c-runtime-library/errno-doserrno-sys-errlist-and-sys-nerr?view=vs-2017

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

STINNER Victor  added the comment:

subprocess_bench_stdout.py: benchmark for PR 11575 using 
stdout=subprocess.PIPE, /usr/bin/pwd, and allocate 2 GiB of memory in the 
parent process. Result on my laptop:

Mean +- std dev: [fork_exec] 28.2 ms +- 0.3 ms -> [posix_spawn] 561 us +- 209 
us: 50.25x faster (-98%)

--
Added file: https://bugs.python.org/file48057/subprocess_bench_stdout.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +11248, 11249, 11250

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +11248, 11249

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +11248

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

STINNER Victor  added the comment:

More benchmarks.

I modified subprocess_bench.py to use:
ARGS = ["/usr/bin/python3", "-S", "-E", "-c", "pass"]
=> Mean +- std dev: [fork_exec] 34.1 ms +- 0.4 ms -> [posix_spawn] 6.85 ms +- 
0.08 ms: 4.97x faster (-80%)

Benchmark using:
ARGS = ["/usr/bin/python3", "-c", "pass"]
=> Mean +- std dev: [fork_exec] 44.5 ms +- 0.6 ms -> [posix_spawn] 17.2 ms +- 
0.2 ms: 2.58x faster (-61%)

Copy of the previous benchmark using /usr/bin/true:
Mean +- std dev: [fork_exec] 27.1 ms +- 0.4 ms -> [posix_spawn] 447 us +- 163 
us: 60.55x faster (-98%)

The benchmark is less impressive with Python which has a longer startup time (7 
to 17 ms depending on the -S option).

The speedup is between 2.6x (default) and 5.0x (-S option) faster for Python... 
to execute "pass" (do nothing).

In short, I understand that vfork removes a fixed cost of 27 ms which is the 
cost of duplicating the 2 GiB of memory pages.

The speedup depends on the memory footprint of the parent process and the 
execution time of the child process. The best speedup is when the parent is the 
largest and the child is the fastest.

--

Another benchmark, I modified subprocess_bench.py with:

BIG_ALLOC = b'x' * (10 * 1024 * 1024 * 1024)   # 10 GiB
ARGS = ["/bin/true"]

Mean +- std dev: [fork_exec] 139 ms +- 9 ms -> [posix_spawn] 420 us +- 208 us: 
331.40x faster (-100%)

Here the cost of copying 10 GiB of memory pages is around 138 ms. It's 331x 
faster ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[miss-islington]

miss-islington  added the comment:


New changeset 06b15424b0dcacb1c551b2a36e739fffa8d0c595 by Miss Islington (bot) 
in branch '2.7':
bpo-35746: Fix segfault in ssl's cert parser (GH-11569)
https://github.com/python/cpython/commit/06b15424b0dcacb1c551b2a36e739fffa8d0c595


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[STINNER Victor]

STINNER Victor  added the comment:

TALOS-2019-0758.txt: "Credit: Discovered by Colin Read and Nicolas Edet of 
Cisco."

Can we credit them somewhere? Maybe edit the NEWS entry to mention their name?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[miss-islington]

miss-islington  added the comment:


New changeset be5de958e9052e322b0087c6dba81cdad0c3e031 by Miss Islington (bot) 
in branch '3.7':
bpo-35746: Fix segfault in ssl's cert parser (GH-11569)
https://github.com/python/cpython/commit/be5de958e9052e322b0087c6dba81cdad0c3e031


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 9daecf37a571e98aaf43a387bcc9e41a7132f477 by Victor Stinner in 
branch 'master':
bpo-35537: subprocess uses os.posix_spawn in some cases (GH-11452)
https://github.com/python/cpython/commit/9daecf37a571e98aaf43a387bcc9e41a7132f477


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

STINNER Victor  added the comment:

Gregory P. Smith:
"""
Thanks for all your research and reference links on this!  As a 
_posixsubprocess maintainer, I am not against either posix_spawn or vfork being 
used directly in the future when feasible.

A challenge, especially with platform specific vfork, is making sure we 
understand exactly which platforms it can work properly on and checking for 
those both at compile time _and_ runtime (running kernel version and 
potentially the runtime libc version?) so that we can only use it in situations 
we are sure it is supposed to behave as desired in.  My guiding philosophy: Be 
conservative on choosing when such a thing is safe to use.
"""

About "My guiding philosophy: Be conservative on choosing when such a thing is 
safe to use.", I modified my PR 11452 to only use posix_spawn() on a very small 
subset of platforms where we know that the implementation is safe. It's 
different than early implementations which tried to use it as soon as it's 
available.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy:  -pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

STINNER Victor  added the comment:

Serhiy Storchaka:
> I mean that after writing tests they can be tested manually by disabling 
> conditions for posix_spawn one by one. I.e. some tests should fail if remove 
> "stdout is None" and some tests should fail if remove "not close_fds", etc.

I made some manual tests on my PR 11452. I changed close_fds default value from 
True to False. I also modified my change to use posix_spawnp using Joannah's PR 
11554 of bpo-35674.

The following tests fail *as expected*:

* test_close_fds_when_max_fd_is_lowered
* test_exception_errpipe_normal
* test_exception_errpipe_bad_data

The 2 errpipe tests mock subprocess to inject errors in the error pipe... but 
posix_spawn() doesn't expose its private "error pipe", so the test is not 
relevant for posix_spawn().

test_close_fds_when_max_fd_is_lowered() tests close_fds=True behavior. It's 
expected that it fails.

At least, I didn't notice any bug.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[miss-islington]

Change by miss-islington :


--
pull_requests: +11242, 11243, 11244, 11245, 11246, 11247

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[miss-islington]

Change by miss-islington :


--
pull_requests: +11242, 11243, 11244, 11245

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[miss-islington]

Change by miss-islington :


--
pull_requests: +11241, 11242, 11243

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[miss-islington]

Change by miss-islington :


--
pull_requests: +11242, 11243, 11244, 11245, 11247

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[miss-islington]

Change by miss-islington :


--
pull_requests: +11241, 11242

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[miss-islington]

Change by miss-islington :


--
pull_requests: +11242, 11243, 11244

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[miss-islington]

Change by miss-islington :


--
pull_requests: +11241

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[miss-islington]

miss-islington  added the comment:


New changeset a37f52436f9aa4b9292878b72f3ff1480e2606c3 by Miss Islington (bot) 
(Christian Heimes) in branch 'master':
bpo-35746: Fix segfault in ssl's cert parser (GH-11569)
https://github.com/python/cpython/commit/a37f52436f9aa4b9292878b72f3ff1480e2606c3


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[Larry Hastings]

Larry Hastings  added the comment:

I can confirm this crashes a freshly-built interpreter from the current 3.5 and 
3.4 branches.

--
nosy: +larry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35688] "pip install --user numpy" fails on Python from the Windows Store

[Steve Dower]

Steve Dower  added the comment:

> use AddDllDirectory, (which is as accessable as os.environ['PATH'] but is not 
> considered a security risk so far)

The parenthical is incorrect. The user-specified DLL search directory is 
separate from PATH, and both appear in the default DLL search order when 
resolving relative paths. In more secure configurations, PATH is not used for 
DLL resolution.

> but this requires using SetDefaultDllDirectories which breaks other things

Specifically, it breaks any extension relying on the implicit default search 
order by enabling one of the more secure configurations.

> put any dlls required for the c-extension pyd in the same directory which 
> means scipy and numpy will be using duplicate and potentially different 
> OpenBLAS dlls, and whoever imports first wins

Doesn't scipy import numpy? Which means numpy wins every time. Or 
alternatively, put "-numpy" in the name of numpy's one and "-scipy" in the name 
of scipy's one, and you can have both.

> load all the required dlls via LoadLibrary, meaning NumPy will have to export 
> a windows-only API to SciPy so the latter can know where the DLL is.

Perhaps that API could be exported via normal module import as is currently is? 
That way scipy can just "import numpy" to locate numpy?

Alternatively, if you do indeed need to have shared state with scipy, then you 
should come up with an API that they can depend on. This is how shared state 
normally works.

> Is there a PEP that describes the overall design of windows directory layout 
> or a design guide for package authors with best practices for additional dll 
> dependencies?

No, but there is a doc page that deserves an update: 
https://docs.python.org/3/extending/windows.html

If we make a dramatic change to CPython here, then there may be a PEP, but it 
should still defer to the documentation as that is what gets updated over time.

Currently, the best info comes from 
https://docs.microsoft.com/windows/desktop/Dlls/dynamic-link-library-search-order
 and awareness that only the LOAD_WITH_ALTERED_SEARCH_PATH flag is used when 
loading extension modules (see 
https://github.com/python/cpython/blob/master/Python/dynload_win.c#L221)


Since I just saw the confirmation at 
https://docs.microsoft.com/en-us/windows/desktop/Dlls/dynamic-link-library-search-order#search-order-using-load_library_search-flags,
 I don't think we can safely change the LoadLibraryEx option in CPython until 
we drop support for Windows 7 completely, as the update containing the new 
flags may not be installed. If/when we do that, it will break any extension 
relying on unsafe DLL search semantics (that is, anything appearing in the 
earlier section but not in this section).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23078] unittest.mock patch autospec doesn't work on staticmethods

[John Parejko]

John Parejko  added the comment:

Were you able to make any progress on this? Do you need any help?

--
nosy: +parejkoj-3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35713] Fatal Python error: _PySys_BeginInit: can't initialize sys module

[Tasy]

Tasy  added the comment:

Configuration Options: 

 ../configure --prefix=$HOME --enable-shared --enable-optimizations 
--with-system-expat --with-system-ffi --with-ensurepip=yes



Make throws the following warning:


*** WARNING: renaming "_curses_panel" since importing it failed: No module 
named '_curses'

Python build finished successfully!
The necessary bits to build these optional modules were not found:
_ssl  _uuid
To find the necessary bits, look in setup.py in detect_modules() for the 
module's name.


The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc  atexitpwd
time   


Failed to build these modules:
_curses


Following modules built successfully but were removed because they could not be 
imported:
_curses_panel  


Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with 
X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, 
https://github.com/libressl-portable/portable/issues/381

There ther is a following error...

0:06:18 load avg: 0.55 [171/416] test_hashlib
*** Error in `./python': corrupted size vs. prev_size: 0x0276b7a0 ***
Fatal Python error: Aborted

Current thread 0x2ba4468c7bc0 (most recent call first):
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/test_hashlib.py", line 
904 in _test_pbkdf2_hmac
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/test_hashlib.py", line 
935 in test_pbkdf2_hmac_c
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/unittest/case.py", line 615 
in run
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/unittest/case.py", line 663 
in __call__
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/unittest/suite.py", line 
122 in run
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/unittest/suite.py", line 84 
in __call__
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/unittest/suite.py", line 
122 in run
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/unittest/suite.py", line 84 
in __call__
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/unittest/suite.py", line 
122 in run
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/unittest/suite.py", line 84 
in __call__
  File 
"/usr/local/data/mySoftware/Python-3.7.2/Lib/test/support/testresult.py", line 
162 in run
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/support/__init__.py", 
line 1895 in _run_suite
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/support/__init__.py", 
line 1991 in run_unittest
  File 
"/usr/local/data/mySoftware/Python-3.7.2/Lib/test/libregrtest/runtest.py", line 
178 in test_runner
  File 
"/usr/local/data/mySoftware/Python-3.7.2/Lib/test/libregrtest/runtest.py", line 
182 in runtest_inner
  File 
"/usr/local/data/mySoftware/Python-3.7.2/Lib/test/libregrtest/runtest.py", line 
137 in runtest
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/libregrtest/main.py", 
line 407 in run_tests_sequential
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/libregrtest/main.py", 
line 514 in run_tests
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/libregrtest/main.py", 
line 615 in _main
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/libregrtest/main.py", 
line 582 in main
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/libregrtest/main.py", 
line 636 in main
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/regrtest.py", line 46 
in _main
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/regrtest.py", line 50 
in 
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/runpy.py", line 85 in 
_run_code
  File "/usr/local/data/mySoftware/Python-3.7.2/Lib/runpy.py", line 193 in 
_run_module_as_main
Aborted (core dumped)
make[1]: Leaving directory `/usr/local/data/mySoftware/Python-3.7.2/build'
make build_all_merge_profile
make[1]: Entering directory `/usr/local/data/mySoftware/Python-3.7.2/build'
true
make[1]: Leaving directory `/usr/local/data/mySoftware/Python-3.7.2/build'
# Remove profile generation binary since we are done with it.
make clean
make[1]: Entering directory `/usr/local/data/mySoftware/Python-3.7.2/build'
find .. -depth -name '__pycache__' -exec rm -rf {} ';'
find .. -name '*.py[co]' -exec rm -f {} ';'
find . -name '*.[oa]' -exec rm -f {} ';'
find . -name '*.s[ol]' -exec rm -f {} ';'
find . -name '*.so.[0-9]*.[0-9]*' -exec rm -f {} ';'
find build -name 'fficonfig.h' -exec rm -f {} ';' || true
find build -name '*.py' -exec rm -f {} ';' || true
find build -name '*.py[co]' -exec rm -f {} ';' || true
rm -f pybuilddir.txt
rm -f Lib/lib2to3/*Grammar*.pickle
rm -f Programs/_testembed Programs/_freeze_importlib
find build -type f -a ! -name '*.gc??' 

Re: get the terminal's size

[Cameron Simpson]

On 15Jan2019 13:08, Alex Ternaute  wrote:

I tried : P = Popen(['stty', '-a'], stdout=subprocess.PIPE,
universal_newlines=True) and it runs fine too, so the output seems not
really related to that fd.



But it is! stty(1) fetches the terminal settings from its standard
input, so "fd" is used to supply this. In your Popen test case you
simply don't set the stdin parameter, so it is the Python process'
input. Which is usually what you want.



But I want to be able to ask this
of any terminal file, thus the parameter.


Ah, Ok; smthlike:
cs.tty.ttysize(0)
WinSize(rows=50, columns=100)
anotherTty=open('/dev/pts/3', 'rw')
cs.tty.ttysize(anotherTty)
WinSize(rows=43, columns=199)

It runs :)


Exactly so.

BTW, you're aware of:

 from cs.tty import ttysize
 ...
 ttysize(0)

I presume, and the above is just your testing?

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

[issue35688] "pip install --user numpy" fails on Python from the Windows Store

[mattip]

mattip  added the comment:

It seems changing os.environ['PATH'] is a security risk and is not allowed for 
Windows Store apps. The suggestion in the NumPy issue is to:

- use AddDllDirectory, (which is as accessable as os.environ['PATH'] but is not 
considered a security risk so far), but this requires using 
SetDefaultDllDirectories which breaks other things

- put any dlls required for the c-extension pyd in the same directory which 
means scipy and numpy will be using duplicate and potentially different 
OpenBLAS dlls, and whoever imports first wins

- load all the required dlls via LoadLibrary, meaning NumPy will have to export 
a windows-only API to SciPy so the latter can know where the DLL is.

I am glad NumPy only has one DLL, and not a dozen like QT or wxPython. 

Is there a PEP that describes the overall design of windows directory layout or 
a design guide for package authors with best practices for additional dll 
dependencies?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[Ned Deily]

Change by Ned Deily :


Removed file: https://bugs.python.org/file48055/image001.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[Ned Deily]

Change by Ned Deily :


Removed file: https://bugs.python.org/file48054/image001.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29871] Enable optimized locks on Windows

[Josh Rosenberg]

Josh Rosenberg  added the comment:

I assume you meant #35662 (based on the superseder note in the history).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35306] OSError [WinError 123] when testing if pathlib.Path('*') (asterisks) exists

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I'm fine with swallowing the error in both exists() and resolve(). We should be 
careful not to swallow errors too broadly, though.  The code paths should be 
audited to check that EINVAL can't mean something else.

--
versions: +Python 3.8 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35692] pathlib.Path.exists() on non-existent drive raises WinError instead of returning False

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I think exists() should simply return False here.  There's no reason a 
non-existing drive should fail differently than a non-existing parent directory.

--
stage:  -> needs patch
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35306] OSError [WinError 123] when testing if pathlib.Path('*') (asterisks) exists

[Steve Dower]

Steve Dower  added the comment:

Pathlib doesn't necessarily directly follow os on its error handling - adding 
Antoine for comment.

Passing strict=False to resolve() should be able to handle an invalid name like 
that. If not, I propose that we change it so that it does.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29515] socket module missing IPPROTO_IPV6, IPPROTO_IPV4 on Windows

[Steve Dower]

Steve Dower  added the comment:

No progress, but I like the extra defines idea best (directly in 
socketmodule.c, not in a public header file). That's the easiest way to close 
the gap between (apparently) real constants used on Windows and the 
preprocessor defines (apparently) used elsewhere.

--
keywords: +easy (C)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29871] Enable optimized locks on Windows

[Steve Dower]

Steve Dower  added the comment:

On issue 35562 Jeff posted a deeper analysis of the issue in TIMEDWAIT. That 
will need fixing along with the other regressions before we can enable these.

--
nosy: +je...@livedata.com

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35662] Windows #define _PY_EMULATED_WIN_CV 0 bug

[Steve Dower]

Steve Dower  added the comment:

It's broken, but unused. And the entire section needs fixing before it can be 
used, which necessitates fixing this function. So issue 29871 covers this 
sufficiently (though I'll post a link back to this one for the added context on 
this particular issue).

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Enable optimized locks on Windows

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35692] pathlib.Path.exists() on non-existent drive raises WinError instead of returning False

[Steve Dower]

Steve Dower  added the comment:

In issue 22759 there was some logic applied for which errors to forward rather 
than hide.

I'm inclined to agree that this one should be hidden, but it may have to be 
done by checking the winerror field rather than the exception type, since other 
PermissionErrors may mean the file is guaranteed to exist (but you can't touch 
it) or that the path exists up to the point where you are not allowed to see.

I'd happily argue that since these permissions indicate that the file does not 
exist *for the current user* and so they should be swallowed more broadly, but 
I'll let Antoine make the call.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35688] "pip install --user numpy" fails on Python from the Windows Store

[Steve Dower]

Steve Dower  added the comment:

I posted on the numpy thread: Most likely the DLL is failing to load, which the 
importer returns as "not found" (as it falls back on other search mechanisms 
and doesn't retain the error). I suggested loading it directly with ctypes to 
see if there's a better error indicator.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35739] Enable verbose of tests during PGO build on amd64 platforms

[Steve Dower]

Steve Dower  added the comment:

You can provide this new default as a command line option when invoking the 
script (--pgo-job, IIRC), which should satisfy the occasional need to do this.

I would rather keep the default quieter so that the build does not take as long 
(though I guess there is the possibility that enabling more output produces a 
better profile, but I doubt it's significant).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: help with python difference between number

[MRAB]

On 2019-01-15 14:07, achyuta2...@gmail.com wrote:


M <01/14/2019 08:07:01> Count:0
Total:50
Free: 20
A
B
 M <01/14/2019 08:07:04> Count:1
Total:5
Free:10
A
B
M <01/14/2019 08:07:07> Count:2
Total:5
Free:3
A
B


I am trying to make a output like where it prints the free and then the 
difference between the current free and previous free
For e.g

M <01/14/2019 08:07:01> Count:0  Free: 20
M <01/14/2019 08:07:04> Count:1  Free: 10  absolute difference between time and 
prev time is -10
M <01/14/2019 08:07:07> Count:2  Free: 3   absolute difference between time and 
prev time is -7


And then later on i need to determine the time when we had the most negative 
free value.


I tried a code like this
Which printed
  with open("summ4.txt") as f:
# get first line/number
nxt = int(next(f))
for n in f:
 print("absolute difference between {} and {} = {}"
   .format(n.rstrip(), nxt, abs(int(nxt) - int(n
 # set nxt equal to the next number
 nxt = int(next(f,0))
a=open('summ1.txt','r').readlines()
b=open('summ3.txt','r').readlines()
 with open('summ.txt','w') as out:
   for i in range(0,365):
print>>out,a[i].rstrip(),b[i]


I hit error as
Traceback (most recent call last):
File "3.py", line 39, 
in 
   .format(n.rstrip(), 
nxt, abs(int(nxt) - int(n
ValueError: 
zero length field name in format

I guess my input file has a tab in the start and not able to get a difference 
rightly.
.
Any pointers on how to achieve the desired result?


You didn't say Which version of Python you're using.

The "print>>" tells me that it's Python 2.

It's complaining about the '{}' in the format string.

Format strings were introduced in Python 2.6 and auto-numbering ('{}' 
allowed instead of '{0}') was introduced in Python 2.7.


As it's complaining about a missing field name in '{}', it must be 
Python 2.6, which is ancient!


And Python 2.7 reaches its end of life soon.

You should switch to Python 3 unless you have a very good reason for 
staying on Python 2, and, if you must use Python 2, use Python 2.7.

--
https://mail.python.org/mailman/listinfo/python-list

[issue35661] Store the venv prompt in pyvenv.cfg

[Steve Dower]

Steve Dower  added the comment:

One other aspect of this may be the confusion that ensues when changing the 
setting doesn't change the prompt when you activate it.

It would be possible (though not necessarily trivial) to update the activate 
scripts to read the prompt from the file, though I don't think it's necessary. 
But we'll get a bug report sooner or later anyway.

Maybe if the setting is called "provided-custom-prompt" that will imply enough 
that it's a optional record of the prompt rather than an active configuration 
setting?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35747] Python threading event wait influenced by date change

[ido k]

ido k  added the comment:

thanks for the comment

please look at the code.
i use wait on event for 60 seconds.
the wait timed out in less than 60 seconds... 

why this is not a bug?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: sampling from frequency distribution / histogram without replacement

[Ian Hobson]

Hi,

If I understand your problem you can do it in two passes through the 
population.


First, however, lets work through taking a sample of 2 from 7 to 
demonstrate the method.


Take the first element with a probability of 2/7. (Note 1).
If you took it, you only want 1 more, so the probability drops to 1/6.
If you didn't take it you want 2 from 6, so probability goes to 2/6.
Take the next in the population with probability 1/6 or 2/6 as appropriate.
Continue in similar manner until the probability
drops to 0 (when you have your whole sample). When the
denominator drops to zero the population is expired.

Your first pass has to categorise the population and create your 
histogram, (index N) of frequencies Y(N).


Then divide up the sample size you wish to take into the histogram,
giving array X(N) of sample sizes. X(N) need not be integer.

Then pass through the population again, for each entry:
   Compute the N it falls in the histogram.
   Take this entry as a sample with a probability of X(N)/Y(N).  Note 2.
   If the element was taken, decrement X(N).
   Decrement Y(N).
   step to next element.

Note 1 - In most languages you can generate a pseudo-random number
with a uniform distribution from 0 to Y(N)-1. Take the element if it is 
in range 0 to floor(X(N))-1.


Note 2 - X(N) need not be integer, but you can't actually take a sample 
of 6.5 out of 1000. You will either run out of population having taken 
6, or, if you take 7, the probability will go negative, and no more 
should be taken (treat as zero). The number taken in slot N will be 
floor(X(N)) or ceiling(X(N)). The average over many tries will however 
be X(N).


Sorry I did not come back to you sooner. It took a while to drag the 
method out of my memory from some 35 years ago when I was working on an 
audit package. That was where I learned two things you may be interested in.


1) Auditors significantly under sample. Our Auditors actually took 
samples that were between 10% and 25% of what was necessary to support 
their claims.


2) Very very few standard pseudo-random number generators are actually 
any good.


Regards

Ian

On 14/01/2019 20:11, duncan smith wrote:

Hello,
   Just checking to see if anyone has attacked this problem before
for cases where the population size is unfeasibly large. i.e. The number
of categories is manageable, but the sum of the frequencies, N,
precludes simple solutions such as creating a list, shuffling it and
using the first n items to populate the sample (frequency distribution /
histogram).

I note that numpy.random.hypergeometric will allow me to generate a
sample when I only have two categories, and that I could probably
implement some kind of iterative / partitioning approach calling this
repeatedly. But before I do I thought I'd ask if anyone has tackled this
before. Can't find much on the web. Cheers.

Duncan



--
Ian Hobson
Tel (+351) 910 418 473
--
https://mail.python.org/mailman/listinfo/python-list

[issue35747] Python threading event wait influenced by date change

[STINNER Victor]

STINNER Victor  added the comment:

Python 3 uses a monotonic clock to implement timeouts, such clock is not 
affected by system clock changes *on purpose*. See time.monotonic() and PEP 418:
https://docs.python.org/dev/library/time.html#time.monotonic
https://www.python.org/dev/peps/pep-0418/

Relying on the system clock can cause severe bugs.

I suggest to close this issue as "not a bug".

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35747] Python threading event wait influenced by date change

[ido k]

New submission from ido k :

Happen on ubuntu 

Opening two threads - one thread alternate system date
The seconds waits for 60 seconds. joining both threads.

The execution should take at least 60 seconds. Takes less then 15 seconds.

Any work around?

--
components: Library (Lib)
files: wrong_wait_behaviour.py
messages: 333718
nosy: ido k
priority: normal
severity: normal
status: open
title: Python threading event wait influenced by date change
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file48056/wrong_wait_behaviour.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35736] [xml.minidom] Missing component in table after getElementsByTagName("nn")

[Brett Cannon]

Change by Brett Cannon :


--
title: Missing component in table after getElementsByTagName("nn") -> 
[xml.minidom] Missing component in table after getElementsByTagName("nn")

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34782] Pdb crashes when code is executed in a mapping that does not define `__contains__`

[BTaskaya]

Change by BTaskaya :


--
keywords: +patch, patch
pull_requests: +11239, 11240
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34782] Pdb crashes when code is executed in a mapping that does not define `__contains__`

[BTaskaya]

Change by BTaskaya :


--
keywords: +patch
pull_requests: +11239
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35661] Store the venv prompt in pyvenv.cfg

[Brett Cannon]

Brett Cannon  added the comment:

First, Cheryl, thanks for taking this on!

I think one way to potentially simplify this whole situation about the 
whitespace for the prompt is to actually store the raw value that gets passed 
into EnvBuilder instead of the prompt as formatted for the activation scripts. 
I personally only want that initial value anyway and not the formatted version 
for the prompt for us in VS Code. Plus if we document that the value that we 
save in the pyvenv.cfg will be stripped then that should help with this.

Otherwise I say go with the repr as Steve suggested, but I would still like to 
have access to the unformatted value (and probably not bother setting it if a 
custom value isn't provided to facilitate relocating venvs).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35701] [uuid] 3.8 breaks weak references for UUIDs

[David Heiberg]

Change by David Heiberg :


--
keywords: +patch, patch, patch
pull_requests: +11236, 11237, 11238
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35701] [uuid] 3.8 breaks weak references for UUIDs

[David Heiberg]

Change by David Heiberg :


--
keywords: +patch, patch
pull_requests: +11236, 11237
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35701] [uuid] 3.8 breaks weak references for UUIDs

[David Heiberg]

Change by David Heiberg :


--
keywords: +patch
pull_requests: +11236
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: sampling from frequency distribution / histogram without replacement

[duncan smith]

On 15/01/2019 02:41, Spencer Graves wrote:
> 
> 
> On 2019-01-14 18:40, duncan smith wrote:
>> On 14/01/2019 22:59, Gregory Ewing wrote:
>>> duncan smith wrote:
 Hello,
    Just checking to see if anyone has attacked this problem before
 for cases where the population size is unfeasibly large.
>>> The fastest way I know of is to create a list of cumulative
>>> frequencies, then generate uniformly distributed numbers and
>>> use a binary search to find where they fall in the list.
>>> That's O(log n) per sample in the size of the list once it's
>>> been set up.
>>>
>> That's the sort of thing I've been thinking about. But once I'd found
>> the relevant category I'd need to reduce its frequency by 1 and
>> correspondingly update the cumulative frequencies. Alternatively, I
>> could add an extra step where I selected a unit from the relevant
>> category with probability equal to the proportion of non-sampled units
>> from the category. I could maybe set up an alias table and do something
>> similar.
>>
>> The other thing I was thinking about was iterating through the
>> categories (ideally from largest frequency to smallest frequency),
>> generating the numbers to be sampled from the current category and the
>> remaining categories (using numpy.random.hypergeometric). With a few
>> large frequencies and lots of small frequencies that could be quite
>> quick (on average). Alternatively I could partition the categories into
>> two sets, generate the number to be sampled from each partition, then
>> partition the partitions etc. binary search style.
>>
>> I suppose I'll try the both the alias table + rejection step and the
>> recursive partitioning approach and see how they turn out. Cheers.
> 
> 
>   R has functions "sample" and "sample.int";  see
> "https://www.rdocumentation.org/packages/base/versions/3.5.2/topics/sample;.
> You can call R from Python,
> "https://sites.google.com/site/aslugsguidetopython/data-analysis/pandas/calling-r-from-python;.
> 
> 
> 
>   These are in the "base" package.  I believe they have been an
> important part of the base R language almost since its inception and
> have been used extensively.  You'd have to work really hard to do
> better, in my judgment.
> 
> 
>       Spencer Graves
> 
> 
> DISCLAIMER:  I'm primarily an R guy and only use Python when I can't
> find a sensible way to do what I want in R.
>>
>> Duncan
> 

Despite being a statistician I'm primarily a Python guy and only use R
when I can't find a sensible way to do what I want in Python :-). The
problem with the R solution is that it doesn't seem to get round the
issue of having an unfeasibly large population size, but a reasonable
number of categories. It turns out I've already coded up a few reservoir
based algorithms for sampling without replacement that work with data
streams. So I can get round the space issues, but it still means
processing each data point rather than generating the sample frequencies
directly.

After much searching all I've been able to find is the approach I
suggested above, iterating through the frequencies. My implementation:


import numpy

def hypgeom_variate(freqs, n):
sample = [0] * len(freqs)
nbad = sum(freqs)
hypergeometric = numpy.random.hypergeometric
for i, ngood in enumerate(freqs):
nbad -= ngood
x = hypergeometric(ngood, nbad, n, 1)[0]
if x:
sample[i] = x
n -= x
if not n:
break
return sample


Duncan
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue33944] Deprecate and remove pth files

[Chris Billington]

Chris Billington  added the comment:

> Linux distros approach to handling this is terrible because they dump all 
> their system packages into a single global site-packages, leading to the 
> every growing sys.path problem that Barry is concerned about.

> However, that's entirely the fault of distro packaging policies, and can be 
> remedied in a far superior way by switching distros to a model where they 
> create a venv per application, and then use .pth files to link in the system 
> packages that they actually want visible to that application.

I'm curious about this since it doesn't make sense to me. Dumping all packages 
at the top level in /usr/lib/pythonX.Y/site-packages means exactly zero .pth 
files. Wouldn't putting each module in its own directory, with all the 
directories necessary for a given app added to the path of a venv for that app 
mean strictly more .pth files, and a sys.path as long as the list of 
dependencies for that app? Whilst this would certainly be more flexible for 
keeping multiple versions of packages around as required by different apps, I 
don't see that it would decrease startup time at all - more folders need to be 
searched for each import, not less, and a recursive hierarchy of .pth files 
would need to be parsed at startup as each package pulled in the directories of 
its own dependencies. A flat structure like most linux distros use would seem 
like it would be as efficient as you could get, unless you think that searching 
through a larger list of strings for the right one is slower than opening a 
 tree of .pth files.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[Christian Heimes]

Change by Christian Heimes :


Added file: https://bugs.python.org/file48053/TALOS-2019-0758 - POC.pem

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[Christian Heimes]

Change by Christian Heimes :


Added file: https://bugs.python.org/file48052/TALOS-2019-0758.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35736] Missing component in table after getElementsByTagName("nn")

[Michael Krötlinger]

Michael Krötlinger  added the comment:

http://schemas.xmlsoap.org/wsdl/; 
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/; 
xmlns:tns="http://soap.ebs.client.chipkarte.at; 
xmlns:wsam="http://www.w3.org/2007/05/addressing/metadata; 
xmlns:wsp="http://www.w3.org/ns/ws-policy; 
xmlns:wsp1_2="http://schemas.xmlsoap.org/ws/2004/09/policy; 
xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd;
 xmlns:xsd="http://www.w3.org/2001/XMLSchema; 
targetNamespace="http://soap.ebs.client.chipkarte.at; name="EbsService">
  
http://www.w3.org/2001/XMLSchema; 
xmlns:tns="http://exceptions.soap.base.client.chipkarte.at; 
elementFormDefault="qualified" 
targetNamespace="http://exceptions.soap.base.client.chipkarte.at; version="1.0">
  
  
  
  
  

  

  

  
  

  
  
  

  
  

  

  

  
  

  

  

  
  

  

  

  

http://www.w3.org/2001/XMLSchema; 
xmlns:baseexc="http://exceptions.soap.base.client.chipkarte.at; 
xmlns:tns="http://exceptions.soap.ebs.client.chipkarte.at; 
elementFormDefault="qualified" 
targetNamespace="http://exceptions.soap.ebs.client.chipkarte.at; version="1.0">
  http://exceptions.soap.base.client.chipkarte.at; />
  
  
  

  

  

  

  
  

  

  

  

http://www.w3.org/2001/XMLSchema; 
xmlns:baseexc="http://exceptions.soap.base.client.chipkarte.at; 
elementFormDefault="qualified" 
targetNamespace="http://soap.base.client.chipkarte.at; version="1.0">
  

  
  
  

  
  

  
  

  
  

  
  
  
  
  
  
  
  
  
  
  

  

http://www.w3.org/2001/XMLSchema; 
xmlns:base="http://soap.base.client.chipkarte.at; 
xmlns:ebsexc="http://exceptions.soap.ebs.client.chipkarte.at; 
xmlns:xmime="http://www.w3.org/2005/05/xmlmime; elementFormDefault="qualified" 
targetNamespace="http://soap.ebs.client.chipkarte.at; version="1.0">
  http://soap.base.client.chipkarte.at; />
  http://www.w3.org/2005/05/xmlmime; 
schemaLocation="http://www.w3.org/2005/05/xmlmime; />
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  

  
  
  
  

  
  

  
  

  
  

  

  
  

  

  
  

  

  
  

  
  
  
  
  
  
  
  
  
  
  
  
  

  
  

  

  
  

  
  
  
  

  
  

  
  
  

  
  

  

  
  

  

  
  

  
  

  
  

  
  
  
  
  
  
  
  

  
  

  
  
  
  
  
  
  

  
  

  

  
  

  
  
  
  

  
  

  

  
  

  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  

  
  

  
  
  
  
  
  
  
  
  
  

  
  

  

  
  

  

  
  

  
  
  
  
  
  
  
  

  
  

  
  
  
  
  

  
  

  
  
  
  
  
  
  
  
  

  
  

  
  
 

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[STINNER Victor]

Change by STINNER Victor :


--
title: TALOS-2018-0758 Denial of Service -> [ssl][CVE-2019-5010] 
TALOS-2018-0758 Denial of Service

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] TALOS-2018-0758 Denial of Service

[Christian Heimes]

Change by Christian Heimes :


--
keywords: +patch, patch, patch
pull_requests: +11233, 11234, 11235

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] TALOS-2018-0758 Denial of Service

[Christian Heimes]

Change by Christian Heimes :


--
keywords: +patch
pull_requests: +11233

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Encounter issues to install Python

[Anthony Flury via Python-list]

What error are you getting ?

'Can not install the software' isn't enough information for me to assist 
in detail


In the mean time - things to check :

 * Does your user have permission to install to the directory you are
   trying to install to ?
 * Does the disk have enough disk space to install Python ?
 * have you actually got a 32 bit windows installation - many modern
   PCs are actually 64 bit now.

On 06/01/2019 15:20, Olivier Oussou wrote:

Hi dear Anthony,
I am using Windows systeme. I have download the set up uf python 3.6.4 
(32-bit) and I can not install the software on my computer.
I need your technical assistance to solve this matter and I will be 
glad if you do so.

Best regard!
Olivier
Medical entomologist, Benin



Le samedi 13 octobre 2018 à 19:24:28 UTC+2, Anthony Flury 
 a écrit :



Olivier,

Welcome to the list - before we can help you, we need some more 
information :


  * What Operating system are you using - Windows/Mac/Linux/Raspberry
Pi/Android for something else ?
  * What command or installer did you use to try to install Python.
  * What issues did you have during installation - if any ?
  * What interface are you trying to access, and how are you doing that ?
  * Do you get error messages?

Unless you tell us what the problem is we can't possibly help.

On 08/10/18 20:21, Olivier Oussou via Python-list wrote:

Hi!I downloaded and installed python 3.6.4 (32-bit) on my computer but I have 
problems and can not access the python interface.
I need your technical assistance to solve this matter.

Best regard!

Olivier OUSSOUMedical entomologist, Benin


--
Anthony Flury
*Email* : anthony.fl...@btinternet.com 


*Twitter* : @TonyFlury 

--
Anthony Flury
*Email* : anthony.fl...@btinternet.com 
*Twitter* : @TonyFlury 
--
https://mail.python.org/mailman/listinfo/python-list

[issue35746] TALOS-2018-0758 Denial of Service

[Christian Heimes]

Change by Christian Heimes :


--
keywords: +patch, patch
pull_requests: +11233, 11234

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] TALOS-2018-0758 Denial of Service

[Christian Heimes]

Christian Heimes  added the comment:

Please leave the bug open and don't remove files. It's too late. The bug report 
has been sent to mailing lists and RSS feeds already.

Also you cannot remove any files from the bug tracker. Only admins are can do 
that.

--
resolution: fixed -> 
stage: resolved -> patch review
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] TALOS-2018-0758 Denial of Service

[STINNER Victor]

STINNER Victor  added the comment:

I close the bug just to hide it from the home page and default search result, 
to have more time to fix it (make the issue less visible).

--
nosy: +vstinner
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] TALOS-2018-0758 Denial of Service

[Cisco Talos]

Cisco Talos  added the comment:

The files are removed and will be reissued to PSIRT.

Regina Wilson
Analyst.Business Operations
regiw...@cisco.com

[cid:CFA14CB5-B7B2-4FF7-8313-22D495F607D5@vrt.sourcefire.com]

On Jan 15, 2019, at 12:11 PM, Cisco Talos 
mailto:rep...@bugs.python.org>> wrote:

Change by Cisco Talos mailto:vuln...@cisco.com>>:

Removed file: https://bugs.python.org/file48052/TALOS-2019-0758.txt

___
Python tracker mailto:rep...@bugs.python.org>>

___

--
Added file: https://bugs.python.org/file48055/image001.png

___
Python tracker 

__
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: 3 random numbers

[Alister via Python-list]

On Tue, 15 Jan 2019 06:13:00 -0800, Gengyang Cai wrote:

> I managed to solve the problem and also another problem with different 3
> random numbers. But it wasn't a very good question in the first place, i
> admit 
> 
>
Indeed it is a poorly write exercise & I suspect it has been 
misinterpreted.

unless i am very much mistaken the tutor expects the program to pick the 
3 random numbers each time it is run, not the programmer  

-- 
We've picked COBOL as the language of choice.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue35746] TALOS-2018-0758 Denial of Service

[Cisco Talos]

Change by Cisco Talos :


Removed file: https://bugs.python.org/file48053/TALOS-2019-0758 - POC.pem

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] TALOS-2018-0758 Denial of Service

[Cisco Talos]

Change by Cisco Talos :


Removed file: https://bugs.python.org/file48052/TALOS-2019-0758.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] TALOS-2018-0758 Denial of Service

[Christian Heimes]

Christian Heimes  added the comment:

I can confirm that CPython is affected.

By the way PyCA cryptography handles the CRL DB just fine.

>>> from cryptography import x509
>>> from cryptography.hazmat.backends import default_backend
>>> with open("Lib/test/talos-2019-0758.pem", "rb") as f:
... pem_data = f.read()
... 
>>> cert = x509.load_pem_x509_certificate(pem_data, default_backend())
>>> cert.extensions[-1]
, 
critical=False, 
value=])>)>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] TALOS-2018-0758 Denial of Service

[Cisco Talos]

Cisco Talos  added the comment:

Thanks for acknowledging.  We look forward to any updates/developments on the 
issue reported.

For further information about the Cisco Vendor Vulnerability Reporting and 
Disclosure Policy please refer to this document which also links to our public 
PGP key. 
https://tools.cisco.com/security/center/resources/vendor_vulnerability_policy.html

Kind Regards,

Regina Wilson
Analyst.Business Operations
regiw...@cisco.com

[cid:CFA14CB5-B7B2-4FF7-8313-22D495F607D5@vrt.sourcefire.com]

On Jan 15, 2019, at 11:30 AM, Christian Heimes 
mailto:rep...@bugs.python.org>> wrote:

Christian Heimes mailto:li...@cheimes.de>> added the comment:

Thanks for the report!

--
assignee:  -> christian.heimes
components: +SSL
nosy: +christian.heimes
stage:  -> needs patch
versions: +Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker mailto:rep...@bugs.python.org>>

___

--
Added file: https://bugs.python.org/file48054/image001.png

___
Python tracker 

__
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] TALOS-2018-0758 Denial of Service

[Christian Heimes]

Christian Heimes  added the comment:

Thanks for the report!

--
assignee:  -> christian.heimes
components: +SSL
nosy: +christian.heimes
stage:  -> needs patch
versions: +Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] TALOS-2018-0758 Denial of Service

[Cisco Talos]

Change by Cisco Talos :


--
versions:  -Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file48053/TALOS-2019-0758 - POC.pem

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] TALOS-2018-0758 Denial of Service

[Cisco Talos]

New submission from Cisco Talos :

An exploitable denial-of-service vulnerability exists in the X509 certificate 
parser of Python.org Python 2.7.11 / 3.6.6. A specially crafted X509 
certificate can cause a NULL pointer dereference, resulting in a denial of 
service. An attacker can initiate or accept TLS connections using crafted 
certificates to trigger this vulnerability.

--
files: TALOS-2019-0758.txt
messages: 333709
nosy: Talos
priority: normal
severity: normal
status: open
title: TALOS-2018-0758 Denial of Service
type: security
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file48052/TALOS-2019-0758.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Python read text file columnwise

[Neil Cerutti]

On 2019-01-15, Juris __  wrote:
> Hi!
>
> On 15/01/2019 17:04, Neil Cerutti wrote:
>> On 2019-01-11, shibashib...@gmail.com  wrote:
>>> Hello

 I'm very new in python. I have a file in the format:

 2018-05-31   16:00:0028.90   81.77   4.3
 2018-05-31   20:32:0028.17   84.89   4.1
 2018-06-20   04:09:0027.36   88.01   4.8
 2018-06-20   04:15:0027.31   87.09   4.7
 2018-06-28   04.07:0027.87   84.91   5.0
 2018-06-29   00.42:0032.20   104.61  4.8
>>>
>>> I would like to read this file in python column-wise.
>>>
>>> I tried this way but not working 
>>>event_list = open('seismicity_R023E.txt',"r")
>>>  info_event = read(event_list,'%s %s %f %f %f %f\n');
>> 
>> If it's really tabular data in fixed-width columns you can read
>> it that way with Python.
>> 
>> records = []
>> for line in file:
>>  record = []
>>  i = 0
>>  for width in (30, 8, 7, 5): # approximations
>>  item = line[i:i+width]
>>  record.append(item)
>>  i += width
>>  records.append(record)
>> 
>> This leaves them all strings, which in my experience is more
>> convenient in practice. You can convert as you go if you
>> want,though it won't look nice and simple any longer.
>>
>
> Perhaps even better approach is to use csv module from standard library:
>
> import csv
>
> csv_reader = csv.reader(file, dialect="excel-tab")
> for row in csv_reader:
>  # do something with record data which is conveniently parsed to list
>  print(row)
>
> ['2018-05-31', '16:00:00', '28.90', '81.77', '4.3']
> ...
> ['2018-06-29', '00.42:00', '32.20', '104.61', '4.8']

Yes, if applicable it is awesome!

-- 
Neil Cerutti
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python read text file columnwise

[Juris __]

Hi!

On 15/01/2019 17:04, Neil Cerutti wrote:
> On 2019-01-11, shibashib...@gmail.com  wrote:
>> Hello
>>>
>>> I'm very new in python. I have a file in the format:
>>>
>>> 2018-05-31   16:00:0028.90   81.77   4.3
>>> 2018-05-31   20:32:0028.17   84.89   4.1
>>> 2018-06-20   04:09:0027.36   88.01   4.8
>>> 2018-06-20   04:15:0027.31   87.09   4.7
>>> 2018-06-28   04.07:0027.87   84.91   5.0
>>> 2018-06-29   00.42:0032.20   104.61  4.8
>>
>> I would like to read this file in python column-wise.
>>
>> I tried this way but not working 
>>event_list = open('seismicity_R023E.txt',"r")
>>  info_event = read(event_list,'%s %s %f %f %f %f\n');
> 
> If it's really tabular data in fixed-width columns you can read
> it that way with Python.
> 
> records = []
> for line in file:
>  record = []
>  i = 0
>  for width in (30, 8, 7, 5): # approximations
>  item = line[i:i+width]
>  record.append(item)
>  i += width
>  records.append(record)
> 
> This leaves them all strings, which in my experience is more
> convenient in practice. You can convert as you go if you
> want,though it won't look nice and simple any longer.
>

Perhaps even better approach is to use csv module from standard library:

import csv

csv_reader = csv.reader(file, dialect="excel-tab")
for row in csv_reader:
 # do something with record data which is conveniently parsed to list
 print(row)

['2018-05-31', '16:00:00', '28.90', '81.77', '4.3']
...
['2018-06-29', '00.42:00', '32.20', '104.61', '4.8']


BR, Juris
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue35745] Add import statement in dataclass code snippet

[Windson Yang]

Windson Yang  added the comment:

I'm not sure if we should put 

from dataclasses import dataclass

everywhere or we should put it just in the first example as I did in the PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35745] Add import statement in dataclass code snippet

[Windson Yang]

New submission from Windson Yang :

Most of the example in https://docs.python.org/3/library/dataclasses.html miss 
code like

from dataclasses import dataclass, field
from typing import List

I think we should add this statement in the code snippet.

--
assignee: docs@python
components: Documentation
messages: 333707
nosy: Windson Yang, docs@python
priority: normal
severity: normal
status: open
title: Add import statement in dataclass code snippet
type: enhancement
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Python read text file columnwise

[Neil Cerutti]

On 2019-01-11, shibashib...@gmail.com  wrote:
> Hello
>> 
>> I'm very new in python. I have a file in the format:
>> 
>> 2018-05-31   16:00:0028.90   81.77   4.3
>> 2018-05-31   20:32:0028.17   84.89   4.1
>> 2018-06-20   04:09:0027.36   88.01   4.8
>> 2018-06-20   04:15:0027.31   87.09   4.7
>> 2018-06-28   04.07:0027.87   84.91   5.0
>> 2018-06-29   00.42:0032.20   104.61  4.8
>
> I would like to read this file in python column-wise.  
>
> I tried this way but not working 
>   event_list = open('seismicity_R023E.txt',"r")
> info_event = read(event_list,'%s %s %f %f %f %f\n');

If it's really tabular data in fixed-width columns you can read
it that way with Python.

records = []
for line in file:
record = []
i = 0
for width in (30, 8, 7, 5): # approximations
item = line[i:i+width]
record.append(item)
i += width
records.append(record)

This leaves them all strings, which in my experience is more
convenient in practice. You can convert as you go if you
want,though it won't look nice and simple any longer.

-- 
Neil Cerutti
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue35741] unittest.skipUnless(time._STRUCT_TM_ITEMS == 11, "needs tm_zone support") doesn't work

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +belopolsky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: 3 random numbers

[Gengyang Cai]

I managed to solve the problem and also another problem with different 3 random 
numbers. But it wasn't a very good question in the first place, i admit 



On Tuesday, January 15, 2019 at 9:55:00 PM UTC+8, Rick Johnson wrote:
> Gengyang Cai wrote:
> > Can anyone understand it and explain it to me please ? 
> 
> Instead of attempting to read source code that you obviously have no 
> qualification to read, why don't you try _thinking_ like a programmer? 
> 
> "But Rick! How will i think like a programmer if i cannot even read source 
> code?!"
> 
> Forget about reading source code, kid! Reading source code is only a small 
> part of being a programmer. Are you any less intelligent because you can't 
> speak every language on the freakin' planet including Klingon, Elvish and the 
> black tongue of the Mordor Orc? No. Of course not! You can still tie you 
> tennis shoes, yes? Okay... Thus, programming languages are like natural 
> language, in that they are merely a means to communicate. Tools. That's all. 
> So, if you can become competent with a rake, then you can probably do the 
> same with a shovel, and a hoe. All of these tools will help you build a 
> garden. Or bury a skeleton in the backyard -- after dark, when the nosy 
> neighbors are sleeping!
> 
> <_<
> 
> >_>
> 
> "Okay Rick... i sorta understand your point here, but... i'm not having that 
> ah-hah! moment. How do i think like a programmer?"
> 
> Simple! You look at a problem, and then you ask yourself: "What are the 
> fundamental steps required to solve this problem?" And sometimes scratching 
> the head helps...
> 
> ASSIGNMENT: "Pick any 3 random ascending numbers and write out a loop 
> function that prints out all 3 numbers"""
> 
> Looking at this sentence, i see two specific problems:
> 
> 
> 
> So, in the case of your assignment, the first step would be to pick three 
> numbers. 
> 
> STEP_1. Pick three numbers. 
> 
>numbers = [400, 467, 851]

-- 
https://mail.python.org/mailman/listinfo/python-list

help with python difference between number

[achyuta2017]

M <01/14/2019 08:07:01> Count:0 
Total:50
Free: 20
A
B
M <01/14/2019 08:07:04> Count:1
Total:5
Free:10
A
B
M <01/14/2019 08:07:07> Count:2
Total:5
Free:3
A
B


I am trying to make a output like where it prints the free and then the 
difference between the current free and previous free
For e.g

M <01/14/2019 08:07:01> Count:0  Free: 20
M <01/14/2019 08:07:04> Count:1  Free: 10  absolute difference between time and 
prev time is -10
M <01/14/2019 08:07:07> Count:2  Free: 3   absolute difference between time and 
prev time is -7


And then later on i need to determine the time when we had the most negative 
free value.


I tried a code like this
Which printed 
 with open("summ4.txt") as f:
# get first line/number
nxt = int(next(f))
for n in f:
print("absolute difference between {} and {} = {}"
  .format(n.rstrip(), nxt, abs(int(nxt) - int(n
# set nxt equal to the next number
nxt = int(next(f,0))
   a=open('summ1.txt','r').readlines()
   b=open('summ3.txt','r').readlines()
with open('summ.txt','w') as out:
  for i in range(0,365): 
print>>out,a[i].rstrip(),b[i]


I hit error as
Traceback (most recent call last):
File "3.py", 
line 39, in 
   .format(n.rstrip(), 
nxt, abs(int(nxt) - int(n
ValueError: 
zero length field name in format 

I guess my input file has a tab in the start and not able to get a difference 
rightly.
.
Any pointers on how to achieve the desired result?

Thanks,
Hare
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue33944] Deprecate and remove pth files

[STINNER Victor]

STINNER Victor  added the comment:

> SyntaxError: encoding problem: future_fstrings

IMHO that's the expected behavior. I would prefer to have to explicitly install 
this special encoding *before* loading a script using it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Jason R. Coombs]

Jason R. Coombs  added the comment:

> `site.addsitedir` is called for every site-packages directory (whether 
> global, within a venv, or at the user level), so my proposal above covers 
> appending multiple segments.

Good point. I think you're assuming that only site dirs are appropriate for 
packages that require arbitrary code execution. I think I'd like to break that 
assumption and allow any location where packages can be installed (PYTHONPATH) 
to install hooks. Consider this use-case:

draft $ mkdir pkgs  
 draft $ python3.5 
-m pip download -d pkgs future_fstrings 
 Collecting future_fstrings
  Using cached 
https://files.pythonhosted.org/packages/36/25/070c2dc1fe1e51901df5875c495d6efbbf945a93a2ca40f47e5225302fb8/future_fstrings-0.4.5-py2.py3-none-any.whl
  Saved ./pkgs/future_fstrings-0.4.5-py2.py3-none-any.whl
Collecting tokenize-rt; python_version < "3.6" (from future_fstrings)
  Using cached 
https://files.pythonhosted.org/packages/76/82/0e6a9dda45dd76be22d74211443e199a330ac7e428b8dbbc5d116651be03/tokenize_rt-2.1.0-py2.py3-none-any.whl
  Saved ./pkgs/tokenize_rt-2.1.0-py2.py3-none-any.whl
Successfully downloaded future-fstrings tokenize-rt
draft $ cat > hello-fstrings.py 
# coding: 
future_fstrings
print(f'hello world') 
draft $ 
PYTHONPATH=pkgs/future_fstrings-0.4.5-py2.py3-none-any.whl:pkgs/tokenize_rt-2.1.0-py2.py3-none-any.whl
 python3.5 hello-fstrings.py   
xonsh: subprocess mode: command not found: 
PYTHONPATH=pkgs/future_fstrings-0.4.5-py2.py3-none-any.whl:pkgs/tokenize_rt-2.1.0-py2.py3-none-any.whl
draft $ env 
PYTHONPATH=pkgs/future_fstrings-0.4.5-py2.py3-none-any.whl:pkgs/tokenize_rt-2.1.0-py2.py3-none-any.whl
 python3.5 hello-fstrings.py   
  File "hello-fstrings.py", line 1
SyntaxError: encoding problem: future_fstrings


If future-fstrings were properly installed, its runtime hook is called and the 
script can run:

draft $ python3.5 -m pip-run -q future-fstrings -- hello-fstrings.py

 
hello world


I'd like for a package like future-fstrings to be able to supply a hook that 
can be executed on startup that can be honored even if the package isn't 
installed in one of the site paths.

> Let's make a PEP.

I'd be delighted to help with the PEP.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23846] asyncio : ProactorEventLoop raised BlockingIOError when ThreadPoolExecutor has many workers

[STINNER Victor]

STINNER Victor  added the comment:

> In any case it looks like self-pipe sock's buffer was overflown because 
> call_soon_threadsafe was called too many times, and loop._read_from_self 
> couldn't empty the buffer promptly.  Then, at some point, _write_to_self 
> failed with an IOError.

I fixed the issue. Thanks for your bug report ;-)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: [tkinter] question about correct use of validation

[steve]

On 15/01/2019 14:16, Rick Johnson wrote:

steve wrote:

Then it's just a matter of passing a keyword argument:

 myEntry = MyEntry(master, maxlen=20)

What you have above is a tightly coiled, steaming dogpile that will litter your 
code base. Encapsulate that stench, would ya?



sure, everything will go into a "label_entry" function

It's similar to this:

def __label_entry (self, frame, text_label, width_label, width_entry, 
maxlen):

 '' 'Private utility function for a couple of labels + entry.
 Return istance of entry itself.
 '' '
 Label (frame, padx=10, pady=10, width=width_label,
text=text_label) .Pack (side=LEFT)
 entry = Entry (frame, width=width_entry)
 entry.configure (highlightcolor='blue')
 entry.pack (side=LEFT)
..
 return entry

I wrote voluntarily in order to better understand the question
--
https://mail.python.org/mailman/listinfo/python-list

[issue23846] asyncio : ProactorEventLoop raised BlockingIOError when ThreadPoolExecutor has many workers

[miss-islington]

miss-islington  added the comment:


New changeset c9f26714d511a338ba2fdd926e3dc62636f31815 by Miss Islington (bot) 
in branch '3.7':
bpo-23846: Fix ProactorEventLoop._write_to_self() (GH-11566)
https://github.com/python/cpython/commit/c9f26714d511a338ba2fdd926e3dc62636f31815


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35744] Problem in the documentation of numpy.random.randint in python 2.7

[STINNER Victor]

STINNER Victor  added the comment:

Sorry but this is the bug tracker of Python, not of numpy. Please use 
https://github.com/numpy/numpy/issues instead.

--
nosy: +vstinner
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35744] Problem in the documentation of numpy.random.randint in python 2.7

[Jay]

New submission from Jay :

Official Documentation of python 2.7 mentions that numpy.random.randint(a,b) 
will return a random integer from N such that a<=N<=b. But I have run the code 
and I have found that it never returns equal to b. So, what I did was I ran 
numpy.random.randint(0,1) for 50 milion times and finally printed the sum. 

The output was 0. 

I don't know if this a documentation or an implementation issue, but this is an 
issue which needs to be looked at. I am attaching the code that I ran.

--
assignee: docs@python
components: Documentation
files: sample.py
messages: 333701
nosy: Jay, docs@python
priority: normal
severity: normal
status: open
title: Problem in the documentation of numpy.random.randint in python 2.7
type: behavior
versions: Python 2.7
Added file: https://bugs.python.org/file48051/sample.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: get the terminal's size

[Alex Ternaute]

Hi Cameron,

>>I tried : P = Popen(['stty', '-a'], stdout=subprocess.PIPE,
>>universal_newlines=True) and it runs fine too, so the output seems not
>>really related to that fd.
 
> But it is! stty(1) fetches the terminal settings from its standard
> input, so "fd" is used to supply this. In your Popen test case you
> simply don't set the stdin parameter, so it is the Python process'
> input. Which is usually what you want.

> But I want to be able to ask this
> of any terminal file, thus the parameter.

Ah, Ok; smthlike:
 cs.tty.ttysize(0)
 WinSize(rows=50, columns=100)
 anotherTty=open('/dev/pts/3', 'rw')
 cs.tty.ttysize(anotherTty)
 WinSize(rows=43, columns=199)

It runs :)

I do not need that today but one day orother it could help.
 
Cheers
-- 
Alex
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue23846] asyncio : ProactorEventLoop raised BlockingIOError when ThreadPoolExecutor has many workers

[miss-islington]

Change by miss-islington :


--
pull_requests: +11230, 11231, 11232

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com