[Python-Dev] buildbot failure in sparc solaris10 gcc trunk
The Buildbot has detected a new failure of sparc solaris10 gcc trunk. Full details are available at: http://www.python.org:9010/sparc%20solaris10%20gcc%20trunk/builds/68 Buildbot URL: http://www.python.org:9010/ Build Reason: The web-page 'rebuild' button was pressed by '': Force compile error Build Source Stamp: [branch trunk] HEAD Blamelist: BUILD FAILED: failed compile sincerely, -The Buildbot _______ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Test
Testing submission from dinsdale. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] buildbot failure in x86 W2k trunk
The Buildbot has detected a new failure of x86 W2k trunk. Full details are available at: http://www.python.org:9010/x86%20W2k%20trunk/builds/83 Buildbot URL: http://www.python.org:9010/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: anthony.baxter BUILD FAILED: failed compile sincerely, -The Buildbot ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] buildbot failure in x86 XP-2 trunk
The Buildbot has detected a new failure of x86 XP-2 trunk. Full details are available at: http://www.python.org:9010/x86%20XP-2%20trunk/builds/69 Buildbot URL: http://www.python.org:9010/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: anthony.baxter BUILD FAILED: failed compile sincerely, -The Buildbot ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] buildbot failure in x86 XP-2 trunk
The Buildbot has detected a new failure of x86 XP-2 trunk. Full details are available at: http://www.python.org:9010/x86%20XP-2%20trunk/builds/71 Buildbot URL: http://www.python.org:9010/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: anthony.baxter,neal.norwitz BUILD FAILED: failed compile sincerely, -The Buildbot ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] buildbot failure in x86 W2k trunk
The Buildbot has detected a new failure of x86 W2k trunk. Full details are available at: http://www.python.org:9010/x86%20W2k%20trunk/builds/85 Buildbot URL: http://www.python.org:9010/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: anthony.baxter,neal.norwitz BUILD FAILED: failed compile sincerely, -The Buildbot ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] buildbot failure in x86 XP 2.4
The Buildbot has detected a new failure of x86 XP 2.4. Full details are available at: http://www.python.org:9010/x86%20XP%202.4/builds/20 Buildbot URL: http://www.python.org:9010/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: vinay.sajip BUILD FAILED: failed failed slave lost sincerely, -The Buildbot ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another case for frozendict
On Sun, Jul 13, 2014 at 02:04:17PM +, Jason R. Coombs wrote: > PEP-416 mentions a MappingProxyType, but that’s no help. Well, it kindof is. By combining MappingProxyType and UserDict the desired effect can be achieved concisely: import collections import types class frozendict(collections.UserDict): def __init__(self, d, **kw): if d: d = d.copy() d.update(kw) else: d = kw self.data = types.MappingProxyType(d) _h = None def __hash__(self): if self._h is None: self._h = sum(map(hash, self.data.items())) return self._h def __repr__(self): return repr(dict(self)) > Although hashability is mentioned in the PEP under constraints, there are many > use-cases that fall out of the ability to hash a dict, such as the one > described above, which are not mentioned at all in use-cases for the PEP. > If there’s ever any interest in reviving that PEP, I’m in favor of its > implementation. In its previous form, the PEP seemed more focused on some false optimization capabilities of a read-only type, rather than as here, the far more interesting hashability properties. It might warrant a fresh PEP to more thoroughly investigate this angle. David ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another case for frozendict
On Sun, Jul 13, 2014 at 06:43:28PM +, dw+python-...@hmmz.org wrote: > if d: > d = d.copy() To cope with iterables, "d = d.copy()" should have read "d = dict(d)". David ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another case for frozendict
On Wed, Jul 16, 2014 at 09:47:59AM -0400, R. David Murray wrote:
> It would be nice to be able to return a frozendict instead of having the
> overhead of building a new dict on each call.
There already is an in-between available both to Python and C:
PyDictProxy_New() / types.MappingProxyType. It's a one line change in
each case to return a temporary intermediary, using something like (C):
Py_INCREF(self->dict)
return self->dict;
To
return PyDictProxy_New(self->dict);
Or Python:
return self.dct
To
return types.MappingProxyType(self.dct)
Which is cheaper than a copy, and avoids having to audit every use of
self->dict to ensure the semantics required for a "frozendict" are
respected, i.e. no mutation occurs after the dict becomes visible to the
user, and potentially has __hash__ called.
> That by itself might not be enough reason. But, if the user wants to
> use the data in modified form elsewhere, they would then have to
> construct a new regular dict out of it, making the decision to vary
> the data from what matches the state of the object it came from an
> explicit one. That seems to fit the Python zen ("explicit is better
> than implicit").
>
> So I'm changing my mind, and do consider this a valid use case, even
> absent the crash.
Avoiding crashes seems a better use for a read-only proxy, rather than a
hashable immutable type.
David
_______
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cStringIO vs io.BytesIO
p;self->initvalue);
+self->flags &= ~IO_SHARED;
+} else if (self->buf != NULL) {
PyMem_Free(self->buf);
-self->buf = NULL;
}
+self->buf = NULL;
Py_RETURN_NONE;
}
@@ -788,10 +853,17 @@
"deallocated BytesIO object has exported buffers");
PyErr_Print();
}
-if (self->buf != NULL) {
+
+if (self->flags & IO_SHARED) {
+/* We borrowed buf from another object */
+PyBuffer_Release(&self->initvalue);
+self->flags &= ~IO_SHARED;
+} else if (self->buf != NULL) {
+/* We owned buf */
PyMem_Free(self->buf);
-self->buf = NULL;
}
+self->buf = NULL;
+
Py_CLEAR(self->dict);
if (self->weakreflist != NULL)
PyObject_ClearWeakRefs((PyObject *) self);
@@ -811,12 +883,6 @@
/* tp_alloc initializes all the fields to zero. So we don't have to
initialize them here. */
-self->buf = (char *)PyMem_Malloc(0);
-if (self->buf == NULL) {
-Py_DECREF(self);
-return PyErr_NoMemory();
-}
-
return (PyObject *)self;
}
@@ -834,13 +900,32 @@
self->string_size = 0;
self->pos = 0;
+/* Release any previous initvalue. */
+if (self->flags & IO_SHARED) {
+PyBuffer_Release(&self->initvalue);
+self->buf = NULL;
+self->flags &= ~IO_SHARED;
+}
+
if (initvalue && initvalue != Py_None) {
-PyObject *res;
-res = bytesio_write(self, initvalue);
-if (res == NULL)
+Py_buffer *buf = &self->initvalue;
+if (PyObject_GetBuffer(initvalue, buf, PyBUF_CONTIG_RO) < 0) {
return -1;
-Py_DECREF(res);
-self->pos = 0;
+}
+self->buf = self->initvalue.buf;
+self->buf_size = (size_t)self->initvalue.len;
+self->string_size = self->initvalue.len;
+self->flags |= IO_SHARED;
+ }
+
+/* If no initvalue provided, prepare a private buffer now. */
+if (self->buf == NULL) {
+self->buf = (char *)PyMem_Malloc(0);
+if (self->buf == NULL) {
+Py_DECREF(self);
+PyErr_NoMemory();
+return -1;
+}
}
return 0;
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cStringIO vs io.BytesIO
> +self->string_size = (Py_ssize_t) copy_size;
> +return 0;
> +}
>
> /* Internal routine to get a line from the buffer of a BytesIO
> object. Returns the length between the current position to the
> @@ -125,11 +176,18 @@
> static Py_ssize_t
> write_bytes(bytesio *self, const char *bytes, Py_ssize_t len)
> {
> +size_t desired;
> +
> assert(self->buf != NULL);
> assert(self->pos >= 0);
> assert(len >= 0);
>
> -if ((size_t)self->pos + len > self->buf_size) {
> +desired = (size_t)self->pos + len;
> +if (unshare(self, desired)) {
> +return -1;
> +}
> +
> +if (desired > self->buf_size) {
> if (resize_buffer(self, (size_t)self->pos + len) < 0)
> return -1;
> }
> @@ -502,6 +560,10 @@
> return NULL;
> }
>
> +if (unshare(self, size)) {
> +return NULL;
> +}
> +
> if (size < self->string_size) {
> self->string_size = size;
> if (resize_buffer(self, size) < 0)
> @@ -655,10 +717,13 @@
> static PyObject *
> bytesio_close(bytesio *self)
> {
> -if (self->buf != NULL) {
> +if (self->flags & IO_SHARED) {
> +PyBuffer_Release(&self->initvalue);
> +self->flags &= ~IO_SHARED;
> +} else if (self->buf != NULL) {
> PyMem_Free(self->buf);
> -self->buf = NULL;
> }
> +self->buf = NULL;
> Py_RETURN_NONE;
> }
>
> @@ -788,10 +853,17 @@
> "deallocated BytesIO object has exported buffers");
> PyErr_Print();
> }
> -if (self->buf != NULL) {
> +
> +if (self->flags & IO_SHARED) {
> +/* We borrowed buf from another object */
> +PyBuffer_Release(&self->initvalue);
> +self->flags &= ~IO_SHARED;
> +} else if (self->buf != NULL) {
> +/* We owned buf */
> PyMem_Free(self->buf);
> -self->buf = NULL;
> }
> +self->buf = NULL;
> +
> Py_CLEAR(self->dict);
> if (self->weakreflist != NULL)
> PyObject_ClearWeakRefs((PyObject *) self);
> @@ -811,12 +883,6 @@
> /* tp_alloc initializes all the fields to zero. So we don't have to
> initialize them here. */
>
> -self->buf = (char *)PyMem_Malloc(0);
> -if (self->buf == NULL) {
> -Py_DECREF(self);
> -return PyErr_NoMemory();
> -}
> -
> return (PyObject *)self;
> }
>
> @@ -834,13 +900,32 @@
> self->string_size = 0;
> self->pos = 0;
>
> +/* Release any previous initvalue. */
> +if (self->flags & IO_SHARED) {
> +PyBuffer_Release(&self->initvalue);
> +self->buf = NULL;
> + self->flags &= ~IO_SHARED;
> +}
> +
> if (initvalue && initvalue != Py_None) {
> -PyObject *res;
> - res = bytesio_write(self, initvalue);
> -if (res == NULL)
> + Py_buffer *buf = &self->initvalue;
> +if (PyObject_GetBuffer(initvalue, buf, PyBUF_CONTIG_RO) < 0) {
> return -1;
> -Py_DECREF(res);
> -self->pos = 0;
> +}
> +self->buf = self->initvalue.buf;
> +self->buf_size = (size_t)self->initvalue.len;
> +self->string_size = self->initvalue.len;
> +self->flags |= IO_SHARED;
> +}
> +
> +/* If no initvalue provided, prepare a private buffer now. */
> +if (self->buf == NULL) {
> +self->buf = (char *)PyMem_Malloc(0);
> +if (self->buf == NULL) {
> +Py_DECREF(self);
> +PyErr_NoMemory();
> +return -1;
> +}
> }
>
> return 0;
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/dw%2Bpython-dev%40hmmz.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #22003: When initialized from a bytes object, io.BytesIO() now
Hi Serhiy, At least conceptually, 15381 seems the better approach, but getting a correct implementation may take more iterations than the (IMHO) simpler change in 22003. For my tastes, the current 15381 implementation seems a little too magical in relying on Py_REFCNT() as the sole indication that a PyBytes can be mutated. For the sake of haste, 22003 only addresses the specific regression introduced in Python 3.x BytesIO, compared to 2.x StringI, where 3.x lacked an equivalent no-copies specialization. David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python process creation overhead
On Mon, May 12, 2014 at 04:22:52PM -0700, Gregory Szorc wrote: > Why can't Python start as quickly as Perl or Ruby? On my heavily abused Core 2 Macbook with 9 .pth files, 2.7 drops from 81ms startup to 20ms by specifying -S, which disables site.py. Oblitering the .pth files immediately knocks 10ms off regular startup. I guess the question isn't why Python is slower than perl, but what aspects of site.py could be cached, reimplemented, or stripped out entirely. I'd personally love to see .pth support removed. David ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Internal representation of strings and Micropython
On Wed, Jun 04, 2014 at 03:17:00PM +1000, Nick Coghlan wrote: > There's a general expectation that indexing will be O(1) because all > the builtin containers that support that syntax use it for O(1) lookup > operations. Depending on your definition of built in, there is at least one standard library container that does not - collections.deque. Given the specialized kinds of application this Python implementation is targetted at, it seems UTF-8 is ideal considering the huge memory savings resulting from the compressed representation, and the reduced likelihood of there being any real need for serious text processing on the device. It is also unlikely to find software or libraries like Django or Werkzeug running on a microcontroller, more likely all the Python code would be custom, in which case, replacing string indexing with iteration, or temporary conversion to a list is easily done. In this context, while a fixed-width encoding may be the correct choice it would also likely be the wrong choice. David ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler
On Fri, Jun 06, 2014 at 03:41:22PM +, Steve Dower wrote: > [snip] Speaking as a third party who aims to provide binary distributions for recent Python releases on Windows, every new compiler introduces a licensing and configuration headache. So I guess the questions are: * Does the ABI stability address some historical real world problem with Python binary builds? (I guess possibly) * Is the existing solution of third parties building under e.g. Mingw as an option of last resort causing real world issues? It seems to work for a lot of people, although I personally avoid it. * Have other compiler vendors indicated they will change their ABI environment to match VS under this new stability guarantee? If not, then as yet there is no real world benefit here. * Has Python ever hit a showstopper release issue as a result of a bug in MSVC? (I guess probably not). * Will VS 14 be golden prior to Python 3.5's release? It would suck to rely on a beta compiler.. :) Sorry for dunking water on this, but I've recently spent a ton of time getting a Microsoft build environment running, and it seems possible a new compiler may not yet justify more effort if there is little tangible benefit. David ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler
On Fri, Jun 06, 2014 at 10:49:24PM +0400, Brian Curtin wrote: > None of the options are particularly good, but yes, I think that's an > option we have to consider. We're supporting 2.7.x for 6 more years on > a compiler that is already 6 years old. Surely that is infinitely less desirable than simply bumping the minor version? David _______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler
On Sat, Jun 07, 2014 at 05:33:45AM +1000, Chris Angelico wrote: > > Is it really any difference in maintenance if you just stop applying > > updates to 2.7 and switch to 2.8? If 2.8 is really just 2.7 with a > > new compiler then there should be no functional difference between > > doing that and doing a 2.7.whatever except all of the tooling that > > relies on the compiler not to change in micro releases won’t > > suddenly break and freak out. > If the only difference between 2.7 and 2.8 is the compiler used on > Windows, what happens on Linux and other platforms? A Python 2.8 would > have to be materially different from Python 2.7, not just binarily > incompatible on one platform. Grrmph, that's fair. Perhaps a final alternative is simply continuing the 2.7 series with a stale compiler, as a kind of carrot on a stick to encourage users to upgrade? Gating 2.7 life on the natural decline of its supported compiler/related ecosystem seems somehow quite a gradual and natural demise.. :) David _______________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] namedtuple implementation grumble
On Sun, Jun 08, 2014 at 07:37:46PM +, dw+python-...@hmmz.org wrote:
> cls = tuple(name, (_NamedTuple,), {
Ugh, this should of course have been type().
David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] namedtuple implementation grumble
On Sun, Jun 08, 2014 at 03:13:55PM -0400, Eric V. Smith wrote:
> > The current implementation is also *really* easy to understand,
> > while writing out the dynamic type creation explicitly would likely
> > require much deeper knowledge of the type machinery to follow.
> As proof that it's harder to understand, here's an example of that
> dynamically creating functions and types:
Probably I'm missing something, but there's a much simpler non-exec
approach, something like:
class _NamedTuple(...):
...
def namedtuple(name, fields):
cls = tuple(name, (_NamedTuple,), {
'_fields': fields.split()
})
for i, field_name in enumerate(cls._fields):
prop = property(functools.partial(_NamedTuple.__getitem__, i)
functools.partial(_NamedTuple.__setitem__, i))
setattr(cls, field_name, prop)
return cls
David
_______________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] namedtuple implementation grumble
On Sun, Jun 08, 2014 at 05:27:41PM -0400, Eric V. Smith wrote:
> How would you write _Namedtuple.__new__?
Knew something must be missing :) Obviously it's possible, but not
nearly as efficiently as reusing the argument parsing machinery as in
the original implementation.
I guess especially the kwargs implementation below would suck..
_undef = object()
class _NamedTuple(...):
def __new__(cls, *a, **kw):
if kw:
a = list(a) + ([_undef] * (len(self._fields)-len(a)))
for k, v in kw.iteritems():
i = cls._name_id_map[k]
if a[i] is not _undef:
raise TypeError(...)
a[i] = v
if _undef not in a:
return tuple.__new__(cls, a)
raise TypeError(...)
else:
if len(a) == len(self._fields):
return tuple.__new__(cls, a)
raise TypeError(...)
def namedtuple(name, fields):
fields = fields.split()
cls = type(name, (_NamedTuple,), {
'_fields': fields,
'_name_id_map': {k: i for i, k in enumerate(fields)}
})
for i, field_name in enumerate(fields):
getter = functools.partial(_NamedTuple.__getitem__, i)
setattr(cls, field_name, property(getter))
return cls
David
___________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Issue 32933
I mentioned when I raised issue 32933 (mock_open doesn't support dunder iter protocol) that I have a one line fix for the issue. Who should I send this detail too ? Obviously it will need test cases too - I wrote a single very simple one just to prove to myself it worked - but it isn't complete by any stretch of the imagination. -- Anthony Flury anthony.fl...@btinternet.com_______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python Test suite hangining
All, Sorry to trouble you all - but I am trying to get the Python 3.8 test suite running on Ubuntu 16.0.4. As per the dev guide - having cloned the repo and run the build I am running the test suite by : ./python -m test -j1 This runs through to test 414/415 - and then start getting this message running: test_poplib (nnn sec) every 30 seconds - nnn got to 1077 secs before I decided to stop it. When I ran test_poplib on it's own - I got this Traceback : Traceback (most recent call last): ... ... File "/home/tony/Development/python/cpython/Lib/ssl.py", line 1108, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:1038) And then lots of reports of unexpected EOFs. I have the latest CA-certificates installed Any advice - I couldn't find anything on google. -- Anthony Flury anthony.fl...@btinternet.com From: "anthony.fl...@btinternet.com" To: "Python-Dev@python.org" Sent: Saturday, March 3, 2018 5:54 PM Subject: Issue 32933 I mentioned when I raised issue 32933 (mock_open doesn't support dunder iter protocol) that I have a one line fix for the issue. Who should I send this detail too ? Obviously it will need test cases too - I wrote a single very simple one just to prove to myself it worked - but it isn't complete by any stretch of the imagination. -- Anthony Flury anthony.fl...@btinternet.com _______________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Hi, I am amar :)
Hi, all. Quick joke, Do you know why all functional programmers are anarchists? Because they want to get rid of the state! :D I know there are a lot more important issues than this, but i feel this is important too. I wish some people could take a look at Python/Cpython pull request #6343[1] and the reopened PR #6362[2]. Little background here, I am a science student working on GIS(Geospatial Information Systems) Web Processing Service as a hobby project. I am a human too, i like jokes. But not at the expense of truth. I hope i can make more meaningful contributions in the future! [1]https://github.com/python/cpython/pull/6343 [2] https://github.com/python/cpython/pull/6362#issuecomment-378174084___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] bpo-33038: Fix gzip.GzipFile for file objects with a non-string name attribute. (GH-6095)
From: Serhiy Storchaka To: python-check...@python.org Sent: Wednesday, 9 May 2018, 10:14 Subject: [Python-checkins] bpo-33038: Fix gzip.GzipFile for file objects with a non-string name attribute. (GH-6095) https://github.com/python/cpython/commit/afe5f633e49e0e873d42088ae56819609c803ba0commit: afe5f633e49e0e873d42088ae56819609c803ba0branch: 2.7author: Bo Bayles <bbay...@gmail.com>committer: Serhiy Storchaka <storch...@gmail.com>date: 2018-05-09T13:14:40+03:00summary:bpo-33038: Fix gzip.GzipFile for file objects with a non-string name attribute. (GH-6095)files:A Misc/NEWS.d/next/Library/2018-03-10-20-14-36.bpo-33038.yA6CP5.rstM Lib/gzip.pyM Lib/test/test_gzip.pyM Misc/ACKSdiff --git a/Lib/gzip.py b/Lib/gzip.pyindex 07c6db493b0b..76ace394f482 100644--- a/Lib/gzip.py+++ b/Lib/gzip.py@@ -95,9 +95,8 @@ def __init__(self, filename=None, mode=None, if filename is None: # Issue #13781: os.fdopen() creates a fileobj with a bogus name # attribute. Avoid saving this in the gzip header's filename field.- if hasattr(fileobj, 'name') and fileobj.name != '':- filename = fileobj.name- else:+ filename = getattr(fileobj, 'name', '')+ if not isinstance(filename, basestring) or filename == '': filename = '' if mode is None: if hasattr(fileobj, 'mode'): mode = fileobj.modediff --git a/Lib/test/test_gzip.py b/Lib/test/test_gzip.pyindex 902d93fe043f..cdb1af5c3d13 100644--- a/Lib/test/test_gzip.py+++ b/Lib/test/test_gzip.py@@ -6,6 +6,7 @@ import os import io import struct+import tempfile gzip = test_support.import_module('gzip') data1 = """ int length=DEFAULTALLOC, err = Z_OK;@@ -331,6 +332,12 @@ def test_fileobj_from_fdopen(self): with gzip.GzipFile(fileobj=f, mode="w") as g: self.assertEqual(g.name, "") + def test_fileobj_from_io_open(self):+ fd = os.open(self.filename, os.O_WRONLY | os.O_CREAT)+ with io.open(fd, "wb") as f:+ with gzip.GzipFile(fileobj=f, mode="w") as g:+ self.assertEqual(g.name, "")+ def test_fileobj_mode(self): gzip.GzipFile(self.filename, "wb").close() with open(self.filename, "r+b") as f:@@ -359,6 +366,14 @@ def test_read_with_extra(self): with gzip.GzipFile(fileobj=io.BytesIO(gzdata)) as f: self.assertEqual(f.read(), b'Test') + def test_fileobj_without_name(self):+ # Issue #33038: GzipFile should not assume that file objects that have+ # a .name attribute use a non-None value.+ with tempfile.SpooledTemporaryFile() as f:+ with gzip.GzipFile(fileobj=f, mode='wb') as archive:+ archive.write(b'data')+ self.assertEqual(archive.name, '')+ def test_main(verbose=None): test_support.run_unittest(TestGzip) diff --git a/Misc/ACKS b/Misc/ACKSindex 580b0c5bf76d..458f31e6a6b7 100644--- a/Misc/ACKS+++ b/Misc/ACKS@@ -94,6 +94,7 @@ Michael R Bax Anthony Baxter Mike Bayer Samuel L. Bayer+Bo Bayles Donald Beaudry David Beazley Carlo Beccarinidiff --git a/Misc/NEWS.d/next/Library/2018-03-10-20-14-36.bpo-33038.yA6CP5.rst b/Misc/NEWS.d/next/Library/2018-03-10-20-14-36.bpo-33038.yA6CP5.rstnew file mode 100644index ..22d394b85ab7--- /dev/null+++ b/Misc/NEWS.d/next/Library/2018-03-10-20-14-36.bpo-33038.yA6CP5.rst@@ -0,0 +1,2 @@+gzip.GzipFile no longer produces an AttributeError exception when used with+a file object with a non-string name attribute. Patch by Bo Bayles.___Python-checkins mailing listpython-check...@python.orghttps://mail.python.org/mailman/listinfo/python-checkins___Python-Dev mailing listPython-Dev@python.orghttps://mail.python.org/mailman/listinfo/python-devUnsubscribe: https://mail.python.org/mailman/options/python-dev/nataliemorrisonxm980xm%40yahoo.com___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] bpo-33038: Fix gzip.GzipFile for file objects with a non-string name attribute. (GH-6095)
From: Serhiy Storchaka To: python-check...@python.org Sent: Wednesday, 9 May 2018, 10:14 Subject: [Python-checkins] bpo-33038: Fix gzip.GzipFile for file objects with a non-string name attribute. (GH-6095) https://github.com/python/cpython/commit/afe5f633e49e0e873d42088ae56819609c803ba0commit: afe5f633e49e0e873d42088ae56819609c803ba0branch: 2.7author: Bo Bayles <bbay...@gmail.com>committer: Serhiy Storchaka <storch...@gmail.com>date: 2018-05-09T13:14:40+03:00summary:bpo-33038: Fix gzip.GzipFile for file objects with a non-string name attribute. (GH-6095)files:A Misc/NEWS.d/next/Library/2018-03-10-20-14-36.bpo-33038.yA6CP5.rstM Lib/gzip.pyM Lib/test/test_gzip.pyM Misc/ACKSdiff --git a/Lib/gzip.py b/Lib/gzip.pyindex 07c6db493b0b..76ace394f482 100644--- a/Lib/gzip.py+++ b/Lib/gzip.py@@ -95,9 +95,8 @@ def __init__(self, filename=None, mode=None, if filename is None: # Issue #13781: os.fdopen() creates a fileobj with a bogus name # attribute. Avoid saving this in the gzip header's filename field.- if hasattr(fileobj, 'name') and fileobj.name != '':- filename = fileobj.name- else:+ filename = getattr(fileobj, 'name', '')+ if not isinstance(filename, basestring) or filename == '': filename = '' if mode is None: if hasattr(fileobj, 'mode'): mode = fileobj.modediff --git a/Lib/test/test_gzip.py b/Lib/test/test_gzip.pyindex 902d93fe043f..cdb1af5c3d13 100644--- a/Lib/test/test_gzip.py+++ b/Lib/test/test_gzip.py@@ -6,6 +6,7 @@ import os import io import struct+import tempfile gzip = test_support.import_module('gzip') data1 = """ int length=DEFAULTALLOC, err = Z_OK;@@ -331,6 +332,12 @@ def test_fileobj_from_fdopen(self): with gzip.GzipFile(fileobj=f, mode="w") as g: self.assertEqual(g.name, "") + def test_fileobj_from_io_open(self):+ fd = os.open(self.filename, os.O_WRONLY | os.O_CREAT)+ with io.open(fd, "wb") as f:+ with gzip.GzipFile(fileobj=f, mode="w") as g:+ self.assertEqual(g.name, "")+ def test_fileobj_mode(self): gzip.GzipFile(self.filename, "wb").close() with open(self.filename, "r+b") as f:@@ -359,6 +366,14 @@ def test_read_with_extra(self): with gzip.GzipFile(fileobj=io.BytesIO(gzdata)) as f: self.assertEqual(f.read(), b'Test') + def test_fileobj_without_name(self):+ # Issue #33038: GzipFile should not assume that file objects that have+ # a .name attribute use a non-None value.+ with tempfile.SpooledTemporaryFile() as f:+ with gzip.GzipFile(fileobj=f, mode='wb') as archive:+ archive.write(b'data')+ self.assertEqual(archive.name, '')+ def test_main(verbose=None): test_support.run_unittest(TestGzip) diff --git a/Misc/ACKS b/Misc/ACKSindex 580b0c5bf76d..458f31e6a6b7 100644--- a/Misc/ACKS+++ b/Misc/ACKS@@ -94,6 +94,7 @@ Michael R Bax Anthony Baxter Mike Bayer Samuel L. Bayer+Bo Bayles Donald Beaudry David Beazley Carlo Beccarinidiff --git a/Misc/NEWS.d/next/Library/2018-03-10-20-14-36.bpo-33038.yA6CP5.rst b/Misc/NEWS.d/next/Library/2018-03-10-20-14-36.bpo-33038.yA6CP5.rstnew file mode 100644index ..22d394b85ab7--- /dev/null+++ b/Misc/NEWS.d/next/Library/2018-03-10-20-14-36.bpo-33038.yA6CP5.rst@@ -0,0 +1,2 @@+gzip.GzipFile no longer produces an AttributeError exception when used with+a file object with a non-string name attribute. Patch by Bo Bayles.___Python-checkins mailing listpython-check...@python.orghttps://mail.python.org/mailman/listinfo/python-checkins___Python-Dev mailing listPython-Dev@python.orghttps://mail.python.org/mailman/listinfo/python-devUnsubscribe: https://mail.python.org/mailman/options/python-dev/nataliemorrisonxm980xm%40yahoo.com___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] bpo-33038: Fix gzip.GzipFile for file objects with a non-string name attribute. (GH-6095)
From: Serhiy Storchaka To: python-check...@python.org Sent: Wednesday, 9 May 2018, 10:14 Subject: [Python-checkins] bpo-33038: Fix gzip.GzipFile for file objects with a non-string name attribute. (GH-6095) https://github.com/python/cpython/commit/afe5f633e49e0e873d42088ae56819609c803ba0commit: afe5f633e49e0e873d42088ae56819609c803ba0branch: 2.7author: Bo Bayles <bbay...@gmail.com>committer: Serhiy Storchaka <storch...@gmail.com>date: 2018-05-09T13:14:40+03:00summary:bpo-33038: Fix gzip.GzipFile for file objects with a non-string name attribute. (GH-6095)files:A Misc/NEWS.d/next/Library/2018-03-10-20-14-36.bpo-33038.yA6CP5.rstM Lib/gzip.pyM Lib/test/test_gzip.pyM Misc/ACKSdiff --git a/Lib/gzip.py b/Lib/gzip.pyindex 07c6db493b0b..76ace394f482 100644--- a/Lib/gzip.py+++ b/Lib/gzip.py@@ -95,9 +95,8 @@ def __init__(self, filename=None, mode=None, if filename is None: # Issue #13781: os.fdopen() creates a fileobj with a bogus name # attribute. Avoid saving this in the gzip header's filename field.- if hasattr(fileobj, 'name') and fileobj.name != '':- filename = fileobj.name- else:+ filename = getattr(fileobj, 'name', '')+ if not isinstance(filename, basestring) or filename == '': filename = '' if mode is None: if hasattr(fileobj, 'mode'): mode = fileobj.modediff --git a/Lib/test/test_gzip.py b/Lib/test/test_gzip.pyindex 902d93fe043f..cdb1af5c3d13 100644--- a/Lib/test/test_gzip.py+++ b/Lib/test/test_gzip.py@@ -6,6 +6,7 @@ import os import io import struct+import tempfile gzip = test_support.import_module('gzip') data1 = """ int length=DEFAULTALLOC, err = Z_OK;@@ -331,6 +332,12 @@ def test_fileobj_from_fdopen(self): with gzip.GzipFile(fileobj=f, mode="w") as g: self.assertEqual(g.name, "") + def test_fileobj_from_io_open(self):+ fd = os.open(self.filename, os.O_WRONLY | os.O_CREAT)+ with io.open(fd, "wb") as f:+ with gzip.GzipFile(fileobj=f, mode="w") as g:+ self.assertEqual(g.name, "")+ def test_fileobj_mode(self): gzip.GzipFile(self.filename, "wb").close() with open(self.filename, "r+b") as f:@@ -359,6 +366,14 @@ def test_read_with_extra(self): with gzip.GzipFile(fileobj=io.BytesIO(gzdata)) as f: self.assertEqual(f.read(), b'Test') + def test_fileobj_without_name(self):+ # Issue #33038: GzipFile should not assume that file objects that have+ # a .name attribute use a non-None value.+ with tempfile.SpooledTemporaryFile() as f:+ with gzip.GzipFile(fileobj=f, mode='wb') as archive:+ archive.write(b'data')+ self.assertEqual(archive.name, '')+ def test_main(verbose=None): test_support.run_unittest(TestGzip) diff --git a/Misc/ACKS b/Misc/ACKSindex 580b0c5bf76d..458f31e6a6b7 100644--- a/Misc/ACKS+++ b/Misc/ACKS@@ -94,6 +94,7 @@ Michael R Bax Anthony Baxter Mike Bayer Samuel L. Bayer+Bo Bayles Donald Beaudry David Beazley Carlo Beccarinidiff --git a/Misc/NEWS.d/next/Library/2018-03-10-20-14-36.bpo-33038.yA6CP5.rst b/Misc/NEWS.d/next/Library/2018-03-10-20-14-36.bpo-33038.yA6CP5.rstnew file mode 100644index ..22d394b85ab7--- /dev/null+++ b/Misc/NEWS.d/next/Library/2018-03-10-20-14-36.bpo-33038.yA6CP5.rst@@ -0,0 +1,2 @@+gzip.GzipFile no longer produces an AttributeError exception when used with+a file object with a non-string name attribute. Patch by Bo Bayles.___Python-checkins mailing listpython-check...@python.orghttps://mail.python.org/mailman/listinfo/python-checkins___Python-Dev mailing listPython-Dev@python.orghttps://mail.python.org/mailman/listinfo/python-devUnsubscribe: https://mail.python.org/mailman/options/python-dev/nataliemorrisonxm980xm%40yahoo.com___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] thingy
Dear thingy, Please replace me with DZWORD. Put in HKEY\SYSTEM_IO_MEMORY\%USB%\%DZWORD%\%ADD\%CDATA\%DATA\ FI thingy Sent from Samsung Mobile___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Terminal console
fopen Terminal.app.python. 3.5.0.() def fopen Termina.app.python.3.5.0.() %add.%data(CDATA[])::true||false fclose(); end Terminal.app.python.3.5.0.() Yours thingy Sent from Samsung Mobile___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Feature request: Change a Dependency Package Version During Package Initiation
A lot of the Python code we use in production are used directly as imports in other python distributions (such as the python comes with the finite element software Abaqus and MSC Marc), many packages (such as matplotlib, numpy) that may have varying versioned dependencies, which makes it is a pain to get all these dependencies met in those close source python distributions. The work around is to let a version to be set within a package and have that propagate to all modules in that package. For example in the root init.py if I set tonado version to be 2.2.1 then all modules in that package will use tornado 2.2.1 when I import tornado. See a relevant issue of a similar package on github: https://github.com/mitsuhiko/multiversion/issues/1 Thank you! Qiang___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Feature request: Change a Dependency Package Version During Package Initiation
A lot of the Python code we use in production are used directly as imports in other python distributions (such as the python comes with the finite element software Abaqus and MSC Marc), many packages (such as matplotlib, numpy) that may have varying versioned dependencies. I was wondering if this could be expanded to allow a version to be set within a package and have that propagate to all modules in that package. For example in the root init.py if I set multiversion(tornado, 2.2.1) then all modules in that package will use tornado 2.2.1 when I import tornado. See a relevant issue on github: https://github.com/mitsuhiko/multiversion/issues/1 Thank you! Qiang___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Feature request: Change a Dependency Package Version During Package Initiation
>From the viewpoint of the package user, people don't want to change every >import to "require a version at runtime". If "set up a python path in a >wrapper script" is a better strategy, can you please give an example for the >following use case: The abaqus python distribution has libA 1.0, the external package to be installed is LibB, which requires libA2.0, and we don't want to change every import in LibB to be import libA 1.0, because the source code may have not been provided. In emacs, you can always advice a command to change its behaviour, which makes it very user friendly, and that what all the following suggestion is about. I have no idea at all how it could be implemented though. https://github.com/mitsuhiko/multiversion/issues/1 "The work around is to let a version to be set within a package and have that propagate to all modules in that package. For example in the root init.py if I set tonado version to be 2.2.1 then all modules in that package will use tornado 2.2.1 when I import tornado." 在 星期五, 2019-05-17 23:38:55 Daniel Holth 撰写 This sounds exactly like what people used to do with eggs. You could have multiple versions of a package on the path as eggs and then require a version at runtime. The approach has problems. Ruby also abandoned a strategy where random app code depends on package management code at runtime. One better strategy is to set up a python path in a wrapper script. On Fri, May 17, 2019, 11:27 Brett Cannon <mailto:bcan...@gmail.com> wrote: Thanks for the idea but there are currently no plans to support such a feature. If you would like to see it then you will need to write a PEP with a proof-of-concept to demonstrate how you would expect such a feature to work. On Fri., May 17, 2019, 07:55 Q via Python-Dev, <mailto:python-dev@python.org> wrote: A lot of the Python code we use in production are used directly as imports in other python distributions (such as the python comes with the finite element software Abaqus and MSC Marc), many packages (such as matplotlib, numpy) that may have varying versioned dependencies. I was wondering if this could be expanded to allow a version to be set within a package and have that propagate to all modules in that package. For example in the root init.py if I set multiversion(tornado, 2.2.1) then all modules in that package will use tornado 2.2.1 when I import tornado. See a relevant issue on github: https://github.com/mitsuhiko/multiversion/issues/1 Thank you! Qiang ___ Python-Dev mailing list mailto:Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/brett%40python.org ___ Python-Dev mailing list mailto:Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/dholth%40gmail.com_______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Feature request: Change a Dependency Package Version During Package Initiation
Sorry, there is an error in my last post. It's corrected as: The abaqus python distribution has libA 1.0, the external package to be installed is LibB, which requires libA2.0, and we don't want to change every import in LibB to be import /somewhere/libA2.0, because the source code may have not been provided. 在 星期六, 2019-05-18 10:26:04 Q 撰写 >From the viewpoint of the package user, people don't want to change every >import to "require a version at runtime". If "set up a python path in a >wrapper script" is a better strategy, can you please give an example for the >following use case: The abaqus python distribution has libA 1.0, the external package to be installed is LibB, which requires libA2.0, and we don't want to change every import in LibB to be import libA 1.0, because the source code may have not been provided. In emacs, you can always advice a command to change its behaviour, which makes it very user friendly, and that what all the following suggestion is about. I have no idea at all how it could be implemented though. https://github.com/mitsuhiko/multiversion/issues/1 "The work around is to let a version to be set within a package and have that propagate to all modules in that package. For example in the root init.py if I set tonado version to be 2.2.1 then all modules in that package will use tornado 2.2.1 when I import tornado." 在 星期五, 2019-05-17 23:38:55 Daniel Holth 撰写 This sounds exactly like what people used to do with eggs. You could have multiple versions of a package on the path as eggs and then require a version at runtime. The approach has problems. Ruby also abandoned a strategy where random app code depends on package management code at runtime. One better strategy is to set up a python path in a wrapper script. On Fri, May 17, 2019, 11:27 Brett Cannon <mailto:bcan...@gmail.com> wrote: Thanks for the idea but there are currently no plans to support such a feature. If you would like to see it then you will need to write a PEP with a proof-of-concept to demonstrate how you would expect such a feature to work. On Fri., May 17, 2019, 07:55 Q via Python-Dev, <mailto:python-dev@python.org> wrote: A lot of the Python code we use in production are used directly as imports in other python distributions (such as the python comes with the finite element software Abaqus and MSC Marc), many packages (such as matplotlib, numpy) that may have varying versioned dependencies. I was wondering if this could be expanded to allow a version to be set within a package and have that propagate to all modules in that package. For example in the root init.py if I set multiversion(tornado, 2.2.1) then all modules in that package will use tornado 2.2.1 when I import tornado. See a relevant issue on github: https://github.com/mitsuhiko/multiversion/issues/1 Thank you! Qiang ___ Python-Dev mailing list mailto:Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/brett%40python.org ___ Python-Dev mailing list mailto:Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/dholth%40gmail.com_______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP-376 and PEP-427 interpretation
Forgive me if this isn't the correct venue for this question and I ask your help directing me to the correct place if it is not. In PEP-376 it states with respect to the valid hashes in a Wheel RECORD file: "The hash is either the empty string or the hash algorithm as named in hashlib.algorithms_guaranteed, followed by the equals character =, followed by the urlsafe-base64-nopad encoding of the digest (base64.urlsafe_b64encode(digest) with trailing = removed)." In PEP-427 it further restricts the valid hashes to omit md5 and sha1 and says: "The hash algorithm must be sha256 or better; specifically, md5 and sha1 are not permitted." No where does it define what on what dimension of a hash "better" should consider. From the context talking about the security of the algorithm I'd infer that "better" is with respect to collision resistance. If so does that mean sha224 should also be excluded from wheel RECORD file hashes? Eldon___________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TP5DEDAW5TUQS44BY6RE2HHRRGOL56HZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP-376 and PEP-427 interpretation
No need to be condescending. Trust me when I say I know the bit length relates to the collision resistance. Also trust me when I say there are other dimensions upon which to consider one hash algo over another other then just collision resistance such as, power consumption, execution time, whether or not the algorithm suffers from length extension attacks. I'm assuming the reason MD5 and SHA1 were both disallowed were because they have been proven to have a collision resistance less then 1/2 their bit length. But this is not the case for SHA224. It is just a truncated version of SHA256 and thus the underlying algorithm is just as strong as SHA256 except that you can expect to find a collision in about 16 bits of work less. So going back to my actual question SHA224 is disallowed in record files because it's bit length is less then 256?_______ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NKMWTOLR5GVSKGYWPBHB7FGMD33IYCNK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP-376 and PEP-427 interpretation
Thank you. I can't think of a compelling reason someone would want to choose SHA224 over SHA256 in the context of wheel generation. It just that the PEPs are usually pretty explicit and SHA224 seemed to be implicitly excluded from RECORD files. And I'm considering the details of making a pretty pedantic wheel generation PEP517 backend. Eldon ‐‐‐ Original Message ‐‐‐ On Monday, March 29, 2021 2:16 PM, Paul Moore wrote: > On Mon, 29 Mar 2021 at 17:40, Theallredman via Python-Dev > python-dev@python.org wrote: > > > So going back to my actual question SHA224 is disallowed in record files > > because it's bit length is less then 256? > > It doesn't look like it's ever been excluded. The only explicit > exclusions are MD5 and SHA1 as you point out. Do you have a particular > reason to want to use SHA224? Pretty much everyone is using SHA256, as > far as I know. > > Paul _______ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NB6MLDDDJHRTRMOEWDFG5IYGZCP65K6V/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Help to Resolve issues with Pull request 25220
All, Can someone better than me (i.e anyone) help me resolve the issues with Pull Request 25220. I followed the dev guide, but I assume that between me taking my fork of the cpython repository, and building my pull request, another pull request was merged into master. It also appears that the Documentation build is failing with an rst reference tag error on part of the document I never changed. I am finding it difficult to drive github to understand how to resolve the conflict (i.e how to retain both sets of changes) - so any help would be appreciated. Regards, -- Anthony Flury *Moble*: +44 07743 282707 *Home*: +44 (0)1206 391294 *email*: anthony.fl...@btinternet.com <mailto:anthony.fl...@btinternet.com> <>_______ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EPLT4DTJCTOJZ26YNMBORTL77WFLMKGD/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Typing syntax and ecosystem
>> Aren't people allowed to have their own opinions? Absolutely >>>>> If criticism of any current implementation of any construct becomes >>>>> off-limits is automatically classed as "denigrating" Without judging the particular instance or the discussion itself, here is the definition of the world "ridiculous": Ridiculous: adjective. Deserving or inviting derision or mockery; absurd. I am quite sure that there a better ways to make a technical argument without describing the target of your objections as "deserving derision or mockery". You could say "I think the current state of X is quite painful" or "I think this is not acceptable" or "I think this is a really bad idea" or "I think that this is going to make a lot of things really bad" or "I really don't like this" or "I deeply think that this was one of the worst decisions". None of those statements are denigrating. Calling something "ridiculous", is, by design, denigrating. Not the worst way to denigrate but a way to denigrate, nevertheless. Rgds, Jossua___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/UTMMKFSU4OXNYVQJWBQPPKF3ND4S7MAP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Windows buildbots may be broken
It is really nice post. https://bit.ly/3fsxwwl ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WMJOVXKRH2ILPADU5QMLIVEIXHWPBOLZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 654 except* formatting
I agree that *(E1, E2) looks like unpacking, how about except *E1 as error: ... except (*E1, *E2) as error: ... even better would be if we could drop the braces: except *E1, *E2 as error: ... ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/PFYQC7XMYFAGOPU5C2YVMND2BQSIJPRC/ Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-Dev] [Webmaster] Unsafe listing by Norton's "File Insight"
(Response at bottom) On 6/07/2016 2:39 AM, Steve Dower wrote: On 04Jul2016 2241, Steve Holden wrote: Hi Peter, While the humble webmasters can do little about this it's possible the developers can, so I am forwarding your email to their mailing list. regards Steve Steve Holden On Tue, Jul 5, 2016 at 3:30 AM, Peter via Webmaster mailto:webmas...@python.org>> wrote: Hi I'm a heavy user of Python on Windows, am a Basic PSF member and have contributed to core Python. The Python 2.7.12 Windows installer download is being marked as untrusted by Norton Internet Security. I've been on chat with Symantec, and they've said that I can't do anything about that rating, but that the site owner can. I've been pointed to: https://safeweb.norton.com/help/site_owners Interestingly, the 3.5.2 download is flagged as safe. Hoping to get more Python out to users! Thanks Peter Peter, can you provide the exact URL that safeweb is complaining about? I tried a few at https://safeweb.norton.com/ and they all showed up as clean. Also please clarify whether this is what you mean. It's not entirely clear whether the download is being scanned or the reputation of the URL is in question. Cheers, Steve Hi Steve It's not the URL it is complaining of, rather On Windows, Norton Internet Security virus checks all downloads. One of the names they give to the result of their scanning is 'File Insight'. From what I can tell, it uses a few checks: - virus scanning using known signatures - observable malicious behaviour - how well known or used it is across other users of Nortons. It seems that the last of these is a causing the warning. Obviously this is not a problem for me, but may be concerning for less tech savvy Windows users that get the warning. There isn't a way within Nortons to 'report for clearance' of the file. And my chat with a (entry level) Norton representative got nowhere. I'll email screen captures in the next email. Let me know if they don't come through and I'll paste them somewhere. Let me know if I can give any more information. Peter ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Webmaster] Unsafe listing by Norton's "File Insight"
On 6/07/2016 9:51 AM, Steve Dower wrote: On 05Jul2016 1615, Peter wrote: It's not the URL it is complaining of, rather On Windows, Norton Internet Security virus checks all downloads. One of the names they give to the result of their scanning is 'File Insight'. From what I can tell, it uses a few checks: - virus scanning using known signatures - observable malicious behaviour - how well known or used it is across other users of Nortons. It seems that the last of these is a causing the warning. Obviously this is not a problem for me, but may be concerning for less tech savvy Windows users that get the warning. There isn't a way within Nortons to 'report for clearance' of the file. And my chat with a (entry level) Norton representative got nowhere. I'll email screen captures in the next email. Let me know if they don't come through and I'll paste them somewhere. Let me know if I can give any more information. I don't think there's anything we can do about this, other than convince more users of Norton Internet Security to use Python 2.7 (apparently Python 3.5 is more popular with that demographic :) ) The installer is signed with the PSF's certificate, which keeps Windows Smartscreen happy and is the only way we can indicate that it's trustworthy. If Norton requires different criteria then that is on them and not something we can fix. Cheers, Steve I suspect you're right. It's a flawed model that they're using, and they are quite impervious to suggestions. Glad 3.5 is winning :-) Keep up the good work. Peter ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Enum conversions in the stdlib
On Thu, Mar 02, 2017 at 04:13:17PM -0800, Ethan Furman wrote: > The resulting enumeration is neither in alpha nor value order. While this > has no bearing on programmatic usage I would like these Enums to be ordered, > preferably by value. > > Would anyone prefer lexicographical ordering, and if so, why? I just tried on my system with python 3.6: ``` >>> pprint(list(signal.Signals)) [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] ``` so I'm not sure what the issue is, but #worksforme. -- zmo ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible bug in class-init, lookin for mentors
On Fri, Apr 21, 2017 at 11:47:24AM +0200, Justus Schwabedal wrote: > At least I think it's a bug. Maybe it's a feature.. it's indeed a feature. > I possibly found a bug in class __init__ and would like to fix it technically, it's a method. More precisely, it's the constructor method. > So I'm looking for a mentor to help me. > > class Foo: > def __init__(self, bar=[]): > self.list = bar > > spam_1 = Foo() > spam_2 = Foo() > > spam_1.list.append(42) > print(spam_2.list)` the argument `bar` of your method is instanciated at the time you're declaring the method. It's happening once for the the lifetime of the execution of your code. By allocating the `bar` reference into the `self.list` member, you're assigning the same *instance* of that list into the `self.list` member. So everytime you create a new Foo instance, you're actually assigning the same `[]` instance into `self.list` which is why, when you mutate the list, it's happening in all the instances of Foo as well. I hope it makes sense to you ! -- Guyzmo ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] "Micro-optimisations can speed up CPython"
On Mon, May 29, 2017 at 05:15:43PM +0300, Serhiy Storchaka wrote: > Interesting articles, thank you. I wonder why the author doesn't propose his > patches for CPython. Does he fear that CPython can become faster than Lua? > ;-) the author's answer: https://twitter.com/corsix/status/869200284261789696 😉 Cheers, -- zmo _______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] for...else
Hello, I think that the expression "for...else" or "while...else" is completely counter-intuitive. Wouldn't it be possible to make it clearer? Maybe something like break in for i in range(n): ... if cond: break else: ... I'm not an English native speaker so I don't know whether "break in" is acceptable English in this context or can only mean "to get into a building by force". Kiuhnm ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Increase of Spammy PRs and PR reviews
As someone who is watching the python/cpython repository, I'm very used to see lots of traffic. But lately there have been a surge of spammy PRs which are about the same, generally very trivial subject but individually fixing each occurrence one by one: - Add coverage to X (tens of them, as separate PRs) - Make `test_xxx` executable with direct invocation (tens of them, as separate PRs) - Lint source with flake8, fix linting errors (stylistic changes) And lots of non-committer PR reviews that only approve. Am I the only one who feels like this is only done to 'grind' commits (get a higher number of commits into the repository, and boast about it)? In the past there were one or two people who would submit typo fixes, but most of them weren't making it continuously. The situation right now feels much worse than those._______ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7RGI4LUJSEKU2JUYSV7UMZ2CXRGAANEF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Increase of Spammy PRs and PR reviews
> Gaming the system doesn't end up working well in the end anyway. The first > time the gamers try to get a job interview and can't explain how they'd do a > code review—something GitHub says they've done hundreds or thousands of > times—the whole thing will fail. Observably, it feels like they are doing this for core privileges (if they don't already exist, they are a member of the python org?). Every time I see one of those PRs (e.g add test for X, add delete redundant variables for Y), the author seem to be cc-ing their mentor. This gives a bad impression to others about their intentions (constant contribution of trivial / low quality stuff with little-to-no-gain to achieve a higher number of commits, since it is a visible metric)._______________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JZBQ2UYTXDCHADW4LEPGPE5SFLRHW5E3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Importing a submodule doesn't always set an attribute on its parent
Hello, I came across what seems like either a bug in the import system or a gap in its documentation, so I'd like to run it by folks here to see if I should submit a bug report. If there's somewhere else more appropriate to discuss this, please let me know. If you import A.B, then remove A from sys.modules and import A.B again, the newly-loaded version of A will not contain an attribute referring to B. Using "collections.abc" as an example submodule from the standard library: >>> import sys >>> import collections.abc >>> del sys.modules['collections'] >>> import collections.abc >>> collections.abc Traceback (most recent call last): File "", line 1, in AttributeError: module 'collections' has no attribute 'abc' This behavior seems quite counter-intuitive to me: why should the fact that B is already loaded prevent adding a reference to it to A? It also goes against the general principle that "import FOO" makes the expression "FOO" well-defined; for example PLR 5.7 states that "'import XXX.YYY.ZZZ' should expose 'XXX.YYY.ZZZ' as a usable expression". Finally, it violates the "invariant" stated in PLR 5.4.2 that if 'A' and 'A.B' both appear in sys.modules, then A.B must be defined and refer to sys.modules['A.B']. On the other hand, PLR 5.4.2 also states that "when a submodule is loaded using any mechanism... a binding is placed in the parent module's namespace to the submodule object", which is consistent with the behavior above, since the second import of A.B does not actually "load" B (only retrieve it from the sys.modules cache). So perhaps Python is working as intended here, and there is an unwritten assumption that if you unload a module from the cache, you must also unload all of its submodules. If so, I think this needs to be added to the documentation (which currently places no restrictions on how you can modify sys.modules, as far as I can tell). This may be an obscure corner case that is unlikely to come up in practice (I imagine few people need to modify sys.modules), but it did actually cause a bug in a project I work on, where it is necessary to uncache certain modules so that they can be reloaded. I was able to fix the bug some other way, but I think it would still be worthwhile to either make the import behavior more consistent (so that 'import A.B' always sets the B attribute of A) or add a warning in the documentation about this case. I'd appreciate any thoughts on this! Thanks, Daniel ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VIPXZRK3OJNSVNSZSAJ7CO6QFC2RX27W/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Importing a submodule doesn't always set an attribute on its parent
Thanks, Brett. I understand why the behavior happens, I just don't understand the decision to implement imports this way. Since there's no warning in the documentation that removing items from sys.modules can break the fact that "import X.Y" defines "X.Y" (note that the "behind the curtain" stuff happens *before* the second import, so it's still the case that the second import does not define "X.Y" as implied by the docs), and there's also no warning that submodules must be removed at the same time as their parent, I would expect my example code to work. I don't see any downside to having "import X.Y" always set the Y attribute of X (instead of only setting it if 'X.Y' is not already in sys.modules), but if you think it's a bad idea, here's a suggestion for a paragraph to add at the end of PLR 5.4.2: "Note that the binding to the submodule object in the parent module's namespace is only added when the submodule is actually *loaded*. If the submodule is already present in `sys.modules` when it is imported (through any of the mechanisms above), then it will not be loaded again and no binding will be added to the parent module." If removing a module but not its submodules from sys.modules is considered "cheating" and could potentially break other parts of the import system, that should also be documented, e.g. by adding the sentence "If you delete a key for a module in `sys.modules`, you must also delete the keys for all submodules of that module." at the end of the 3rd paragraph of PLR 5.3.1. However, I would much rather not impose this restriction, since it seems unnecessarily restrictive (indeed, my code violates it but works fine, and changing it to transitively remove all submodules would necessitate reloading many modules which do not actually need to be reloaded). (Terry, thanks for your suggestion. My concern about adding such a vague warning is that to me, it reads as saying that all bets are off if you modify sys.modules by hand, which means it would never be safe to do so, i.e., the behavior might change arbitrarily in a future Python version. But in my opinion there are legitimate cases where it is necessary to ensure a module will be reloaded the next time it is imported, and the documented way to do that is to remove entries from sys.modules.) Daniel _______ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/N763W6AGD6NQ4IXVWMNGDL4DBN3LXBJ7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Importing a submodule doesn't always set an attribute on its parent
Brett Cannon wrote: > So we don't want to strengthen the definition at > all; best we are comfortable with is put up a warning that you don't want > to do stuff with sys.modules unless you know what you're doing. OK, thanks for the clarification. Having read through the source of importlib one too many times, I guess I will declare that I "know what [I'm] doing" for now and keep on mutating sys.modules, since the alternative (intercepting all imports) seems more painful to me. If my code breaks in a future Python version I'll only blame myself :) Best, Daniel _______________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HZEPOAI3YME4GD2M6RPWG2KG4OTSB5KX/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] [CVE-2022-37454] SHA3 vulnerability and upcoming Python patches for 3.7 - 3.10
I’m looking for help understanding how Python will release fixes related to the SHA3 critical security vulnerability (CVE-2022-37454). I’ve tried to figure this out myself, but I’m far from a Python expert and I’m not sure where else I should look. Apologies in advance if this is the wrong place to ask - if it is, a redirect to the correct place would be most appreciated. Here’s what I’ve found so far: * Python versions 3.6 through 3.10 appear to be affected * 3.6 is end of life, so no fix is expected * A code fix appears to have been applied to 3.7 through 3.10 https://github.com/python/cpython/issues/98517 * 3.9 and 3.10 by default use OpenSSL1.1.1+ if it’s available, appearing to default to the builtin, vulnerable SHA3 implementation if OpenSSL is not found (if there’s an exception) * 3.9 introduced this change via bpo-37630 in release 3.9.0 beta1 * 3.10 appears to have had this functionality since it was originally released * 3.11 uses tiny_sha3 and AFAICT was never affected by the CVE But what I’m having trouble figuring out is when/how these fixes will become generally available and ready for users of Python to download. * When will patched releases for Python 3.7-3.10 be released? * If pending releases are not in the release pipeline, what other patching opportunities exist? Ultimately I’m trying to set patching expectations for my company’s engineering teams who are still running vulnerable versions of Python. More notes around what i’ve found, in case it helps clarify my questions: From the Python project GitHub I can see gh-98517 to fix the buffer overflow in Python’s internal _sha3 module (CVE-2022-37454) has been committed to the Python 3.7 - 3.10 branches. I understand that for Python releases 3.9 and 3.10 if one is using the OpenSSL 1.1.1+ sha3 modules instead of the internal _sha3 module that is already a mitigation. I also understand that Python 3.11 and later has switched to using tiny_sha3, and no longer relies on the vulnerable _sha3 module. Any information you could point me at would be most helpful. If there is a more ideal forum to raise this question, please redirect me there. Thank you in advance ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/K7IYZUGOOLCGKZOLCZ27RSWZ7OWIP575/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: [CVE-2022-37454] SHA3 vulnerability and upcoming Python patches for 3.7 - 3.10
Thank you all for your responses! Best, Mark ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WWVXPV5BOFOHE6ENRNSLZAQAP22E5ZLK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Cassandra-driver acquiring tstate_lock in python3.6/threading.py while doing cluster.connect and got stuck forever
Hi, We are running cassandra using mock cassandra and interacting with python cassandra-driver. We are running python3.6 and we are using latest cassandra-driver i.e cassandra-driver 3.21.0. What we are seeing is when calling cluster.connect it is trying to acquire a tstate_lock from here *https://github.com/python/cpython/blob/3.6/Lib/threading.py#L1072* but we are running another process which has acquired this lock already, here *https://github.com/python/cpython/blob/3.6/Lib/threading.py#L899* but here the lock that is acquired is not getting released, it will be released after the process who has acquired the lock gets killed. So my program got stuck at cluster.connect call forever. My question is why the lock which has been acquired here *https://github.com/python/cpython/blob/3.6/Lib/threading.py#L899* not getting released there itself. It will be released when process gets killed. Due to this my program is getting stuck forever. Traceback (most recent call first): File "/usr/lib64/python3.6/threading.py", line 1072, in _wait_for_tstate_lock elif lock.acquire(block, timeout): File "/usr/lib64/python3.6/threading.py", line 1056, in join self._wait_for_tstate_lock() File "/root/mockcassandra.py", line 179, in verify_cassandra session = cluster.connect() In the traceback, cluster.connect are trying to acquire the tstate_lock which is already acquired by another process and not releasing it until that process gets killed. In my case both process has to run in parallel. Can anyone suggest how can we proceed further? ___________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6EILC2GPPMX3QQQZT3L47PKRH4DR36T5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Aaron Hall, Introduction and pull-request bump
Hi there,
I have had the privilege of getting to know some core developers at PyCon in
Portland, and I've met some others through my volunteering at PyGotham and the
NYC Python meetup group (resisting urge to name-drop).
In spite of not joining the mailing list until about a week ago at Brett
Cannon's suggestion, I have managed to submit 3 bug-report fixes, with 1 closed
by pull-request, 1 nearly closed with the pull-request accepted, and 1 with the
pull request not yet accepted (looks like there's a conflict in misc/NEWS now,
not sure how to use Github to resolve it either). That last pull request is
here: bpo-26103 resolve Data descriptor contradiction by aaronchall · Pull
Request #1959 · python/cpython - probably good if
| | |
bpo-26103 resolve Data descriptor contr...
|
I also had a "MutableNamedTuple" implementation I put on
codereview.stackexchange.com
(https://codereview.stackexchange.com/q/173045/23451) and Ashwini Chaudhury
gave me some excellent pointers, and inspired by the recent discussion on "Data
Classes" I now have a half-baked implementation that leverages __slots__ with a
Mapping, for example (from a unittest, hence the indentation):
class MNTDemo(MutableNamedTuple):
"Demo"
__slots__ = {
'arg0': arg(),
'arg1': arg(typing.List),
'kwarg': arg(default=0), # must be positional if passed *args
'args': arg(stars=Stars.ARGS),
# everything after *args must be keyword argument
'kwarg1': arg(list, []),
'kwarg2': None,
'kwargs': arg(typing.Any, stars=Stars.KWARGS)
}
mnt = MNTDemo(1, 2, 3, 'stararg1',
kwarg1=1, kwarg2=2, kwarg3=3)
Maybe there's a nicer way to do it, but arg is a function returning a
namedtuple (attrs: type, default, stars) with some defaults. It allows the
__init__ to have required positional arguments and required named arguments,
pretty much as flexible as a regular function signature, but communicated via
__slots__. And thanks to the slots, they're every bit as performant as a
namedtuple, I presume (I gave the __slots__ talk at PyCon).
Anyways, to wrap up my introduction, I'm a moderator on StackOverflow
(https://stackoverflow.com/users/541136/aaron-hall?tab=profile) and nearly
about to catch Raymond in reputation points, mostly due to Python, I'm
currently teaching Python at NYU (Introduction to Programming), and I'm an
all-around Python evangelist (as much as I know how to be.) I owe a lot to the
Python community, especially the meetup community in NYC, but also virtual
community (Ned Batchelder in IRC comes to mind).
Thank you for everything, I'm looking for my chance to give back!
Cheers,
Aaron Hall___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__
Thanks for working on this and writing up the details Victor. For those confused about why this matters, routinely having every object in your application participating in one (or more) giant reference cycles makes reasoning about and fixing resource issues very difficult. For instance, a while back I made some changes in Bazaar so that each TestCase instance was dereferenced after being run: <https://bugs.launchpad.net/bzr/+bug/613247> Which mattered when running ~25000 tests, to keep peak memory usage sane. Also there were various other object lifetime issues, and tracking down which specific test failed to join a thread, or close a file on Windows was much simpler after making sure cleanup actually happened at the time a test was deleted. Keeping more stuff alive for longer periods also makes peak memory requirements higher, and the gc has to do more work each time. On 18/09/2017, Victor Stinner wrote: > > Ideally, CPython 3.x should never create reference cycles. Removing > Exception.__traceback__ is the obvious "fix" for the issue. But I > expect that slowly, a lot of code started to rely on the attribute, > maybe even for good reasons :-) > > A more practical solution would be to log a warning. Maybe the garbage > collector can emit a warning if it detects an exception part of a > reference cycle? Or maybe detect frames? Logging a warning is unlikely to be practical. I had an optional test flag that complained about reference cycles, and it produced a lot of output. Martin _______________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] \G (match last position) regex operator non-existant in python?
On 28 October 2017 at 16:48, Steven D'Aprano wrote: > On Sun, Oct 29, 2017 at 12:31:01AM +0100, MRAB wrote: > > > Not that I'm planning on making any further additions, just bug fixes > > and updates to follow the Unicode updates. I think I've crammed enough > > into it already. There's only so much you can do with the regex syntax > > with its handful of metacharacters and possible escape sequences... > > What do you think of the Perl 6 regex syntax? > > https://en.wikipedia.org/wiki/Perl_6_rules#Changes_from_Perl_5 If you're going to change the notation, why not use notations similar to what linguists use for FSTs? These allow building FSTs (with operations such as adding/subtracting/composing/projecting FSTs) with millions of states — and there are some impressive optimisers for them also, so that encoding a dictionary with inflections is both more compact and faster than a hash of just the words without inflections. Some of this work is open source, but I haven't kept up with it. If you're interested, you can start here: http://web.stanford.edu/~laurik/ http://web.stanford.edu/~laurik/publications/TR-2010-01.pdf http://web.stanford.edu/group/cslipublications/cslipublications/site/1575864347.shtml etc. ;) > > > > > -- > Steve > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > pludemann%40google.com > ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Official port of Python on EDK2
Hi, UEFI has become the standard for firmware (BIOS) interface. Intel has provided an open source implementation under the name EDK2 (part of the TianoCore initiative) [1] for some time. This implementation has evolved significantly and now provides the functionalities of a small OS with a standard library similar to POSIX. In 2011, a port of Python 2.7.1 was added to the EDK2 repository [2]. This port then evolved to 2.7.2 which is still defined as the reference port [3]. In 2015, another port was added of Python 2.7.10 in parallel of 2.7.2 [4]. Since then, both implementations have diverged from upstream and know vulnerabilities have not been fixed. I would like to bring support for edk2 in the official Python repository to remediate this situation, that is officially support edk2 as a platform. Technically, there would be three main aspects for the on-boarding work: 1) Fix headers and source to resolve definition conflicts, similarly to ABS definition in [5]; 2) Add the edk2module.c [6] to handle platform-specific functionalities, similarly to the posixmodule.c; 3) Add the build configuration file [7] and necessary modifications within Python to handle the edk2 toolchain; This work would target the master branch (that is Python 3). I would be interested in hearing your thoughts on this idea. Thanks, Thiebaud [1] https://github.com/tianocore/edk2 [2] https://github.com/tianocore/edk2/commit/006fecd5a177b4b7b6b36fab6690bf2b2fa11829 [3] https://github.com/tianocore/edk2/blob/master/AppPkg/Applications/Python/PythonReadMe.txt [4] https://github.com/tianocore/edk2/commit/c8042e10763bca064df257547d04ae3dfcdfaf91 [5] https://gist.github.com/tweksteen/ed516ca7ab7dfa8d18428f59d9c22a3e [6] https://github.com/tianocore/edk2/blob/master/AppPkg/Applications/Python/Efi/edk2module.c [7] https://github.com/tianocore/edk2/blob/master/AppPkg/Applications/Python/PythonCore.inf ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [edk2] Official port of Python on EDK2
Christian, Antoine, Brett: Thanks for the clarification on what an official support would require. As Christian mentioned, sending simple headers patches is an obvious starting point, no matter if the support becomes official or not. Brian: Thanks for your email. As I suggested, by having the support directly within the Python community, you would avoid having to maintain a separate port. I don't think that having a new Python3 port as part of EDK2 is a good idea. What I am suggesting is that Intel should contribute directly to the Python repository by sending your modifications upstream and not expect someone to re-import Python into EDK2. That is, bringing your UEFI experience to Python and not the opposite. This would be a much better use of anyone's time. Thanks, Thiebaud On Thu, Nov 2, 2017 at 2:36 AM, Richardson, Brian < brian.richard...@intel.com> wrote: > Thiebaud: > > Thank you. I have started discussions within Intel for updating the UEFI > CPython implementation to Python 3.x. The TianoCore community would > appreciate contributions by people with Python experience to bring this > code up to current standards. > > Please review the contribution guidelines for TianoCore and let me know if > you have any questions. > http://www.tianocore.org/contrib/ > > Thanks ... br > --- > Brian Richardson, Senior Technical Marketing Engineer, Intel Software > brian.richard...@intel.com -- @intel_brian (Twitter & WeChat) > https://software.intel.com/en-us/meet-the-developers/evangel > ists/team/brian-richardson > > -Original Message- > From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of > Thiebaud Weksteen > Sent: Wednesday, November 1, 2017 5:07 AM > To: python-dev@python.org > Cc: edk2-de...@lists.01.org > Subject: [edk2] Official port of Python on EDK2 > > Hi, > > UEFI has become the standard for firmware (BIOS) interface. Intel has > provided an open source implementation under the name EDK2 (part of the > TianoCore initiative) [1] for some time. This implementation has evolved > significantly and now provides the functionalities of a small OS with a > standard library similar to POSIX. > > In 2011, a port of Python 2.7.1 was added to the EDK2 repository [2]. > This port then evolved to 2.7.2 which is still defined as the reference > port [3]. In 2015, another port was added of Python 2.7.10 in parallel of > 2.7.2 [4]. Since then, both implementations have diverged from upstream and > know vulnerabilities have not been fixed. > > I would like to bring support for edk2 in the official Python repository > to remediate this situation, that is officially support > edk2 as a platform. Technically, there would be three main aspects for the > on-boarding work: > > 1) Fix headers and source to resolve definition conflicts, similarly to > ABS definition in [5]; > 2) Add the edk2module.c [6] to handle platform-specific functionalities, > similarly to the posixmodule.c; > 3) Add the build configuration file [7] and necessary modifications within > Python to handle the edk2 toolchain; > > This work would target the master branch (that is Python 3). I would be > interested in hearing your thoughts on this idea. > > Thanks, > Thiebaud > > [1] https://github.com/tianocore/edk2 > [2] https://github.com/tianocore/edk2/commit/006fecd5a177b4b7b6b > 36fab6690bf2b2fa11829 > [3] https://github.com/tianocore/edk2/blob/master/AppPkg/Applica > tions/Python/PythonReadMe.txt > [4] https://github.com/tianocore/edk2/commit/c8042e10763bca064df > 257547d04ae3dfcdfaf91 > [5] https://gist.github.com/tweksteen/ed516ca7ab7dfa8d18428f59d9c22a3e > [6] https://github.com/tianocore/edk2/blob/master/AppPkg/Applica > tions/Python/Efi/edk2module.c > [7] https://github.com/tianocore/edk2/blob/master/AppPkg/Applica > tions/Python/PythonCore.inf > ___ > edk2-devel mailing list > edk2-de...@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations
If type annotations are treated like implicit lambdas, then that's a first step to something similar to Lisp's "special forms". A full generalization of that would allow, for example, logging.debug to not evaluate its args unless debugging is turned on (I use a logging.debug wrapper that allows lambdas as args, and evaluates them if debugging is turned on). Maybe a better question is whether we want "special forms" in Python. It complicates some things but simplifies others. But things that satisfy Lisp programmers might not make Python programmers happy. ;) On 4 November 2017 at 09:42, Guido van Rossum wrote: > I'm very worried about trying to come up with a robust implementation of > this in under 12 weeks. By contrast, the stringification that Łukasz is > proposing feels eminently doable. > > On Sat, Nov 4, 2017 at 6:51 AM, Nick Coghlan wrote: > >> On 4 November 2017 at 00:40, Guido van Rossum wrote: >> > IMO the inability of referencing class-level definitions from >> annotations on >> > methods pretty much kills this idea. >> >> If we decided we wanted to make it work, I think the key runtime >> building block we would need is a new kind of cell reference: an >> IndirectAttributeCell. >> >> Those would present the same interface as a regular nonlocal cell (so >> it could be stored in __closure__ just as regular cells are, and >> accessed the same way when the function body is executed), but >> internally it would hold two references: >> >> - one to another cell object (__class__ for this use case) >> - an attribute name on the target object that get/set/del operations >> on the indirect cell's value should affect >> >> As Python code: >> >> class IndirectAttributeCell: >> def __new__(cls, cell, attr): >> self._cell = cell >> self._attr = attr >> >> @property >> def cell_contents(self): >> return getattr(self._cell.cell_contents, self._attr) >> >> @cell_contents.setter >> def cell_contents(self, value): >> setattr(self._cell.cell_contents, self._attr, value) >> >> @cell_contents.deleter >> def cell_contents(self): >> delattr(self._cell.cell_contents, self._attr) >> >> The class body wouldn't be able to evaluate the thunks (since >> `__class__` wouldn't be set yet), but `__init_subclass__` >> implementations could, as could class decorators. >> >> It would require some adjustment in the compiler as well (in order to >> pass the class level attribute definitions down to these implicitly >> defined scopes as a new kind of accessible external namespace during >> the symbol analysis pass, as well as to register the use of >> "__class__" if one of the affected names was referenced), but I think >> it would work at least at a technical level (by contrast, every other >> idea I came up with back when I was working on the list comprehension >> change was sufficiently flawed that it fell apart within a few hours >> of starting to tinker with the idea). >> >> As an added bonus, we could potentially also extend the same >> permissive name resolution semantics to the implicit scopes used in >> comprehensions, such that it was only the explicitly defined scopes >> (i.e. lambda expressions, function definitions, and nested classes) >> that lost implicit access to the class level variables. >> >> Cheers, >> Nick. >> >> P.S. If we subsequently decided to elevate expression thunks to a >> first class language primitive, they shouldn't need any further >> semantic enhancements beyond that one, since the existing scoping >> rules already give the desired behaviour at module and function scope. >> >> -- >> Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia >> > > > > -- > --Guido van Rossum (python.org/~guido) > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > pludemann%40google.com > > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Guarantee ordered dict literals in v3.7?
Isn't ordered dict also useful for **kwargs? If it turns out that there's a dict implementation that's faster by not preserving order, collections.UnorderedDict could be added. There could also be specialized implementations that pre-size the dict (cf: C++ unordered_map::reserve), etc., etc. But these are all future things, which might not be necessary. On 5 November 2017 at 12:44, Sven R. Kunze wrote: > +1 from me too. > > On 04.11.2017 21:55, Jim Baker wrote: > > +1, as Guido correctly recalls, this language guarantee will work well > with Jython when we get to the point of implementing 3.7+. > > On Sat, Nov 4, 2017 at 12:35 PM, Guido van Rossum > wrote: > >> This sounds reasonable -- I think when we introduced this in 3.6 we were >> worried that other implementations (e.g. Jython) would have a problem with >> this, but AFAIK they've reported back that they can do this just fine. So >> let's just document this as a language guarantee. >> >> On Sat, Nov 4, 2017 at 10:30 AM, Stefan Krah wrote: >> >>> >>> Hello, >>> >>> would it be possible to guarantee that dict literals are ordered in v3.7? >>> >>> >>> The issue is well-known and the workarounds are tedious, example: >>> >>>https://mail.python.org/pipermail/python-ideas/2015-Decembe >>> r/037423.html >>> >>> >>> If the feature is guaranteed now, people can rely on it around v3.9. >>> >>> >>> >>> Stefan Krah >>> >>> >>> >>> ___ >>> Python-Dev mailing list >>> Python-Dev@python.org >>> https://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: https://mail.python.org/mailma >>> n/options/python-dev/guido%40python.org >>> >> >> >> >> -- >> --Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>) >> >> _______ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: https://mail.python.org/mailman/options/python-dev/jbaker% >> 40zyasoft.com >> >> > > > ___________ > Python-Dev mailing > listPython-Dev@python.orghttps://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/srkunze%40mail.de > > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > pludemann%40google.com > > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The current dict is not an "OrderedDict"
Does it matter whether the dict order after pop/delete is explicitly specified, or just specified that it's deterministic? On 7 November 2017 at 11:28, Antoine Pitrou wrote: > On Wed, 8 Nov 2017 06:19:46 +1100 > Chris Angelico wrote: > > > > I've used a good few dictionary objects in my time, but most of them > > have literally never had any items deleted from them. > > Well... It really depends what kind of problem you're solving. I > certainly delete or pop items from dicts quite often. > > Let's not claim that deleting items from a dict is a rare or advanced > feature. It is not. > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > pludemann%40google.com > ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations
On 10 November 2017 at 19:17, Greg Ewing wrote: > Ethan Furman wrote: > >> Contriwise, "annotation_strings" sounds like a different type of >> annotation -- they are now being stored as strings, instead of something >> else. >> > > How about "annotations_as_strings"? That feels unambiguous. "annotations_to_str" is shorter, given that "str" is a type in Python, and "to" says that it's converting *to* string (it's given *as* an expression). > > > -- > Greg > > ___________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/pludemann > %40google.com > ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] "CPython loves your Pull Requests" talk by Stéphane Wirtel
Hi, Thank you for this post to python-dev. About my talk, it was a real pleasure to give it at PyCon Canada, and I hope I could propose it to PyCon US for a larger public. But the goal behind this talk was to show that we have a good community, firstly by the external contributors and by the core-dev. For the statistics, I used the REST API v3 of GitHub and matplotlib. For the future, I would like to update them via a service running on my own server and maybe submit it to the Python Software Foundation, because I think it's a good indicator for the future contributors of the project. But seriously, I was surprised by the number of Pull Requests and by the number of contributors from Feb 2017 to Oct 2017. Here is my graph for October and November 2017. I will share my scripts on Github and if you want to help me with some good ideas, you are welcome. Stéphane Le 05/12/17 à 15:25, Victor Stinner a écrit : > Hi, > > Stéphane Wirtel gave a talk last month at Pycon CA about CPython pull > requests. His slides: > >https://speakerdeck.com/matrixise/cpython-loves-your-pull-requests > > He produced interesting statistics that we didn't have before on pull > requests (PR), from February 2017 to October 2017: > > * total number of merged PR: 4204 > * number of contributors: 586 !!! (96%) > * number of core developers: 27 (4%) > * Time to merge a PR: 3 days in average, good! > * etc. > > It would be nice to get these statistics updated regularly on a > service running somewhere. > > By the way, I'm also looking for statistics on reviews on GitHub. Does > someone know how to do that? > > Victor > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] "CPython loves your Pull Requests" talk by Stéphane Wirtel
Hi Mariatta, Thank you, I was really happy to see you at my talk, usually this kind of talk is boring ;-) just kidding, but usually I prefer a technical talk. Le 05/12/17 à 16:25, Mariatta Wijaya a écrit : > I saw the talk in person :) Congrats Stéphane! > > You can get the reviews from a specific PR using the API: > https://developer.github.com/v3/pulls/reviews/#list-reviews-on-a-pull-request > > For example, for reviews made to CPython PR number 1: > > https://api.github.com/repos/python/cpython/pulls/1/reviews > > * Time to merge a PR: 3 days in average, good! > > > Regarding the average time to merge PR, I'm interested to know the > average time to merge for PRs not made by Python Core Devs. +1 I could add this point in my scripts. Have a nice day and thank you for your feedback. Stéphane _______________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issues with PEP 526 Variable Notation at the class level
I'm not a typing expert, but I want to second Raymond's concerns, and perhaps I'm qualified to do so as I gave the PyCon USA __slots__ talk this year and I have a highly voted answer describing them on Stack Overflow. Beautiful thing we're doing here with the dataclasses, by the way. I think addressing the slots issue could be a killer feature of dataclasses. I hope this doesn't muddy the water: If I could change a couple of things about __slots__ it would be 1. to allow multiple inheritance with multiple parents with nonempty slots (raises "TypeError: multiple bases have instance lay-out conflict"), and 2. to avoid creating redundant slots if extant in a parent (but maybe we should do this in the C level for all classes?). It seems to me that Dataclasses could (and should) help us avoid the second issue regardless (should be trivial to look in the bases for preexisting slots, right?). My workaround for the first issue is to inherit from ABCs with empty slots, but you need cooperative multiple inheritance for this - and you need to track the expected attributes (easy if you use abstract properties, which slots provide for. Maybe not all users of Dataclasses are advanced enough to do this? So, maybe this is crazy (please don't call the nice men in white coats on me), came to me as I was responding, and definitely outside the box here, but perhaps we could make decorated dataclass be the abstract parent of the instantiated class? Thanks, Aaron Hall On Friday, December 8, 2017, 1:31:44 PM EST, Raymond Hettinger wrote: The way __slots__ works is that the type() metaclass automatically assigns member-objects to the class variables 'x' and 'y'. Member objects are descriptors that do the actual lookup. So, I don't think the language limitation can be "fixed". Essentially, we're wanting to use the class variables 'x' and 'y' to hold both member objects and a default value. I recommend that you follow the path taken by attrs and return a new class. Otherwise, we're leaving users with a devil's choice. You can have default values or you can have slots, but you can't have both. The slots are pretty important. With slots, a three attribute instance is only 64 bytes. Without slots, it is 296 bytes. I'm hoping the typing experts will chime in here. The question is straight-forward. Where should we look for the signature and docstring for constructing instances? Should they be attached to the class, to __init__(), or to __new__() when it used. It would be nice to have an official position on that before, it gets set in stone through arbitrary choices made by pycharm, pydoc, mypy, typing.NamedTuple, and dataclasses.dataclass. Raymond ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/aaronchall%40yahoo.com ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Unexpected bytecode difference
As a general rule, you should not expect the bytecode to be the same
between different versions of CPython, including minor version changes. For
example, the instructions for dictionary literals are different in 3.4,
3.5, and 3.6.
On Fri, Jan 19, 2018 at 6:54 PM, Victor Stinner
wrote:
> Python bytecode format changed deeply in Python 3.6. It now uses
> regular units of 2 bytes, instead of 1 or 3 bytes depending if the
> instruction has an argument.
>
> See for example https://bugs.python.org/issue26647 "wordcode".
>
> But CALL_FUNCTION bytecode also evolved.
>
> Victor
>
> 2018-01-20 0:46 GMT+01:00 Alexander Belopolsky <
> alexander.belopol...@gmail.com>:
> > I have encountered the following difference between Python 3 and 2:
> >
> > (py3)
> >>>> compile('xxx', '<>', 'eval').co_code
> > b'e\x00S\x00'
> >
> > (py2)
> >>>> compile('xxx', '<>', 'eval').co_code
> > 'e\x00\x00S'
> >
> > Note that 'S' (the code for RETURN_VALUE) and a zero byte are swapped
> > in Python 2 compared to Python 3. Is this change documented
> > somewhere?
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> victor.stinner%40gmail.com
> ___________________
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> joe%40quantopian.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] CLion IDE
On 01/24, Steve Holden wrote: I've just start using CLion from JetBrains, and I wondered if anyone on the list is using this product in CPython development. Links to any guidance would be useful. regards Steve Holden Hi Steve, I tried to use it for CPython, but it uses CMake and not the autotools. I have found a this repo https://github.com/python-cmake-buildsystem/python-cmake-buildsystem but not yet tested Stephane -- Stéphane Wirtel - http://wirtel.be - @matrixise ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] inconsistency in annotated assigned targets
Currently there are many ways to introduce variables in Python; however, only a few allow annotations. I was working on a toy language and chose to base my syntax on Python's when I noticed that I could not annotate a loop iteration variable. For example: for x: int in range(5): ... This led me to search for other places where new variables are introduced and I noticed that the `as` target of a context manager cannot have an annotation. In the case of a context manager, it would probably need parenthesis to avoid ambiguity with a single-line with statement, for example: with ctx as (variable: annotation): body Finally, you cannot annotate individual members of a destructuring assignment like: a: int, b: int, c: int = 1, 2, 3 Looking at the grammar, these appear to be `expr` or `exprlist` targets. One change may be to allow arbitrary expressions to have an annotation . This would be a small change to the grammar but would potentially have a large effect on the language or static analysis tools. I am posting on the mailing list to see if this is a real problem, and if so, is it worth investing any time to address it. I would be happy to attempt to fix this, but I don't want to start if people don't want the change. Also, I apologize if this should have gone to python-idea; this feels somewhere between a bug report and implementation question more than a new feature so I wasn't sure which list would be more appropriate. _______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] inconsistency in annotated assigned targets
Thank you for the clarification! I should have looked through the PEPs first. On Thu, Jan 25, 2018 at 10:14 PM, Guido van Rossum wrote: > PEP 526 has this in the "Rejected/Postponed Proposals" section: > > - **Allow annotations in** ``with`` **and** ``for`` **statement:** > This was rejected because in ``for`` it would make it hard to spot the > actual > iterable, and in ``with`` it would confuse the CPython's LL(1) parser. > > > On Thu, Jan 25, 2018 at 3:17 PM, Jelle Zijlstra > wrote: > >> >> >> 2018-01-25 15:00 GMT-08:00 Joe Jevnik via Python-Dev < >> python-dev@python.org>: >> >>> Currently there are many ways to introduce variables in Python; however, >>> only a few allow annotations. I was working on a toy language and chose to >>> base my syntax on Python's when I noticed that I could not annotate a loop >>> iteration variable. For example: >>> >>> for x: int in range(5): >>> ... >>> >>> This led me to search for other places where new variables are >>> introduced and I noticed that the `as` target of a context manager cannot >>> have an annotation. In the case of a context manager, it would probably >>> need parenthesis to avoid ambiguity with a single-line with statement, for >>> example: >>> >>> with ctx as (variable: annotation): body >>> >>> Finally, you cannot annotate individual members of a destructuring >>> assignment like: >>> >>> a: int, b: int, c: int = 1, 2, 3 >>> >>> Looking at the grammar, these appear to be `expr` or `exprlist` targets. >>> One change may be to allow arbitrary expressions to have an annotation . >>> This would be a small change to the grammar but would potentially have a >>> large effect on the language or static analysis tools. >>> >>> I am posting on the mailing list to see if this is a real problem, and >>> if so, is it worth investing any time to address it. I would be happy to >>> attempt to fix this, but I don't want to start if people don't want the >>> change. Also, I apologize if this should have gone to python-idea; this >>> feels somewhere between a bug report and implementation question more than >>> a new feature so I wasn't sure which list would be more appropriate. >>> >> I have written a fair amount of code with variable annotations, and I >> don't remember ever wanting to add annotations in any of the three contexts >> you mention. In practice, variable annotations are usually needed for >> class/instance variables and for variables whose type the type checker >> can't infer. The types of loop iteration variables and context manager >> assignment targets can almost always be inferred trivially. >> >> >>> >>> ___________ >>> Python-Dev mailing list >>> Python-Dev@python.org >>> https://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: https://mail.python.org/mailma >>> n/options/python-dev/jelle.zijlstra%40gmail.com >>> >>> >> >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% >> 40python.org >> >> > > > -- > --Guido van Rossum (python.org/~guido) > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The `for y in [x]` idiom in comprehensions
On 22/02/2018 19:04, Serhiy Storchaka wrote: Yet one discussion about reusing common subexpressions in comprehensions took place last week on the Python-ideas maillist (see topic "Temporary variables in comprehensions" [1]). The problem is that in comprehension like `[f(x) + g(f(x)) for x in range(10)]` the subexpression `f(x)` is evaluated twice. In normal loop you can introduce a temporary variable for `f(x)`. The OP wanted to add a special syntax for introducing temporary variables in comprehensions. This idea already was discussed multiple times in the past. There are several ways of resolving this problem with existing syntax. [snip] Stephan Houben proposed an idiom which looks similar to new hypothetic syntax: result = [y + g(y) for x in range(10) for y in [f(x)]] `for y in [expr]` in a comprehension means just assigning expr to y. I never seen this idiom before, but it can be a good replacement for a hypothetic syntax for assignment in comprehensions. It changes the original comprehension less than other approaches, just adds yet one element in a sequence of for-s and if-s. I think that after using it more widely it will become pretty idiomatic. I have created a patch that optimizes this idiom, making it as fast as a normal assignment. [2] Yury suggested to ask Guido on the mailing list if he agrees that this language patten is worth optimizing/promoting. Here's a thought: allow the syntax for VAR = EXPR to define a for-loop that is executed exactly once (both inside and outside comprehensions), i.e. pretty much a synonym for for VAR in [ EXPR ] for VAR in ( EXPR , ) especially if Serhiy's optimisation means that the list/tuple is not actually constructed in the latter. Pros: (1) Stephan Houben's example could be written as result = [y + g(y) for x in range(10) for y = f(x)] which I find more readable. (2) Code such as for i in xrange(10): could be changed on the fly to: for i = 1: I see this as especially useful in debugging, where you want to limit the program execution to a known problematic bit. But it some contexts it could be good style. (3) Preserves the compatibility between a list comprehension and its "expansion" into for-loops. (4) Backward compatible, since it is currently illegal syntax (5) No extra keyword needed (6) It goes some way towards providing the functionality of with VAR as EXPR that has been discussed multiple times. Best wishes Rob Cliffe _______________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The `for y in [x]` idiom in comprehensions
On 26/02/2018 19:08, Guido van Rossum wrote: I would like to remind all wannabe language designers that grammar design is not just solving puzzles. It's also about keeping the overall feel of the language readable. I'm getting the idea that none of the proposals discussed so far (whether new syntax or clever use of existing syntax) satisfy that constraint. Sometimes a for-loop is just better. I don't know if you intended these remarks to include my proposal (to allow "for VAR = EXPR"), as your message was posted only 27 minutes after mine. With respect, I honestly feel that this is a relatively small change that makes the language *more* readable. Feel free, one and all, to tell me why I'm wrong. Best wishes, Rob Cliffe ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The `for y in [x]` idiom in comprehensions
On Mon, Feb 26, 2018 at 7:51 PM, Guido van Rossum wrote: .. > The reason is that for people who are not Python experts there's no obvious > reason why `for VAR = EXPR` should mean one thing and `for VAR in EXPR` > should mean another. This would be particularly surprising for people exposed to Julia where these two forms are equivalent: julia> for x = [1,2] println(x); end 1 2 julia> for x in [1,2] println(x); end 1 2 _______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Git hub : CLA Not Signed label
All, I submitted two Pull Requests last Sunday, only a few hours after I signed the CLA. I understand why the 'Knights who say ni' marked the Pull request as 'CLA Not Signed' Label at the time I submitted the Pull requests, but I was wondering when the Labels get reset. How often (if at all) does the bot look at old pull requests ? Thanks for any help you can give, I am sorry if the question sounds basic. _______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Do we have vlookup function
This mailing list is for the development of CPython, not for the end user, please could you move your question on python-l...@python.org ? Thank you, Le 22/03/18 à 07:27, Rohit Adhikari a écrit : > Do we have vlookup function which can be used in dataframe same as used > in excel. > Can you please provide the pointer for the same? > > Thanks!!! > > > ___________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/stephane%40wirtel.be > ___________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Timeline for Pull request reviews in 3.8
Can anyone enlighten me on what the expected time-line is for reviewing pull requests made on 3.8. I made a few simple fixes in Early March - and I understand everyone is busy. What is the time line and cut off dates for backports to 3.7 and 3.6. I also made a documentation change (based on a open bug report) into 2.7, and I am keen to understand the planned time-line for those too. -- Anthony Flury email : *anthony.fl...@btinternet.com* Twitter : *@TonyFlury <https://twitter.com/TonyFlury/>* ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Timeline for Pull request reviews in 3.8
All, The three pull requests are : Python 2.7 - doc string fix : https://github.com/python/cpython/pull/6015 Python 3.8 - documentation fix : https://github.com/python/cpython/pull/5982 Python 3.8 - Small bug fix on unittest.mock.mock_open : https://github.com/python/cpython/pull/5974 The Py2.7 change does not need to be rolled forward to Python3 documentation The two Py3.8 fixes could/should/can ? be backported to earlier versions These are all trivial with no conflicts with their target branch (or at least there wasn't when I made the requests). -- Anthony Flury email : *anthony.fl...@btinternet.com* Twitter : *@TonyFlury <https://twitter.com/TonyFlury/>* _______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 572: Assignment Expressions
On 17/04/2018 15:01, Chris Angelico wrote: On Tue, Apr 17, 2018 at 10:17 PM, Nick Coghlan wrote: Style guide recommendations === As this adds another way to spell some of the same effects as can already be done, it is worth noting a few broad recommendations. These could be included in PEP 8 and/or other style guides. 1. If either assignment statements or assignment expressions can be used, prefer statements; they are a clear declaration of intent. 2. If using assignment expressions would lead to ambiguity about execution order, restructure it to use statements instead. 3. Chaining multiple assignment expressions should generally be avoided. More than one assignment per expression can detract from readability. Given the many different uses for ":" identified on python-ideas, I'm inclined to suggest making these proposed style guidelines more prescriptive (at least initially) by either: 1. Listing out specific approved unambiguous use cases (i.e. if statement conditions, while loop conditions, list comprehensions, generation expressions) 2. Making the 3rd admonition more general by advising against using ":" for more than one purpose in the same expression (i.e. don't combine assignment expressions with slicing syntax, lambda expressions, function headers, variable annotations, dict or set displays, dict or set comprehensions) I'm actually dubious about the third point as it stands. I'm more than dubious - I disagree with Nick on this point. It is already possible to have multiple uses of ":" in an expression; surely we wouldn't advise that such existing code should be changed, in cases where it is arises naturally and is genuinely useful. Best wishes Rob Cliffe ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 572: Assignment Expressions
I am entirely new to this list, but if I can I would like share my comments : * I do think this proposal := has merit in my opinion; it does make some code more readable. * I think readability is only improved if : * target is restricted to a simple name - I don't see a benefit in more complex targets * chaining is not allowed - I think the construct : while (line := input.read_row()) is not None: process_line(line) Is readable, but : while (current_line := line := input.read_row()) is not None: line = process_line(line) is not obvious - and certainly isn't any more obvious than : while (line := input.read_row()) is not None: current_line = line line = process_line(line) * The current expectations of how comprehensions work should also be honored; I don't claim to have fully followed all of the discussions around this, but it seems to me that comprehensions work in a particular way because of a concerted effect (especially in Python 3) to make them that way. They are self contained and don't leak values in their containing scope. Similarly I think that setting variables within a comprehension is just for the benefit of readable code within the comprehension - i.e. : stuff = [[y, x/y] for x in range(5) for y in [f(x)]] can become : stuff = [[y := f(x), x/y] for x in range(5)] So - overall from me a conditional +1 - conditions as above; if they are not possible then -1 from me. -- Anthony Flury email : *anthony.fl...@btinternet.com* Twitter : *@TonyFlury <https://twitter.com/TonyFlury/>* ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 572: Assignment Expressions
On 21/04/18 08:46, Chris Angelico wrote: doubled_items = [x for x in (items := get_items()) if x * 2 in items] This will leak 'items' into the surrounding scope (but not 'x'). At the risk of stating the obvious - wasn't there work in Python 3 to prevent leakage from comprehensions ? [x for x in x if x] # This works [x for y in x if x := y] # UnboundLocalError The standard library example given earlier notwithstanding, I can see no benefit in using the same name as the iterator and the loop target name. To be honest I have trouble parsing that first version, and keeping track of which x is which (especially which x is being used in the conditional clause) : surely this would be better : [x_item for x_item in x if x_item] Your 2nd example makes no sense to me as to the intention of the code - the re-use of the name x is confusing at best. -- Anthony Flury email : *anthony.fl...@btinternet.com* Twitter : *@TonyFlury <https://twitter.com/TonyFlury/>* ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 572: Assignment Expressions
On 21/04/18 11:18, Chris Angelico wrote: But you haven't answered anything about what "readable" means. Does it mean "if I look at this code, I can predict what dis.dis() would output"? Or does it mean "this code clearly expresses an algorithm and the programmer's intent"? Frequently I hear people complain that something is unreadable because it fails the former check. I'm much more interested in the latter check. For instance, this line of code expresses the concept "generate the squares of odd numbers": [x*x for x in range(100) if x % 2] But it doesn't clearly express the disassembly. Is that a problem? Are list comprehensions a bad feature for that reason? I don't think so. ChrisA For what it worth - readability for me is all about understanding the intent. I don't care (most of the time) about how the particular code construct is actually implemented. When I am maintaining code (or trying to) I need to understand what the developer intended (or in the case of a bug, the gap between the outcome and the intention). One of the challenges about readability is it partially depends on skill level - for a beginner the comprehension may well be baffling where as someone with more skills would understand it - almost intuitively; as an example: I have been using Python for 7 years - and comprehensions with more than one for loop still are not intuitive for me, I can't read them without an amount of deep thought about how the loops work together. -- -- Anthony Flury email : *anthony.fl...@btinternet.com* Twitter : *@TonyFlury <https://twitter.com/TonyFlury/>* ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] assignment expressions: an alternative proposal
On 24/04/18 14:50, Yury Selivanov wrote: On Tue, Apr 24, 2018 at 9:46 AM, Nick Coghlan wrote: On 24 April 2018 at 23:38, Yury Selivanov wrote: I propose to use the following syntax for assignment expressions: ( NAME = expr ) I know that it was proposed before and this idea was rejected, because accidentally using '=' in place of '==' is a pain point in C/C++/JavaScript. That said, I believe we can still use this syntax as long as we impose the following three restrictions on it: 1. Only NAME token is allowed as a single target. 2. Parenthesis are required. 3. Most importantly: it is *not* allowed to mask names in the current local scope. While I agree this would be unambiguous to a computer, I think for most humans it would be experienced as a confusing set of arcane and arbitrary rules about what "=" means in Python. I respectfully disagree. There are no "arcane and confusing rules" about "=", it's rather simple: "=" is always an assignment. But it isn't - in your proposed syntax : * * = * is an assignment with no return value * *( = )* is an assignment with a returned value So now '=' is always an assignment, it is an assignment with extra semantics depending on surrounding syntax. As discussed previously by others on this exact proposals, you now have the issue of confusion when using keyword arguments : *my_func(a = b)* : clearly that is a call to `my_func' where argument a has the value of b, but if you want to do an assigment expression when calling the function you now have to do *my_func((a=b)) -* which frankly looks messy in my opinion; you get the same issue when you are wanting to do assignment expressions in tuples. Using a different operator for assignments which return values avoids the messy potentially multiple level brackets, and means that the semantics of an operator depends only on that operator and not on syntax elements before and after it. -- -- Anthony Flury email : *anthony.fl...@btinternet.com* Twitter : *@TonyFlury <https://twitter.com/TonyFlury/>* ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] assignment expressions: an alternative proposal
On 24/04/18 17:11, Yury Selivanov wrote: On Tue, Apr 24, 2018 at 12:03 PM, Ethan Furman wrote: [..] But I do write this: def wrapper(func, some_value): value_I_want = process(some_value) def wrapped(*args, **kwds): if value_I_want == 42: ... But this pattern is more rare than comparing local variables. That's the point I'm trying to use. Besides, to make it an assignment expression under my proposal you would need to use parens. Which makes it even less likely that you confuse '=' and '=='. Just because you wrap a set of character in parens doesn't mean that you wont potentially mistype what you should type inside the parens. The failure mode of in C : if (a = 3) do_something_with_a(a); Is Incredibly common even with very experienced developers - so much so that most linters flag it as a likely error, and I think gcc has an option to flag it as a warning - even though it is valid and very occasionally it is useful. Also many developers who come to Python from languages such as C will still place parens around conditionals - this means that a typo which will cause a Syntax Error in current versions, but would cause a potentially subtle bug under your implementation (unless you maintain the rule that you can't rebind currently bound names - which renders the whole idea useless in loops (as already discussed at length). I also still can't think of a single other Python construct where the semantics of an operator are explicitly modified by syntaxtic elements outside the operator. For mathematical operators, the surrounding parens modifies the grouping of the operators but not the semantics (* means *, it is just the operands which potentially change). You could argue that your proposal overloads the semantics of the parens (similar to how braces are overloaded to implement dictionaries and set literals), but I don't think that overloading the semantics of parens is good idea. If Python is going to do assignment expressions we shouldn't overload parens in my opinion - we should have a separate operator - doing this avoids needing to exclude rebinding, and makes such expressions considerably more useful. -- Anthony Flury email : *anthony.fl...@btinternet.com* Twitter : *@TonyFlury <https://twitter.com/TonyFlury/>* ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Review of Pull Request 5974 please
All, Can someone please review Pull Request 5974 <https://github.com/python/cpython/pull/5974> on Python3.8 - the Pull request was submitted on 4th March - this pull request is associated with bpo-32933 <https://bugs.python.org/issue32933> To summarize the point of this pull request: It fixes a bug of omission within mock_open <https://docs.python.org/3/library/unittest.mock.html?highlight=mock_open#unittest.mock.mock_open> (part of unittest.mock) The functionality of mock_open enables the test code to mock a file being opened with some data which can be read. Importantly, mock_open has a read_data attrribute which can be used to specify the data to read from the file. The mocked file which is opened correctly supports file.read(), file.readlines(), file.readline(). These all make use of the read_data as expected, and the mocked file also supports being opened as a context manager. But the mock_open file does not support iteration - so pythonic code which uses a for loop to iterate around the file content will only ever appear to iterate around an empty file, regardless of the read_data attribute when the mock_open is created So non-pythonic methods to iterate around the file contents - such as this : data = opened_file.readlines() for line in data: process_line(line) and this : line = opened_file.readline() while line: process_line(line) line = opened_file.readline() Can both be tested with the mocked file containing simulated data (using the read_data attribute) as expected. But this code (which by any standard is the 'correct' way to iterate around the file content of a text file): for line in opened_file: process_line(line) Will only ever appear to iterate around an empty file when tested using mock_open. I would like this to be reviewed so it can be back-ported into Python3.7 and 3.6 if at all possible. I know that the bug has existed since the original version of mock_open, but it does seem strange that code under test which uses a pythonic code structure can't be fully tested fully using the standard library. -- Anthony Flury email : *anthony.fl...@btinternet.com* Twitter : *@TonyFlury <https://twitter.com/TonyFlury/>* _______________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] bpo-33257: seeking advice & approval on the course of action
The bottom line is: Tkinter is currently broken -- as in, it's not thread-safe (in both Py2 and Py3) despite being designed and advertizing itself as such. All the fix options require some redesign of either `_tkinter', or some of the core as well. So, I'd like to get some kind of core team's feedback and/or approval before pursuing any of them. The options are outlined in https://bugs.python.org/issue33257#msg316087 . If anyone of you is in Moscow, we can meet up and discuss this in a more time-efficient manner. -- Regards, Ivan _______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Drop/deprecate Tkinter?
As https://bugs.python.org/issue33257 and https://bugs.python.org/issue33316 showed, Tkinter is broken, for both Py2 and Py3, with both threaded and non-threaded Tcl, since 2002 at least, and no-one gives a damn. This seems to be a testament that very few people are actually interested in or are using it. If that's so, there's no use keeping it in the standard library -- if anything, because there's not enough incentive and/or resources to support it. And to avoid screwing people (=me) up when they have the foolishness to think they can rely on it in their projects -- nowhere in the docs it is said that the module is only partly functional. -- Regards, Ivan _______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Drop/deprecate Tkinter?
On 03.05.2018 1:01, Antoine Pitrou wrote:
On Wed, 2 May 2018 22:54:04 +0100
Paul Moore wrote:
On 2 May 2018 at 22:37, Antoine Pitrou wrote:
To elaborate a bit: the OP, while angry, produced both a detailed
analysis *and* a PR. It's normal to be angry when an advertised
feature doesn't work and it makes you lose hours of work (or, even,
forces you to a wholesale redesign). Producing a detailed analysis and a
PR is more than most people will ever do.
His *other* email seems reasonable, and warrants a response, yes. But
are we to take the suggestion made here (to drop tkinter) seriously,
based on the fact that there's a (rare - at least it appears that the
many IDLE users haven't hit it yet) race condition that causes a crash
in Python 2.7? (It appears that the problem doesn't happen in the
python.org 3.x builds, if I understand the description of the issue).
In 3.x, Tkinter+threads is broken too, albeit in a different way -- see
https://bugs.python.org/issue33412 (this should've been the 2nd link in
the initial message, sorry for the mix-up).
The 2.x bug also shows in 3.x if it's linked with a nonthreaded version
of Tcl (dunno how rare that is, but the code still supports this setup).
I and others actually suggested it seriously in the past. Now,
admittedly, at least IDLE seems better maintained than it used to
be -- not sure about Tkinter itself.
Nor do I think the tone of his message here is acceptable - regardless
of how annoyed he is, posting insults ("no-one gives a damn") about
volunteer contributors in a public mailing list isn't reasonable or
constructive. Call that "playing speech police" if you want, but I
think that being offended or annoyed and saying so is perfectly
reasonable.
Will all due respect, it's sometimes unpredictable what kind of wording
Anglo-Saxons will take as an insult, as there's lot of obsequiosity
there that doesn't exist in other cultures. To me, "not give a damn"
reads like a familiar version of "not care about something", but
apparently it can be offensive.
Confirm, never meant this as an insult.
I had to use emotional language to drive the point home that it's not
some nitpick, it really causes people serious trouble (I lost a source
of income, for the record).
Without the emotional impact, my message could easily be ignored as some
noise not worth attention. This time, it's just too damn important to
allow this possibility.
The module being abandoned and unused is truly the only explanation I
could think of when seeing that glaring bugs have stayed unfixed for 15
years (an infinity in IT), in an actively developed and highly used
software.
This may be flattering for my ego, but if the module really is in any
production use to speak of, then in all these years, with all this
humongous user base, someone, somewhere in the world, at some point,
should have looked into this. I don't even program in C professionally,
yet was able to diagnose it and make a PR!
---
I'll make a PR with the doc warning as Guido suggested unless there are
any better ideas.
Meanwhile, I'd really appreciate any response to my other message -- it
is about actually fixing the issue, and I do need feedback to be able to
proceed.
No need to delve all the way in and give an official authorization or
something. I'm only looking for an opinion poll on which redesign option
(if any) looks like the most reasonable way to proceed and/or in line
with the big picture (the last one -- to provide a unifying vision -- is
_the_ job of a BDFL IIRC).
Regards
Antoine.
___________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru
--
Regards,
Ivan
___________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Drop/deprecate Tkinter?
On 03.05.2018 20:11, Ryan Gonzalez wrote: On May 3, 2018 11:56:24 AM MRAB wrote: On 2018-05-03 13:24, Steve Holden wrote: On Thu, May 3, 2018 at 12:12 AM, Ivan Pozdeev via Python-Dev mailto:python-dev@python.org>> wrote: On 03.05.2018 1:01, Antoine Pitrou wrote: On Wed, 2 May 2018 22:54:04 +0100 Paul Moore <mailto:p.f.mo...@gmail.com>> wrote: On 2 May 2018 at 22:37, Antoine Pitrou mailto:solip...@pitrou.net>> wrote: To elaborate a bit: the OP, while angry, produced both a detailed analysis *and* a PR. It's normal to be angry when an advertised feature doesn't work and it makes you lose hours of work (or, even, forces you to a wholesale redesign). Producing a detailed analysis and a PR is more than most people will ever do. His *other* email seems reasonable, and warrants a response, yes. But are we to take the suggestion made here (to drop tkinter) seriously, based on the fact that there's a (rare - at least it appears that the many IDLE users haven't hit it yet) race condition that causes a crash in Python 2.7? (It appears that the problem doesn't happen in the python.org <http://python.org> 3.x builds, if I understand the description of the issue). In 3.x, Tkinter+threads is broken too, albeit in a different way -- see https://bugs.python.org/issue33412 <https://bugs.python.org/issue33412> (this should've been the 2nd link in the initial message, sorry for the mix-up). The observation in that issue that tkinter and threads should be handled in specific ways is certainly a given for old hands, who have long put the GUI code in one thread with one or more concurrent worker threads typically communicating through queues. But I haven't built anything like that recently, so I couldn't say how helpful the current documenation might be. Interacting with the GUI only in the main thread is something that I've had to do in other languages (it is/was the recommended practice), so I naturally do the same with Python and tkinter. It's also easier to reason about because you don't get elements of the GUI changing unexpectedly. To add to this, most GUI frameworks disallow modifications outside the main thread altogether. IIRC both GTK+ and Qt require this, or else it's undefined altogether. You still need some facility (*cough*SendMessage*cough*) to send update commands to the GUI (the model->view link in MVC, presenter->view in MVP). Who and how specifically carries out these commands is unimportant, an implementation detail. Every GUI has an event/message queue exactly for that, that other threads can sent work requests into: https://doc.qt.io/qt-5.10/qcoreapplication.html#postEvent , https://developer.gnome.org/gdk3/stable/gdk3-Threads.html#gdk3-Threads.description , https://en.wikipedia.org/wiki/Event_dispatching_thread#Submitting_user_code_to_the_EDT , the aforementioned WinAPI's SendMessage() and PostMessage() just to name a few. Tcl/Tk, being arguably the oldest usable GUI toolkit in existence, has an event queue likewise but doesn't provide a complete event loop implementation, only the building blocks for it. Tkinter fills that gap with its `tk.mainloop()`. It fails to provide a working means to send work into it though. Having to use a second, duplicating event queue and poll it (=busy loop) instead is an obvious crutch. [snip] ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com -- Ryan (ライアン) Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone else https://refi64.com/ _______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru -- Regards, Ivan _______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dealing with tone in an email
On 03.05.2018 21:31, Brett Cannon wrote:
On Thu, 3 May 2018 at 01:27 Paul Moore <mailto:p.f.mo...@gmail.com>> wrote:
On 3 May 2018 at 03:26, Steven D'Aprano mailto:st...@pearwood.info>> wrote:
>> Will all due respect, it's sometimes unpredictable what kind of
wording
>> Anglo-Saxons will take as an insult, as there's lot of obsequiosity
>> there that doesn't exist in other cultures. To me, "not give a
damn"
>> reads like a familiar version of "not care about something", but
>> apparently it can be offensive.
>
> I'm Anglo-Saxon[1], and honestly I believe that it is
thin-skinned to
> the point of ludicrousness to say that "no-one gives a damn" is an
> insult. This isn't 1939 when Clark Gable's famous line "Frankly
my dear,
> I don't give a damn" was considered shocking. Its 2018 and to
not give a
> damn is a more forceful way of saying that people don't care,
that they
> are indifferent.
Sigh. That's not what I was saying at all. I was trying to point out
that Antoine's claim that people should ignore the rhetoric and that
complaining about the attitude was unreasonable, was in itself unfair.
People have a right to point out that a mail like the OP's was badly
worded.
> With respect to Paul, I literally cannot imagine why he thinks that
> *anyone*, not even the tkinter maintainers or developers themselves,
> ought to feel *offended* by Ivan's words.
Personally, they didn't offend me. I don't pretend to know how others
might take them. But they *did* annoy me. I'm frankly sick of people
(not on this list) complaining that people who work on projects in
their own time, free of charge, "don't care enough" or "are ignoring
my requirement". We all do it, to an extent, and it's natural to get
frustrated, but the onus is on the person asking for help to be polite
and fair. And maybe this response was the one where I finally let that
frustration show through. I may read less email for a week or two,
just to get a break.
I had the same response as Paul: annoyed. And while Ivan thought he
was using "emotional language to drive the point home that it's not
some nitpick", it actually had the reverse effect on me and caused me
not to care because I don't need to invite annoyance into my life when
putting in my personal time into something.
No one is saying people can't be upset and if you are ever upset
there's something wrong; we're human beings after all. But those of us
speaking up about the tone are saying that you can also wait until
you're not so upset to write an email. This was never going to be
resolved in an hour, so waiting an hour until you're in a better place
to write an email that wasn't quite so inflammatory seems like a
reasonable thing to ask.
Let me express things right from the horse's mouth.
The sole purpose of the tone was to not let the mesage be flat-out ignored.
I had my neutral-toned, to-the-point messages to mailing lists flat-out
ignored one too many times for reasons that I can only guess about.
This time, the situation was too important to let that happen.
Whatever anyone may think of this, it worked. I got my message through,
and got the feedback on the topic that I needed to proceed in resolving
the problem that caused it.
I seriously doubt I could achieve that with a neutral-toned message just
stating the facts: dry facts would not show ppl how this could be
important ("ah, just another n00b struggling with Tkinter basics" or
something).
_______________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru
--
Regards,
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20
Reading this sub-thread, it struck me that a good way to make PEP 562 more likely to be accepted is to launch an over-the-top attack on it. Then more moderate people - who were/are not necessarily in favour of the PEP - feel pressurised into defending it. Hah! Watch this space for my vicious, vitriolic, withering attack on PEP 463 (Exception-catching expressions)! :-) Best wishes Rob Cliffe ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dealing with tone in an email
fact that they chose to work with other tickers over mine shows that
they considered those more important. So it does matter if they
underestimate a topic.)
That's why I had to resort to shock value. First, it would guarantee
that my message won't fall on deaf ears this time as well. Second, I had
to express, somehow, that there indeed was a systemic disaster, not just
your average newbie conundrum, in graphic details to shock the team and
disrupt their current way of thinking about the Tkinter case.
Putting the question point-blank: "drop/deprecate" -- also helped the
shock value, and would also force the team to reassess if they really
have the will or resources to bring the module up to Python standards or
at least prevent any more such damage.
I also did require the team's feedback on this question to assess the
perspectives for results of my efforts -- thus if they're worth the time
-- as explained in
https://mail.python.org/pipermail/python-dev/2018-May/153330.html .
By no means I consider this a standard way of action. This is only the
second such case in my life. The previous one was when I created a
translation project for the GPL and found that I cannot legally do that
the way that was required to make it work due to overly defensive FSF's
terms.
This is the justification. It's up to you to judge how sound it is, but
you need to know what to judge, at least. I wasn't happy to have to
resort to this, but found no better way that would be guaranteed to work.
Now that the social issues are out of the way and I got the required
feedback to continue, I can finally concentrate on the patches, with
confidence that my efforts won't go to waste.
Intemperate and impolite it certainly was, as well as full of factual
inaccuracies, but to call it "close to abusive" is a hostile over-
reaction. We ought to be kinder than that. Our response to Ivan
has been
more hostile, and less open and respectful, than his email that
triggered the response.
Brett is right to say people can afford to wait a little while before
firing off an angry email. But the same applies to us: we too can
afford
to wait a little while before raising the threat of the CoC over a
minor
social faux pas. This community isn't so fragile that we have to jump
down the throat of a newcomer lest the community immediately
collapses
into Call Of Duty gamer culture.
--
Steve
___
Python-Dev mailing list
Python-Dev@python.org <mailto:Python-Dev@python.org>
https://mail.python.org/mailman/listinfo/python-dev
<https://mail.python.org/mailman/listinfo/python-dev>
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org
<https://mail.python.org/mailman/options/python-dev/guido%40python.org>
--
--Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru
--
Regards,
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Windows 10 build agent failures
For me, Tcl/Tk failed to build with SDK 10.0.16299.0 , I had to expicitly fall back to 10.0.15063.0 ( https://stackoverflow.com/questions/48559337/error-when-building-tcltk-in-visual-studio-2017 ). May be related if VS was (auto)updated on the builders.|| On 06.05.2018 11:05, Paul Goins wrote: || Hi, Just kind of "looking around" at stuff I can help with, and I noticed a few days ago that Windows 10 AMD64 builds of Python 3.6/3.7/3.x are generally failing. It seems like the failures started April 16th around 1am per BuildBot and went from consistently passing to consistently failing. The errors appear to be timeouts; the test runs are going over 15 minutes. I can run the tests locally and things seem fine. The one thing which seems to have changed is, under the "10 slowest tests" header, test_io has shot up in terms of time to complete: 3.6: 2 min 13 sec <http://buildbot.python.org/all/#/builders/31/builds/273/steps/3/logs/stdio> to 9 min 25 sec <http://buildbot.python.org/all/#/builders/31/builds/297/steps/3/logs/stdio> 3.7: 2 min 10 sec <http://buildbot.python.org/all/#/builders/121/builds/182/steps/3/logs/stdio> to 9 min 26 sec <http://buildbot.python.org/all/#/builders/121/builds/215/steps/3/logs/stdio> 3.x: 3 min 39 sec <http://buildbot.python.org/all/#/builders/3/builds/794/steps/3/logs/stdio> to 9 min 17 sec <http://buildbot.python.org/all/#/builders/3/builds/844/steps/3/logs/stdio> Locally this test suite runs in around 36 seconds. I see no real change between running one of the last "good" changesets versus the current head of master. I'm suspecting an issue on the build agent perhaps? Thoughts? Best Regards, Paul Goins _______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru -- Regards, Ivan _______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] bpo-33038: Fix gzip.GzipFile for file objects with a non-string name attribute. (GH-6095)
From: Serhiy Storchaka
To: python-check...@python.org
Sent: Wednesday, 9 May 2018, 10:14
Subject: [Python-checkins] bpo-33038: Fix gzip.GzipFile for file objects with
a non-string name attribute. (GH-6095)
https://github.com/python/cpython/commit/afe5f633e49e0e873d42088ae56819609c803ba0
commit: afe5f633e49e0e873d42088ae56819609c803ba0
branch: 2.7
author: Bo Bayles
committer: Serhiy Storchaka
date: 2018-05-09T13:14:40+03:00
summary:
bpo-33038: Fix gzip.GzipFile for file objects with a non-string name attribute.
(GH-6095)
files:
A Misc/NEWS.d/next/Library/2018-03-10-20-14-36.bpo-33038.yA6CP5.rst
M Lib/gzip.py
M Lib/test/test_gzip.py
M Misc/ACKS
diff --git a/Lib/gzip.py b/Lib/gzip.py
index 07c6db493b0b..76ace394f482 100644
--- a/Lib/gzip.py
+++ b/Lib/gzip.py
@@ -95,9 +95,8 @@ def __init__(self, filename=None, mode=None,
if filename is None:
# Issue #13781: os.fdopen() creates a fileobj with a bogus name
# attribute. Avoid saving this in the gzip header's filename field.
- if hasattr(fileobj, 'name') and fileobj.name != '':
- filename = fileobj.name
- else:
+ filename = getattr(fileobj, 'name', '')
+ if not isinstance(filename, basestring) or filename == '':
filename = ''
if mode is None:
if hasattr(fileobj, 'mode'): mode = fileobj.mode
diff --git a/Lib/test/test_gzip.py b/Lib/test/test_gzip.py
index 902d93fe043f..cdb1af5c3d13 100644
--- a/Lib/test/test_gzip.py
+++ b/Lib/test/test_gzip.py
@@ -6,6 +6,7 @@
import os
import io
import struct
+import tempfile
gzip = test_support.import_module('gzip')
data1 = """ int length=DEFAULTALLOC, err = Z_OK;
@@ -331,6 +332,12 @@ def test_fileobj_from_fdopen(self):
with gzip.GzipFile(fileobj=f, mode="w") as g:
self.assertEqual(g.name, "")
+ def test_fileobj_from_io_open(self):
+ fd = os.open(self.filename, os.O_WRONLY | os.O_CREAT)
+ with io.open(fd, "wb") as f:
+ with gzip.GzipFile(fileobj=f, mode="w") as g:
+ self.assertEqual(g.name, "")
+
def test_fileobj_mode(self):
gzip.GzipFile(self.filename, "wb").close()
with open(self.filename, "r+b") as f:
@@ -359,6 +366,14 @@ def test_read_with_extra(self):
with gzip.GzipFile(fileobj=io.BytesIO(gzdata)) as f:
self.assertEqual(f.read(), b'Test')
+ def test_fileobj_without_name(self):
+ # Issue #33038: GzipFile should not assume that file objects that have
+ # a .name attribute use a non-None value.
+ with tempfile.SpooledTemporaryFile() as f:
+ with gzip.GzipFile(fileobj=f, mode='wb') as archive:
+ archive.write(b'data')
+ self.assertEqual(archive.name, '')
+
def test_main(verbose=None):
test_support.run_unittest(TestGzip)
diff --git a/Misc/ACKS b/Misc/ACKS
index 580b0c5bf76d..458f31e6a6b7 100644
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -94,6 +94,7 @@ Michael R Bax
Anthony Baxter
Mike Bayer
Samuel L. Bayer
+Bo Bayles
Donald Beaudry
David Beazley
Carlo Beccarini
diff --git a/Misc/NEWS.d/next/Library/2018-03-10-20-14-36.bpo-33038.yA6CP5.rst
b/Misc/NEWS.d/next/Library/2018-03-10-20-14-36.bpo-33038.yA6CP5.rst
new file mode 100644
index ..22d394b85ab7
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2018-03-10-20-14-36.bpo-33038.yA6CP5.rst
@@ -0,0 +1,2 @@
+gzip.GzipFile no longer produces an AttributeError exception when used with
+a file object with a non-string name attribute. Patch by Bo Bayles.
_______________
Python-checkins mailing list
python-check...@python.org
https://mail.python.org/mailman/listinfo/python-checkins
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bpo-33257: seeking advice & approval on the course of action
On Wed, May 2, 2018 at 8:21 PM, Terry Reedy wrote: > On 5/2/2018 4:38 PM, Ivan Pozdeev via Python-Dev wrote: > >> The bottom line is: Tkinter is currently broken >> > > This is way over-stated. Many modules have bugs, somethings in features > more central to their main purpose. I'll suggest a re-statement: tkinter is not thread safe, and yet it is documented as being thread safe (or at least not documented as NOT being thread safe) This is either a bug(s) in the implementation or the docs. So what are the solutions? 1) fix the docs -- unless tkInter is made thread safe really soon, and fixes are back-ported, this seems like a no brainer -- at least temporarily. 2) fix the issues that make tkInter not thread safe -- apparently there is a thread safe tcl/tk, so it should be possible, though I have to say I'm really surprised that that's the case for an old C code base -- but great! The problem here is that we'll need qualified people to submit and review the code, and it really should get some extensive testing -- that's a lot of work. And it's going to take time, so see (1) above. Another issue: Many GUI toolkits are not thread safe (I have personal experience with wxPython), so it's not so bad to simply say "don't do that" for tkInter -- that is, don't make tkInter calls from more than one thread. However, wxPython (for example) makes this not-too-bad for multi-threaded programs by providing thread-safe ways to put events on the event queue -- whether with wx.PostEvent, or the utilities wx.CallAfter() and wx.CallLater(). This makes it pretty easy to keep the GUI in one thread while taking advantage of threads for long running tasks, etc. IIUC, tkinter does not have such an easy way to interact with the GUI from another thread -- so maybe adding that would be a good first step. I neither use tkinter, nor have the expertise to contribute this -- but clearly Ivan does -- and maybe others would want to help. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python startup time
On Fri, May 11, 2018 at 11:05 AM, Ryan Gonzalez wrote: > https://refi64.com/uprocd/ very cool -- but *nix only, of course :-( But it seems that there is a demand for this sort of thing, and a few major projects are rolling their own. So maybe it makes sense to put something into the standard library that everyone could contribute to and use. With regard to forking -- is there another way? I don't have the expertise to have any idea if this is possible, but: start up python capture the entire runtime image as a single binary blob. could that blob be simply loaded into memory and run? (hmm -- probably not -- memory addresses would be hard-coded then, yes?) or is memory virtualized enough these days? -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov _______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python startup time
On Mon, May 14, 2018 at 12:33 PM, INADA Naoki wrote: > It will broke hash randomization. > > See also: https://www.cvedetails.com/cve/CVE-2017-11499/ I'm not enough of a security expert to know how much that matters in this case, but I suppose one could do a bit of post-proccessing on the image to randomize the hashes? or is that just insane? Also -- I wasn't thinking it would be a pre-build binary blob that everyone used -- but one built on the fly on an individual system, maybe once per reboot, or once per shell instance even. So if you are running, e.g. hg a bunch of times in a shell, does it matter that the instances are all identical? -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov _______ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3
On Sat, May 12, 2018 at 8:14 AM, Skip Montanaro wrote: > > I have found 2to3 conversion to be remarkably easy and painless. > > > And the whole Unicode thing is much easier. > Another point here: between 3.0 and 3.6 (.5?) -- py3 grew a lot of minor features that made it easier to write py2/py3 compatible code. u"string", b'bytes %i' % something -- and when where the various __future__ imports made available? If these had been in place in 3.0, the whole process would have been easier :-( -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bpo-33257: seeking advice & approval on the course of action
On 14.05.2018 21:58, Terry Reedy wrote: On 5/14/2018 12:20 PM, Chris Barker via Python-Dev wrote: On Wed, May 2, 2018 at 8:21 PM, Terry Reedy <mailto:tjre...@udel.edu>> wrote: On 5/2/2018 4:38 PM, Ivan Pozdeev via Python-Dev wrote: The bottom line is: Tkinter is currently broken This is way over-stated. Many modules have bugs, somethings in features more central to their main purpose. I'll suggest a re-statement: tkinter is not thread safe, Still over-stated. If one uses tcl/tk compiled with thread support, tkinter *is* thread-safe. This is 'as far as I know' from running posted 'failing' examples (possible with bug fixes) with 3.5+ on Windows, which is installed with tcl/tk 8.6, which defaults to thread-safe. This means that you didn't (yet) read the letter that I attached to https://bugs.python.org/issue33479 . Reciting the relevant section: === The reality is that with neither flavor of Tcl is Tkinter completely thread-safe, but with threaded flavor, it's more so: * with nonthreaded Tcl, making concurrent Tcl calls leads to crashes due to incorrect management of the "Tcl lock" as per https://bugs.python.org/issue33257 * with threaded Tcl, the only issue that I found so far is that a few APIs must be called from the interpreter's thread (https://bugs.python.org/issue33412#msg316152; so far, I know `mainloop()` and `destroy()` to be this) -- while most can be called from anywhere. Whether the exceptions are justified is a matter of discussion (e.g. at first glance, `destroy()` can be fixed). === Tkinter was intended to also be thread-safe when using tcl/tk without thread support, which was the default for tcl/tk 8.5 and before. The posted examples can fail on 2.x on Windows, which comes with tcl/tk 8.5 or before. _tkinter.c has some different #ifdefs for the two situations. and yet it is documented as being thread safe True in https://docs.python.org/3/library/tk.html Unspecified in https://docs.python.org/3/library/tkinter.html This is either a bug(s) in the implementation or the docs. Both So what are the solutions? 1) fix the docs -- unless tkInter is made thread safe really soon, and fixes are back-ported, this seems like a no brainer -- at least temporarily. https://bugs.python.org/issue33479 'Document tkinter and threads' 2) fix the issues that make tkInter not thread safe with non-thread tcl/tk. https://bugs.python.org/issue33257 has a patch that might improve the situation for one type of call. Fixing everything might not be possible. AFAIK, there are currently no tests of thread safety. -- Regards, Ivan ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bpo-33257: seeking advice & approval on the course of action
On 14.05.2018 22:05, Ivan Pozdeev wrote: On 14.05.2018 21:58, Terry Reedy wrote: On 5/14/2018 12:20 PM, Chris Barker via Python-Dev wrote: On Wed, May 2, 2018 at 8:21 PM, Terry Reedy <mailto:tjre...@udel.edu>> wrote: On 5/2/2018 4:38 PM, Ivan Pozdeev via Python-Dev wrote: The bottom line is: Tkinter is currently broken This is way over-stated. Many modules have bugs, somethings in features more central to their main purpose. I'll suggest a re-statement: tkinter is not thread safe, Still over-stated. If one uses tcl/tk compiled with thread support, tkinter *is* thread-safe. This is 'as far as I know' from running posted 'failing' examples (possible with bug fixes) with 3.5+ on Windows, which is installed with tcl/tk 8.6, which defaults to thread-safe. This means that you didn't (yet) read the letter that I attached to https://bugs.python.org/issue33479 . Reciting the relevant section: === The reality is that with neither flavor of Tcl is Tkinter completely thread-safe, but with threaded flavor, it's more so: * with nonthreaded Tcl, making concurrent Tcl calls leads to crashes due to incorrect management of the "Tcl lock" as per https://bugs.python.org/issue33257 * with threaded Tcl, the only issue that I found so far is that a few APIs must be called from the interpreter's thread (https://bugs.python.org/issue33412#msg316152; so far, I know `mainloop()` and `destroy()` to be this) -- while most can be called from anywhere. Whether the exceptions are justified is a matter of discussion (e.g. at first glance, `destroy()` can be fixed). And another undocumented limitation for threaded Tcl: when calling anything from outside the interpreter thread, `mainloop()` must be running in the interpreter threads, or the call will either raise or hang (dunno any more details atm). === Tkinter was intended to also be thread-safe when using tcl/tk without thread support, which was the default for tcl/tk 8.5 and before. The posted examples can fail on 2.x on Windows, which comes with tcl/tk 8.5 or before. _tkinter.c has some different #ifdefs for the two situations. and yet it is documented as being thread safe True in https://docs.python.org/3/library/tk.html Unspecified in https://docs.python.org/3/library/tkinter.html This is either a bug(s) in the implementation or the docs. Both So what are the solutions? 1) fix the docs -- unless tkInter is made thread safe really soon, and fixes are back-ported, this seems like a no brainer -- at least temporarily. https://bugs.python.org/issue33479 'Document tkinter and threads' 2) fix the issues that make tkInter not thread safe with non-thread tcl/tk. https://bugs.python.org/issue33257 has a patch that might improve the situation for one type of call. Fixing everything might not be possible. AFAIK, there are currently no tests of thread safety. -- Regards, Ivan ___________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Making Tcl/Tk more suitable for embedding (was: [issue33479] Document tkinter and threads)
from different threads. Consider those a side-effect of present implementation, not a guarantee. Future core changes could change what can be called from different threads, making the situation better or worse. From the Tcl/Tk perspective, this is not a problem, and would not be caught by any testing, etc. Even if it were, it likely wouldn't be fixed. It would be considered an "abuse" of their API (I think correctly). My suggestion, given the philosophical and practical mismatch, is that Tkinter move towards operating as if the API Tk provides is inviolate. In other words, all calls into a Tcl interpreter happen from the same thread that created the Tcl interpreter. Tkinter acts as a bridge between Python and Tcl/Tk. It should present an API to Python programs compatible with the Python threading model. It's Tkinter's responsibility to map that onto Tcl/Tk's single threaded API through whatever internal mechanism is necessary (i.e. pass everything to main thread, block caller thread until get response, etc.) That's exactly what Tkinter currently does, see the letter attached to the ticket. Writing clean and correct code is Python's principal standpoint, it doesn't use unsupported functions. I'd go so far as to suggest that all the Tkapp 'call' code (i.e. every place that Tkinter calls Tcl_Eval) check what thread it's in, and issue a warning or error (at least for testing purposes) if it's being called from the "wrong" thread. Having this available in the near future would help people who are debugging what are fairly inexplicable problems now. The approach of making Tkinter responsible also has the advantage of dealing with far more Tcl/Tk versions and builds. Given in practice that few people are really running into things, and that if they are, they know enough to be able to follow the instruction "all Tkinter calls from the same thread" for now, add the warnings/errors in via whatever "turn on debugging" mechanism makes sense. A future version of Python would include a fully thread-safe Tkinter that internally makes all Tcl/Tk calls from a single thread, as per above. Sorry this is so incredibly long-winded. I hope the context at least is useful information. -- ___________ Python tracker <https://bugs.python.org/issue33479> ___ -- -- Regards, Ivan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
