Victor Stinner schrieb am 15.04.2016 um 00:33:
> 2016-04-15 0:22 GMT+02:00 Brett Cannon:
>> And even if it was GIL-free you do run the risk of two dicts ending up at
>> the same version # by simply mutating the same number of times if the
>> counters were per-dict instead of process-wide.
>
> For
On Wed, Apr 13, 2016 at 4:40 AM, Victor Stinner
wrote:
> Last months, most 3.x buildbots failed randomly. Some of them were
> always failing. I spent some time to fix almost all Windows and Linux
> buildbots. There were a lot of different issues.
>
> So please try to not break buildbots again and
Now that PEP 506 has been approved, I've checked in the secrets module,
but an implementation question has come up regarding compare_digest.
Currently, the module tries to import hmac.compare_digest, and if that
fails, then it falls back to a Python version. But since compare_digest
has been av
On 04/14/2016 06:01 PM, Ethan Furman wrote:
On 04/14/2016 05:20 PM, Stephen J. Turnbull wrote:
you'll have to impose it on me.
Hmm. Well, the good news is you have convinced me that letting bytes
through willy-nilly is akin to loosing the hounds of hell on our code.
The bad news is I was ne
On Thu, Apr 14, 2016, 17:14 MRAB wrote:
> On 2016-04-14 21:42, Armin Rigo wrote:
> > Hi Victor,
> >
> > On 14 April 2016 at 17:19, Victor Stinner
> wrote:
> >> Each time a dictionary is created, the global
> >> version is incremented and the dictionary version is initialized to the
> >> global v
On 04/14/2016 05:20 PM, Stephen J. Turnbull wrote:
However, the proposed polymorphism does create ambiguity and risk for
my uses. I rarely have the luxury of *not* ensuring paths are text,
regardless of the bytes-ness of the underlying application, because I
can be pretty darn sure that somebod
Ethan Furman writes:
> Substitute open() with sending those bytes somewhere else:
Eg, pathlib.Path, which will raise? Surely it should be safe to pass
a DirEntry to a pathlib constructor? Note that having Path call
fsdecode implicitly is a bad idea, because we don't know the
provenance of gene
On 2016-04-14 21:42, Armin Rigo wrote:
Hi Victor,
On 14 April 2016 at 17:19, Victor Stinner wrote:
Each time a dictionary is created, the global
version is incremented and the dictionary version is initialized to the
global version.
A detail, but why not set the version tag of new empty dict
On 2016-04-14 4:42 PM, Armin Rigo wrote:
Hi Victor,
On 14 April 2016 at 17:19, Victor Stinner wrote:
Each time a dictionary is created, the global
version is incremented and the dictionary version is initialized to the
global version.
A detail, but why not set the version tag of new empty d
On 4/14/2016 3:33 PM, Victor Stinner wrote:
When we will be able to get ride of the GIL for the dict type, we will
probably be able to get an atomic "global_version++" for 64-bit
integer. Right now, I don't think that an atomic int64++ is available
on 32-bit archs.
By the time we get an atomic in
2016-04-15 0:22 GMT+02:00 Brett Cannon :
> And even if it was GIL-free you do run the risk of two dicts ending up at
> the same version # by simply mutating the same number of times if the
> counters were per-dict instead of process-wide.
For some optimizations, it is not needed to check if the di
On Thu, 14 Apr 2016 at 15:14 Victor Stinner
wrote:
> 2016-04-14 23:29 GMT+02:00 Barry Warsaw :
> > I can see why you might want a global version number, but not doing so
> would
> > eliminate an implicit reliance on the GIL, or in a GIL-less
> implementation
> > a lock around incrementing the gl
2016-04-14 23:29 GMT+02:00 Barry Warsaw :
> I can see why you might want a global version number, but not doing so would
> eliminate an implicit reliance on the GIL, or in a GIL-less implementation
> a lock around incrementing the global version number.
It's not like the builtin dict type is goin
On Apr 14, 2016, at 11:17 PM, Victor Stinner wrote:
>You're right that incrementing the global version is useless for these
>specific cases, and using the version 0 should work. It only matters
>that the version (version? version tag?) is different.
>
>I will play with that. If I don't see any iss
2016-04-14 22:50 GMT+02:00 Barry Warsaw :
> Although I'm not totally convinced, I won't continue to object. You've
> provided some performance numbers in the PEP even without FAT, and you aren't
> exposing the API to Python, so it's not a burden being imposed on other
> implementations.
Cool!
Ah
Hi,
2016-04-14 22:42 GMT+02:00 Armin Rigo :
> Hi Victor,
>
> On 14 April 2016 at 17:19, Victor Stinner wrote:
>> Each time a dictionary is created, the global
>> version is incremented and the dictionary version is initialized to the
>> global version.
>
> A detail, but why not set the version ta
> From Armin Rigo
> Sent: Thursday, April 14, 2016 4:42 PM
> To: Victor Stinner
> Cc: Python Dev
> Subject: Re: [Python-Dev] RFC: PEP 509: Add a private version to dict
>
> Hi Victor,
>
> On 14 April 2016 at 17:19, Victor Stinner wrote:
> > Each time a dictionary is created, the global
> > versi
On Apr 14, 2016, at 09:49 PM, Victor Stinner wrote:
>It would be nice to hear Barry Warsow who was opposed to the PEP in
>january. He wanted to wait until FAT Python was proven to really be faster,
>which is still not case right now. (I mean that I didnt't run seriously
>benchmarks, but early macr
Hi Victor,
On 14 April 2016 at 17:19, Victor Stinner wrote:
> Each time a dictionary is created, the global
> version is incremented and the dictionary version is initialized to the
> global version.
A detail, but why not set the version tag of new empty dictionaries to
zero, always? Same afte
Victor Stinner schrieb am 14.04.2016 um 21:56:
> Which kind of usage do you see in Cython?
Mainly caching, I guess. We could avoid global/module name lookups in some
cases, especially inside of loops.
> Off-topic (PEP 510):
>
> I really want to experiment automatic generation of Cython code fro
Which kind of usage do you see in Cython?
Off-topic (PEP 510):
I really want to experiment automatic generation of Cython code from the
Python using profiling to discover function parameters types. Then use the
PEP 510 to attach the fast Cython code to a Python function, but fallback
to bytecode
It would be nice to hear Barry Warsow who was opposed to the PEP in
january. He wanted to wait until FAT Python was proven to really be faster,
which is still not case right now. (I mean that I didnt't run seriously
benchmarks, but early macro benchmarks are not really promising, only micro
benchma
On Thu, Apr 14, 2016 at 9:35 PM, Random832 wrote:
> On Thu, Apr 14, 2016, at 13:56, Koos Zevenhoven wrote:
>> (1) Code that has access to pathname/filename data and has some level
>> of control over what data type comes in. This code may for instance
>> choose to deal with either bytes or str
>>
>
On 04/13/2016 02:37 PM, Victor Stinner wrote:
I'm not a big fan of a flag parameter to change the return type of a
function. Usually, two functions are preferred. In the os module we have
getcwd/getcwdb for example. I don't know if it's a good example
I think of os.fspath() as more of a filter
On Thu, Apr 14, 2016, at 13:56, Koos Zevenhoven wrote:
> (1) Code that has access to pathname/filename data and has some level
> of control over what data type comes in. This code may for instance
> choose to deal with either bytes or str
>
> (2) Code that takes the path or file name that it happe
On 04/14/2016 10:22 AM, Paul Moore wrote:
On 14 April 2016 at 17:46, Ethan Furman wrote:
If you are not working at the bytes layer, you shouldn't be getting bytes
objects because:
- you specified str when asking for data from the OS, or
- you transformed the incoming bytes from whatever exter
On 04/14/2016 10:02 AM, Random832 wrote:
"between versions" is ambiguous. It could mean that there's no guarantee
that there will be no changes from one version to the next, or it could
mean, even more strongly, that there's no guarantee that there will be
no changes in a maintenance release (wh
On Thu, Apr 14, 2016 at 7:46 PM, Ethan Furman wrote:
>
> What many folks seem to be missing is that *you* (generic you) have control
> of your data.
>
> If you are not working at the bytes layer, you shouldn't be getting bytes
> objects because:
>
> - you specified str when asking for data from th
On 14 April 2016 at 17:46, Ethan Furman wrote:
> On 04/14/2016 08:59 AM, Michael Mysinger via Python-Dev wrote:
>
>> I am saying that if os.path.join now accepts RichPath objects, and those
>> objects can return either str or bytes, then its much harder to reason
>> about
>> when I have all bytes
On Thu, Apr 14, 2016, at 12:56, Terry Reedy wrote:
> https://docs.python.org/3/library/dis.html#module-dis
> CPython implementation detail: Bytecode is an implementation detail of
> the CPython interpreter. No guarantees are made that bytecode will not
> be added, removed, or changed between vers
I'll wait a day before formally pronouncing to see if any objections
are made, but it looks good to me.
On Thu, Apr 14, 2016 at 8:19 AM, Victor Stinner
wrote:
> Hi,
>
> I updated my PEP 509 to make the dictionary version globally unique.
> With *two* use cases of this PEP (Yury's method call patc
On 4/14/2016 12:03 PM, Nikita Nemkin wrote:
I think that Python should make bytecode explicitly unstable and subject
to change with any major release.
https://docs.python.org/3/library/dis.html#module-dis
CPython implementation detail: Bytecode is an implementation detail of
the CPython inter
+1 from me, too. I'm sure we can make some use of this in Cython.
Stefan
Victor Stinner schrieb am 14.04.2016 um 17:19:
> PEP: 509
> Title: Add a private version to dict
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailma
On 04/14/2016 08:59 AM, Michael Mysinger via Python-Dev wrote:
I am saying that if os.path.join now accepts RichPath objects, and those
objects can return either str or bytes, then its much harder to reason about
when I have all bytes or all strings. In essence, you will force me to pre-
wrap al
2016-04-14 18:28 GMT+02:00 Brett Cannon :
> +1 from me!
Thanks.
> A couple of grammar/typo suggestions below.
Fixed. (Yes, I want to use unsigned type, so PY_UINT64_T.)
Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/m
On 04/14/2016 09:09 AM, Victor Stinner wrote:
2016-04-14 16:54 GMT+02:00 Ethan Furman:
I consider that the final goal of the whole discussion is to support
something like:
path = os.path.join(pathlib_path, "str_path", direntry)
(...)
I expect that DirEntry.__fspath__ uses os.fsdecode()
Donald Stufft stufft.io> writes:
> > On Apr 14, 2016, at 11:59 AM, Michael Mysinger via Python-Dev python.org> wrote:
> >
> > In essence, you will force me to pre-
> > wrap all RichPath objects in either os.fsencode(os.fspath(path)) or
> > os.fsdecode(os.fspath(path)), just so I can reason abou
On Thu, 14 Apr 2016 at 09:16 Nikita Nemkin wrote:
> On Thu, Apr 14, 2016 at 8:32 PM, Victor Stinner
> wrote:
> >
> > Would you like to work on a patch to implement that change?
>
> I'll work on a patch. Should I post it to bugs.python.org?
>
Yep.
>
> > Since Python 3.6 may get a new bytecode
+1 from me!
A couple of grammar/typo suggestions below.
On Thu, 14 Apr 2016 at 08:20 Victor Stinner
wrote:
> Hi,
>
> I updated my PEP 509 to make the dictionary version globally unique.
> With *two* use cases of this PEP (Yury's method call patch and my FAT
> Python project), I think that the P
On Thu, Apr 14, 2016, at 12:05, Stephen J. Turnbull wrote:
> Random832 writes:
>
> > And what such incompatibilities exist between bytes and str for the
> > purpose of representing file paths?
>
> A plethora of encodings.
Only one encoding, fsencode/fsdecode. All other encodings are not for
fi
On Thu, Apr 14, 2016 at 8:27 PM, Guido van Rossum wrote:
> Great analysis! What might stand in the way of adoption is concern for
> bytecode manipulation libraries that would have to be changed. What
> might encourage adoption would be a benchmark showing this saves a lot
> of time.
>
> Personally
On Thu, Apr 14, 2016 at 8:32 PM, Victor Stinner
wrote:
>
> Would you like to work on a patch to implement that change?
I'll work on a patch. Should I post it to bugs.python.org?
> Since Python 3.6 may get a new bytecode format (wordcode, see the
> other thread on this mlailing list), I think th
> On Apr 14, 2016, at 11:59 AM, Michael Mysinger via Python-Dev
> wrote:
>
> In essence, you will force me to pre-
> wrap all RichPath objects in either os.fsencode(os.fspath(path)) or
> os.fsdecode(os.fspath(path)), just so I can reason about the type.
This is only the case if you have a sin
2016-04-14 17:29 GMT+02:00 Ethan Furman :
> Interoperability with other systems and/or libraries. If we use
> surrogateescape to transform str to bytes, and the other side does not, we
> no longer have a workable path.
I guess that you mean a Python library? When you exchange with
external progra
2016-04-14 16:54 GMT+02:00 Ethan Furman :
>> I consider that the final goal of the whole discussion is to support
>> something like:
>>
>> path = os.path.join(pathlib_path, "str_path", direntry)
>>
>> (...)
>> I expect that DirEntry.__fspath__ uses os.fsdecode() to return str,
>> just to make
Random832 writes:
> And what such incompatibilities exist between bytes and str for the
> purpose of representing file paths?
A plethora of encodings.
> At the end of the day, there's exactly one answer to "what file on
> disk this represents (or would represent if it existed)".
Nope. Supp
Ethan Furman stoneleaf.us> writes:
> On 04/14/2016 12:03 AM, Michael Mysinger via Python-Dev wrote:
> > In particular, one RichPath
> > class might return bytes and another str, or even worse the same class
might
> > sometimes return bytes and sometimes str. When will os.path.join blow up
due
>
On Thu, 14 Apr 2016 at 03:26 Victor Stinner
wrote:
>
> Le 14 avr. 2016 11:16 AM, "Serhiy Storchaka" a
> écrit :
> > A desirable but nonexistent feature is to write emails to authors of
> commits that broke buildbots. How hard to implement this?
>
> Yeah I also had this idea since many years but
2016-04-14 17:27 GMT+02:00 Guido van Rossum :
> Great analysis! What might stand in the way of adoption is concern for
> bytecode manipulation libraries that would have to be changed.
> (...)
> There's also talk of switching to wordcode, in a different thread.
I agree that breaking backward compat
2016-04-14 11:04 GMT+02:00 Nikita Nemkin :
> MAKE_FUNCTION opcode is complex due to the way it receives
> input arguments: (...)
Yeah, I was always disturbed how this opcode gets parameters.
> My suggestion is to pre-package 1-4 before calling MAKE_FUNCTION,
> i.e. explicitly emit BUILD_TUPLE for
Great analysis! What might stand in the way of adoption is concern for
bytecode manipulation libraries that would have to be changed. What
might encourage adoption would be a benchmark showing this saves a lot
of time.
Personally I'm expecting it won't make much of a difference for real
programs s
On 04/14/2016 07:01 AM, Random832 wrote:
On Thu, Apr 14, 2016, at 09:50, Chris Angelico wrote:
Adding integers and floats is considered "safe" because most people's
use of floats completely compasses their use of ints. (You'll get
OverflowError if it can't be represented.) But float and Decimal
On 04/14/2016 07:52 AM, Stephen J. Turnbull wrote:
Nick Coghlan writes:
The use case for returning bytes from __fspath__ is DirEntry, so you
can write things like this in low level code:
def myscandir(dirpath):
for entry in os.scandir(dirpath):
if entry.is_file():
Hi,
I updated my PEP 509 to make the dictionary version globally unique.
With *two* use cases of this PEP (Yury's method call patch and my FAT
Python project), I think that the PEP is now ready to be accepted.
Globally unique identifier is a requirement for Yury's patch
optimizing method calls (
MAKE_FUNCTION opcode is complex due to the way it receives
input arguments:
1) default args, individually;
2) default kwonly args, individual name-value pairs;
3) a tuple of parameter names (single constant);
4) annotation values, individually;
5) code object;
6) qualname.
The counts for 1,
On 04/14/2016 06:56 AM, Victor Stinner wrote:
2016-04-14 15:40 GMT+02:00 Nick Coghlan:
>> Even earlier, Victor Stinner wrote:
I consider that the final goal of the whole discussion is to support
something like:
path = os.path.join(pathlib_path, "str_path", direntry)
That's not a *new*
Random832 writes:
> On Thu, Apr 14, 2016, at 03:02, Stephen J. Turnbull wrote:
> > I have a strong preference for str only, because I still don't see a
> > use case for polymorphic __fspath__.
>
> Ultimately we're talking about redundancy and performance here.
Ultimately, yes. Right now I
On 04/14/2016 05:16 AM, Victor Stinner wrote:
I consider that the final goal of the whole discussion is to support
something like:
path = os.path.join(pathlib_path, "str_path", direntry)
Even if direntry uses a bytes filename. I expect genericpath.join() to
be patched to use os.fspath().
Nick Coghlan writes:
> The use case for returning bytes from __fspath__ is DirEntry, so you
> can write things like this in low level code:
>
> def myscandir(dirpath):
> for entry in os.scandir(dirpath):
> if entry.is_file():
> with open(entry) as f:
On 04/14/2016 12:03 AM, Michael Mysinger via Python-Dev wrote:
Brett Cannon writes:
After playing with and considering the 4 possibilities, anything where
__fspath__ can return bytes seems like insanity that flies in the face of
everything Python 3 is trying to accomplish. In particular, one R
2016-04-14 15:40 GMT+02:00 Nick Coghlan :
>> I consider that the final goal of the whole discussion is to support
>> something like:
>>
>> path = os.path.join(pathlib_path, "str_path", direntry)
>
> That's not a *new* problem though, it already exists if you pass in a
> mix of bytes and str:
>
On Thu, Apr 14, 2016, at 09:50, Chris Angelico wrote:
> Adding integers and floats is considered "safe" because most people's
> use of floats completely compasses their use of ints. (You'll get
> OverflowError if it can't be represented.) But float and Decimal are
> considered "unsafe":
>
> >>> 1.
On Thu, Apr 14, 2016, at 09:40, Nick Coghlan wrote:
> That's not a *new* problem though, it already exists if you pass in a
> mix of bytes and str:
>
> There's also already a solution (regardless of whether you want bytes
> or str as the result), which is to explicitly coerce all the arguments
> t
On Thu, Apr 14, 2016 at 11:45 PM, Random832 wrote:
> On Thu, Apr 14, 2016, at 09:40, Nick Coghlan wrote:
>> That's not a *new* problem though, it already exists if you pass in a
>> mix of bytes and str:
>>
>> There's also already a solution (regardless of whether you want bytes
>> or str as the re
On 14 April 2016 at 22:16, Victor Stinner wrote:
> 2016-04-13 19:10 GMT+02:00 Brett Cannon :
>> https://gist.github.com/brettcannon/b3719f54715787d54a206bc011869aa1 has the
>> four potential approaches implemented (although it doesn't follow the
>> "separate functions" approach some are proposing
On Thu, Apr 14, 2016, at 03:02, Stephen J. Turnbull wrote:
> I have a strong preference for str only, because I still don't see a
> use case for polymorphic __fspath__.
Ultimately we're talking about redundancy and performance here. The "use
case" such as there is one, is if there's a class (be it
On Thu, Apr 14, 2016, at 02:00, Nick Coghlan wrote:
> > If the protocol can return bytes, then that means that types (DirEntry?
> > someone had an alternate path library with a bPath?) which return bytes
> > via the protocol will proliferate, and cannot be safely passed to
> > anything that uses os
2016-04-13 19:10 GMT+02:00 Brett Cannon :
> https://gist.github.com/brettcannon/b3719f54715787d54a206bc011869aa1 has the
> four potential approaches implemented (although it doesn't follow the
> "separate functions" approach some are proposing and instead goes with the
> allow_bytes approach I orig
Le jeudi 14 avril 2016, Nick Coghlan a écrit :
>
> > IHMO it's not a big deal to update these projects for the future
> > Python 3.6. I can even help them to support the new bytecode format.
>
> We've also had previous discussions on adding a "minimum viable
> bytecode editing" API to the standard
On 14 April 2016 at 17:02, Stephen J. Turnbull wrote:
> But WDOT? I'd especially like to hear if Nick is tempted to flip-flop
> (so far he's been in the "pathlib is a text utility" camp).
pathlib is too high level (i.e. has too many dependencies) to be used
in low level boundary code.
The use c
2016-04-14 13:01 GMT+02:00 Serhiy Storchaka :
> But this filter is not quite robust, for example it will cause this mail to
> be moved to the folder for Rietveld reviews.
Right, it's just a workaround since I'm unable to fix the root cause
(emails marked as spam which looks like a configuration is
On 14.04.16 13:33, Martin Panter wrote:
On 14 April 2016 at 08:51, Serhiy Storchaka wrote:
Most bug tracker emails still went in the Spam folder. I have a filter for
Roundap emails, but there is no any mark that I can use for filtering
Rietveld emails.
FWIW I set up the following filter in Gm
On 14 April 2016 at 08:51, Serhiy Storchaka wrote:
> On 13.04.16 07:39, Terry Reedy wrote:
>>
>> On 4/4/2016 5:05 PM, Terry Reedy wrote:
>>
>> Since a few days, I am getting bug tracker emails again, in my Inbox. I
>> just got a Rietveld review in the Inbox and I believe it went there
>> directly
Le 14 avr. 2016 10:53 AM, "Serhiy Storchaka" a écrit :
> Most bug tracker emails still went in the Spam folder. I have a filter
for Roundap emails, but there is no any mark that I can use for filtering
Rietveld emails.
I'm using the base URL of Rietveld and match it in the mail body. Gmail
filter
IMHO it's more a side effect of the implementation than a deliberate
choice. For new code which really want to support bytes paths, I suggest to
only accept bytes and bytes subclasses.
Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail
Le 14 avr. 2016 11:16 AM, "Serhiy Storchaka" a écrit :
> A desirable but nonexistent feature is to write emails to authors of
commits that broke buildbots. How hard to implement this?
Yeah I also had this idea since many years but buildbots were quite
unstable. Maybe we should be more strict to c
On 14 April 2016 at 09:15, Serhiy Storchaka wrote:
> On 13.04.16 14:40, Victor Stinner wrote:
>> By the way, I don't understand why "AMD64 OpenIndiana 3.x" is
>> considered as stable since it's failing with multiple issues since
>> many months and nobody is working on these failures. I suggest to
On 14 April 2016 at 08:02, Stephen J. Turnbull wrote:
> So let me propose what I think is the elephant in the room. If you're
> going to have a polymorphic __fspath__, then pathlib is *the* example
> of a module that *desperately* needs to be polymorphic. Consider:
>
> A non-text Application
On 13.04.16 14:40, Victor Stinner wrote:
Last months, most 3.x buildbots failed randomly. Some of them were
always failing. I spent some time to fix almost all Windows and Linux
buildbots. There were a lot of different issues.
Excelent! Many thanks for doing this. And new features of regrtest l
On 13.04.16 07:39, Terry Reedy wrote:
On 4/4/2016 5:05 PM, Terry Reedy wrote:
Since a few days, I am getting bug tracker emails again, in my Inbox. I
just got a Rietveld review in the Inbox and I believe it went there
directly instead of first to Junk. Thank you to whoever made the
improvement
What types should be accepted as bytes path?
For now os.path is strict and accepts only bytes and bytes subclasses
(even bytearray is not accepted) as bytes path. This is enough for
working with low-level Posix paths and supporting backward compatibility.
On other hand, most os functions is t
Brett Cannon python.org> writes:
> https://gist.github.com/brettcannon/b3719f54715787d54a206bc011869aa1 has
the four potential approaches implemented (although it doesn't follow the
"separate functions" approach some are proposing and instead goes with the
allow_bytes approach I originally pro
I was going to read the new posts that came in since I started this
one (at one point it was 5X as long as it is now), but this thread is
way out of control. My apologies to anybody who has presented[1] use
cases in support of the wildly speculative proposals under discussion,
but my bet is that t
Please please please, junk both "filter out bytes" proposals.
Since they involve an exception, they impose an unnecessary "try" on
all text applications that fear death on bytes returns. May as well
just wrap all objects with __fspath__ in fsdecode, and all is
happy.
Counterproposal: make fsdeco
84 matches
Mail list logo