Re: [Python-Dev] \G (match last position) regex operator non-existant in python?

2017-10-28 Thread Nick Coghlan 
On 28 October 2017 at 01:57, Guido van Rossum  wrote:

> Oh. Yes, that is being discussed about once a year two. It seems Matthew
> isn't very interested in helping out with the port, and there are some
> concerns about backwards compatibility with the `re` module. I think it
> needs a champion!
>

Matthew's been amenable to the idea when it comes up, and he explicitly
wrote the module to be usable as a drop-in replacement for "re" (hence the
re-compatible v0 behaviour still being the default).

The resistance has more been from our side, since this is a case where
existing regex module users are clearly better off if it remains a separate
project, as that keeps upgrades independent of the relatively slow standard
library release cycle (and allows it to be used on Python 2.7 as well as in
3.x). By contrast, the potential benefits of standard library inclusion
accrue primarily to Python newcomers and folks writing scripts without the
benefit of package management tools, since they'll have a more capable
regex engine available as part of the assumed language baseline.

That means that if we add regex to the standard library in the regular way,
there's a more than fair chance that we'll end up with an outcome like the
json vs simplejson split, where we have one variant in the standard
library, and another variant on PyPI, and the variants may drift apart over
time if their maintenance is being handled by different people. (Note: one
may argue that we already have this split in the form of re vs regex. So if
regex was brought in specifically to replace _sre as the re module
implementation, rather than as a new public API, then we at least wouldn't
be making anything *worse* from a future behavioural consistency
perspective, but we'd be risking a compatibility break for anyone depending
on _sre and other internal implementation details of the re module).

One potential alternative approach that is then brought up (often by me) is
to suggest instead *bundling* the regex module with CPython, without
actually bringing it fully within the regular standard library maintenance
process. The idea there would be to both make the module available by
default in python.org downloads, *and* make it clear to redistributors that
the module is part of the expected baseline of Python functionality, but
otherwise keep it entirely in its current independently upgradable form.

That would still be hard (since it would involve establishing new
maintenance policy precedents that go beyond the current special-casing of
`pip` in order to bootstrap PyPI access), but would have the additional
benefit of paving the way for doing similar things with other modules where
we'd like them to be part of the assumed baseline for end users, but also
have reasons for wanting to avoid tightly coupling them to the standard
libary's regular maintenance policy (most notably, requests).

And that's where discussions tend to fizzle out:

* outright replacement of the current re module implementation with a
private copy of the regex module introduces compatibility risks that would
need a fiat decision from you as BDFL to say "Let's do it anyway, make sure
the test suite still works, and then figure out how to cope with any other
consequences as they arise"
* going down the bundling path requires making some explicit community
management decisions around what we actually want the standard library to
*be* (and whether or not there's a difference between "the standard
library" and "the assumed available package set" for Python installations
that are expected to run arbitrary third party scripts rather than specific
applications)
* having both the current re API and implementation *and* a new regex based
API and implementation in the standard library indefinitely seems like it
would be a maintainability nightmare that delivered the worst of all
possible outcomes for everyone involved (CPython maintainers, regex
maintainers, Python end users)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-28 Thread francismb 
Hi David,

On 10/22/2017 07:30 PM, David Mertz wrote:
> The 'time' module is about
> wall clock out calendar time, not about *simulation time*.
means that the other scale direction makes more sense for the module?
aka 'get_time('us'), get_time('ms'), 'get_time('s')

Thanks,
--francis
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] If aligned_alloc() is missing on your platform, please let us know.

2017-10-28 Thread Stefan Krah 
On Sat, Oct 28, 2017 at 03:43:47PM +1000, Nick Coghlan wrote:
> 1. CPython's own support for platforms where we don't have a native aligned
> memory allocation API to call is covered by PEP 11, so if all current
> buildbots still work, then it will be up to the folks interested in a
> platform that *doesn't* offer aligned allocations to provide both a
> suitable emulation and a buildbot to test it on.

Indeed, the feature is backed up by PEP 11.


> 2.While all of the CPython-provided memory allocators will implement the
> new slots, the folks implementing their own custom allocators will need a
> defined upgrade path in the "Porting" section of the What's New guide. For
> that, an explicit error on 3.7+ that says "Configured custom allocator
> doesn't implement aligned memory allocations" is likely going to be easier
> to debug than subtle issues with the way an implicit emulation layer
> interacts with the other memory allocator slots.
> 
> To appropriately address 2, we need more info not about which platforms
> natively support aligned memory allocations, but rather from folks that
> actually are implementing their own custom allocators. It may be that
> making the low level cross-platform raw alligned allocators available as a
> public API (independently of the switchable allocator machinery) will be a
> more appropriate way of handling that case than providing an emulation
> layer in terms of the old slots.

I don't have an opinion whether new slots should be available.  For my use
case I just need PyMem_AlignedAlloc(), PyMem_AlignedFree() that automatically
use the faster allocator for 'sizeof(void *) <= align <= ALIGNMENT' and
'size <= SMALL_REQUEST_THRESHOLD'.


So yes, it would be nice to hear from people who implement custom allocators.



Stefan Krah



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 530

2017-10-28 Thread London 
can you help me get idol for my computer

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] \G (match last position) regex operator non-existant in python?

2017-10-28 Thread Guido van Rossum 
On Sat, Oct 28, 2017 at 12:09 AM, Nick Coghlan  wrote:

> On 28 October 2017 at 01:57, Guido van Rossum  wrote:
>
>> Oh. Yes, that is being discussed about once a year two. It seems Matthew
>> isn't very interested in helping out with the port, and there are some
>> concerns about backwards compatibility with the `re` module. I think it
>> needs a champion!
>>
>
> Matthew's been amenable to the idea when it comes up, and he explicitly
> wrote the module to be usable as a drop-in replacement for "re" (hence the
> re-compatible v0 behaviour still being the default).
>
> The resistance has more been from our side, since this is a case where
> existing regex module users are clearly better off if it remains a separate
> project, as that keeps upgrades independent of the relatively slow standard
> library release cycle (and allows it to be used on Python 2.7 as well as in
> 3.x). By contrast, the potential benefits of standard library inclusion
> accrue primarily to Python newcomers and folks writing scripts without the
> benefit of package management tools, since they'll have a more capable
> regex engine available as part of the assumed language baseline.
>
> That means that if we add regex to the standard library in the regular
> way, there's a more than fair chance that we'll end up with an outcome like
> the json vs simplejson split, where we have one variant in the standard
> library, and another variant on PyPI, and the variants may drift apart over
> time if their maintenance is being handled by different people. (Note: one
> may argue that we already have this split in the form of re vs regex. So if
> regex was brought in specifically to replace _sre as the re module
> implementation, rather than as a new public API, then we at least wouldn't
> be making anything *worse* from a future behavioural consistency
> perspective, but we'd be risking a compatibility break for anyone depending
> on _sre and other internal implementation details of the re module).
>
> One potential alternative approach that is then brought up (often by me)
> is to suggest instead *bundling* the regex module with CPython, without
> actually bringing it fully within the regular standard library maintenance
> process. The idea there would be to both make the module available by
> default in python.org downloads, *and* make it clear to redistributors
> that the module is part of the expected baseline of Python functionality,
> but otherwise keep it entirely in its current independently upgradable form.
>
> That would still be hard (since it would involve establishing new
> maintenance policy precedents that go beyond the current special-casing of
> `pip` in order to bootstrap PyPI access), but would have the additional
> benefit of paving the way for doing similar things with other modules where
> we'd like them to be part of the assumed baseline for end users, but also
> have reasons for wanting to avoid tightly coupling them to the standard
> libary's regular maintenance policy (most notably, requests).
>
> And that's where discussions tend to fizzle out:
>
> * outright replacement of the current re module implementation with a
> private copy of the regex module introduces compatibility risks that would
> need a fiat decision from you as BDFL to say "Let's do it anyway, make sure
> the test suite still works, and then figure out how to cope with any other
> consequences as they arise"
> * going down the bundling path requires making some explicit community
> management decisions around what we actually want the standard library to
> *be* (and whether or not there's a difference between "the standard
> library" and "the assumed available package set" for Python installations
> that are expected to run arbitrary third party scripts rather than specific
> applications)
> * having both the current re API and implementation *and* a new regex
> based API and implementation in the standard library indefinitely seems
> like it would be a maintainability nightmare that delivered the worst of
> all possible outcomes for everyone involved (CPython maintainers, regex
> maintainers, Python end users)
>

Maybe it would be easier if Matthew were amenable to maintaining the stdlib
version and only add new features to the PyPI version when they've also
been added to the stdlib version. IOW if he were committed to *not* letting
the [simple]json thing happen.

I don't condone having two different regex implementations/APIs bundled in
any form, even if one were to be deprecated -- we'd never get rid of the
deprecated one until 4.0. (FWIW I don't condone this pattern for other
packages/modules either.) Note that even if we outright switched there
would *still* be two versions, because regex itself has an internal
versioning scheme where V0 claims to be strictly compatible with re and V1
explicitly changes the matching rules in some cases. (I don't know if this
means that you have to request V1 to use \G though.)

The other problem with outright replac

Re: [Python-Dev] PEP 530

2017-10-28 Thread Terry Reedy 

On 10/27/2017 4:43 PM, London wrote:

can you help me get idol for my computer


Post questions about using python on python-list and include information 
about what OS you are running and what version of Python you want.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] \G (match last position) regex operator non-existant in python?

2017-10-28 Thread MRAB 

On 2017-10-28 22:05, Guido van Rossum wrote:
On Sat, Oct 28, 2017 at 12:09 AM, Nick Coghlan > wrote:


On 28 October 2017 at 01:57, Guido van Rossum mailto:gu...@python.org>> wrote:

Oh. Yes, that is being discussed about once a year two. It
seems Matthew isn't very interested in helping out with the
port, and there are some concerns about backwards
compatibility with the `re` module. I think it needs a champion!


Matthew's been amenable to the idea when it comes up, and he
explicitly wrote the module to be usable as a drop-in replacement
for "re" (hence the re-compatible v0 behaviour still being the
default).

The resistance has more been from our side, since this is a case
where existing regex module users are clearly better off if it
remains a separate project, as that keeps upgrades independent of
the relatively slow standard library release cycle (and allows it
to be used on Python 2.7 as well as in 3.x). By contrast, the
potential benefits of standard library inclusion accrue primarily
to Python newcomers and folks writing scripts without the benefit
of package management tools, since they'll have a more capable
regex engine available as part of the assumed language baseline.

That means that if we add regex to the standard library in the
regular way, there's a more than fair chance that we'll end up
with an outcome like the json vs simplejson split, where we have
one variant in the standard library, and another variant on PyPI,
and the variants may drift apart over time if their maintenance is
being handled by different people. (Note: one may argue that we
already have this split in the form of re vs regex. So if regex
was brought in specifically to replace _sre as the re module
implementation, rather than as a new public API, then we at least
wouldn't be making anything *worse* from a future behavioural
consistency perspective, but we'd be risking a compatibility break
for anyone depending on _sre and other internal implementation
details of the re module).

One potential alternative approach that is then brought up (often
by me) is to suggest instead *bundling* the regex module with
CPython, without actually bringing it fully within the regular
standard library maintenance process. The idea there would be to
both make the module available by default in python.org
 downloads, *and* make it clear to
redistributors that the module is part of the expected baseline of
Python functionality, but otherwise keep it entirely in its
current independently upgradable form.

That would still be hard (since it would involve establishing new
maintenance policy precedents that go beyond the current
special-casing of `pip` in order to bootstrap PyPI access), but
would have the additional benefit of paving the way for doing
similar things with other modules where we'd like them to be part
of the assumed baseline for end users, but also have reasons for
wanting to avoid tightly coupling them to the standard libary's
regular maintenance policy (most notably, requests).

And that's where discussions tend to fizzle out:

* outright replacement of the current re module implementation
with a private copy of the regex module introduces compatibility
risks that would need a fiat decision from you as BDFL to say
"Let's do it anyway, make sure the test suite still works, and
then figure out how to cope with any other consequences as they arise"
* going down the bundling path requires making some explicit
community management decisions around what we actually want the
standard library to *be* (and whether or not there's a difference
between "the standard library" and "the assumed available package
set" for Python installations that are expected to run arbitrary
third party scripts rather than specific applications)
* having both the current re API and implementation *and* a new
regex based API and implementation in the standard library
indefinitely seems like it would be a maintainability nightmare
that delivered the worst of all possible outcomes for everyone
involved (CPython maintainers, regex maintainers, Python end users)


Maybe it would be easier if Matthew were amenable to maintaining the 
stdlib version and only add new features to the PyPI version when 
they've also been added to the stdlib version. IOW if he were 
committed to *not* letting the [simple]json thing happen.


I don't condone having two different regex implementations/APIs 
bundled in any form, even if one were to be deprecated -- we'd never 
get rid of the deprecated one until 4.0. (FWIW I don't condone this 
pattern for other packages/modules either.) Note that even if we 
outright switched there would *still* be two versions, becaus

Re: [Python-Dev] \G (match last position) regex operator non-existant in python?

2017-10-28 Thread Steven D'Aprano 
On Sun, Oct 29, 2017 at 12:31:01AM +0100, MRAB wrote:

> Not that I'm planning on making any further additions, just bug fixes 
> and updates to follow the Unicode updates. I think I've crammed enough 
> into it already. There's only so much you can do with the regex syntax 
> with its handful of metacharacters and possible escape sequences...

What do you think of the Perl 6 regex syntax?

https://en.wikipedia.org/wiki/Perl_6_rules#Changes_from_Perl_5



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] \G (match last position) regex operator non-existant in python?

2017-10-28 Thread Peter Ludemann via Python-Dev 
On 28 October 2017 at 16:48, Steven D'Aprano  wrote:

> On Sun, Oct 29, 2017 at 12:31:01AM +0100, MRAB wrote:
>
> > Not that I'm planning on making any further additions, just bug fixes
> > and updates to follow the Unicode updates. I think I've crammed enough
> > into it already. There's only so much you can do with the regex syntax
> > with its handful of metacharacters and possible escape sequences...
>
> What do you think of the Perl 6 regex syntax?
>
> https://en.wikipedia.org/wiki/Perl_6_rules#Changes_from_Perl_5


​If you're going to change the notation, why not use notations similar to
what linguists use for FSTs? These allow building FSTs (with operations
such as adding/subtracting/composing/projecting FSTs) with millions of
states — and there are some impressive optimisers for them also, so that
encoding a dictionary with inflections is both more compact and faster than
a hash of just the words without inflections. Some of this work is open
source, but I haven't kept up with it.

If you're interested, you can start here:
http://web.stanford.edu/~laurik/​
http://web.stanford.edu/~laurik/publications/TR-2010-01.pdf
http://web.stanford.edu/group/cslipublications/cslipublications/site/1575864347.shtml
etc.

;)




>
>
>
>
> --
> Steve
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> pludemann%40google.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] \G (match last position) regex operator non-existant in python?

2017-10-28 Thread MRAB 

On 2017-10-29 00:48, Steven D'Aprano wrote:

On Sun, Oct 29, 2017 at 12:31:01AM +0100, MRAB wrote:

Not that I'm planning on making any further additions, just bug fixes 
and updates to follow the Unicode updates. I think I've crammed enough 
into it already. There's only so much you can do with the regex syntax 
with its handful of metacharacters and possible escape sequences...


What do you think of the Perl 6 regex syntax?

https://en.wikipedia.org/wiki/Perl_6_rules#Changes_from_Perl_5

I think I prefer something that's more like PEG, with quoted literals, 
perhaps because it looks more like a programming language, but also 
because it's clearer than saying "these characters are literal, but 
those aren't". That webpage says "Literals: word characters (letters, 
numbers and underscore) matched literally", but is that all letters? And 
what about diacritics, and combining characters?


I'm not keen on  and , I like & and ! better, 
but then how would you write a lookbehind?


Named rules are good, better than regex's use of named capture groups, 
and if you quote literal, then you wouldn't need to wrap rule call in 
<...>, as Perl 6 does.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] \G (match last position) regex operator non-existant in python?

2017-10-28 Thread Nick Coghlan 
On 29 October 2017 at 09:31, MRAB  wrote:

> On 2017-10-28 22:05, Guido van Rossum wrote:
>
>> I don't condone having two different regex implementations/APIs bundled
>> in any form, even if one were to be deprecated -- we'd never get rid of the
>> deprecated one until 4.0. (FWIW I don't condone this pattern for other
>> packages/modules either.) Note that even if we outright switched there
>> would *still* be two versions, because regex itself has an internal
>> versioning scheme where V0 claims to be strictly compatible with re and V1
>> explicitly changes the matching rules in some cases. (I don't know if this
>> means that you have to request V1 to use \G though.)
>>
>> The other problem with outright replacement is that despite Matthew's
>> best efforts there may be subtle incompatibilities that will break people's
>> code in surprising ways. I don't recall much about our current 're' test
>> suite -- I'm sure it tests every feature, but I'm not sure how far it goes
>> in testing edge cases. IIRC this is where in the past we've always erred on
>> the side of (extreme) caution, and my recollection is of Matthew being
>> (understandably!) pretty lukewarm about doing extra work to help assess
>> this -- IIRC he's totally fine with the status quo.
>>
>> If there's new information or a change in Matthew's outlook I'd be happy
>> to reconsider it.
>>
>> At one time I was in favour of including it in the stdlib, but then I
> changed my mind. Having it outside gives me more flexibility, and I'm happy
> with just using pip.
>

OK, so I think that leaves the notion of a "Recommended baseline package
set" as the most realistic option we have for improvement in this area -
coming up with a way for us as the standard library maintainers to make
particular 3rd party components more readily available to end users, while
also providing explicit guidance to 3rd party redistributors that we think
those components should be offered by default in general purpose scripting
environments.

I'll start a thread on python-ideas about that, as I think we could get
quite some way towards that goal with just some minor additions to the
ensurepip and venv modules (using existing documented third party
recommendations like those in the re docs for regex and the urllib.request
docs for requests), without actually bundling anything directly into the
python.org installers.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] \G (match last position) regex operator non-existant in python?

2017-10-28 Thread Greg Ewing 

Guido van Rossum wrote:
From this I understand that when using e.g. findall() it forces 
successive matches to be adjacent.


Seems to me this would be better addressed using an option
to findall() rather than being part of the regex. That would
avoid the issue of where to keep the state.

--
Greg

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com