[Python-Dev] Re: Changing Python's string search algorithms

[Tim Peters]

[Tim Peters, explains one of the new algorithm's surprisingly
 effective moving parts]

[Chris Angelico ]
> Thank you, great explanation. Can this be added to the source code
> if/when this algorithm gets implemented?

No ;-)  While I enjoy trying to make hard things clear(er), I need to
understand them inside out myself first, and I'm still not there with
this algorithm.

Even of what I do grok now, I cheated some to make that post simple.
For example, if you read it carefully, you'll realize that the
explanation I gave doesn't actually explain why the "x" * 100 example
is correct: the explanation implicitly relied on that the left half
the splitting (u) is at least as long as the right half (v).  But
that's not always the case (and, in particular, for "x" * 100, u is
empty!).

Tal Einat posted earlier that he was keen to try to work up a clear
explanation, and I look forward to that. All the expositions I've
found of this algorithm so far are hard to approach.  The original
paper is still the best I've seen.  Everything I wrote was but a gloss
on its Lemma 3.2.

> I've learned all kinds of things from the huge block comments
> about timsort, list growth, and the like. They make great reading
> when you're trying to figure out how to explain things (or if
> you're a bored nerd browsing for curiosity's sake - that works too!).

Takes one to know one ;-)  There's beauty and elegance in this
algorithm, but they'll go unloved without explanations clear enough
for the non-obsessed to follow.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HMSOQD22CLHLZ4VEWV6FPRA2RC3TM75H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Changing Python's string search algorithms

[Chris Angelico]

On Sat, Oct 17, 2020 at 12:30 PM Tim Peters  wrote:
>
> I don't plan on making a series of these posts, just this one, to give
> people _some_ insight into why the new algorithm gets systematic
> benefits the current algorithm can't.  It splits the needle into two
> pieces, u and v, very carefully selected by subtle linear-time needle
> preprocessing (and it's quite a chore to prove that a suitable
> splitting always exists!):
>
> needle == u + v
>
> [... more details...]

Thank you, great explanation. Can this be added to the source code
if/when this algorithm gets implemented? I've learned all kinds of
things from the huge block comments about timsort, list growth, and
the like. They make great reading when you're trying to figure out how
to explain things (or if you're a bored nerd browsing for curiosity's
sake - that works too!).

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7QZPJAELSJ426IF6TCVWACZ4P5RXEJMJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Changing Python's string search algorithms

[Tim Peters]

I don't plan on making a series of these posts, just this one, to give
people _some_ insight into why the new algorithm gets systematic
benefits the current algorithm can't.  It splits the needle into two
pieces, u and v, very carefully selected by subtle linear-time needle
preprocessing (and it's quite a chore to prove that a suitable
splitting always exists!):

needle == u + v

There are a whole bunch of things that can be said about this
splitting (and must be said to prove worst-case linear time), but
here's just one for which it's easy to show the benefit:  no
(non-empty) prefix of v is a suffix of u.

Now I think it's quite easy to twist your head into knots trying to
see what follows from that, so I'll just give a highly generalizable
concrete example.  Suppose v starts with "xyza".  Then u cannot end
with "x", "xy", "xyz", or "xyza". If u did, then u would have a suffix
that's also a prefix of v.

A match attempt at a given haystack position starts by matching the
characters in v one at a time, left to right. Here with haystack on
the first line, needle on the second, and "0" denoting "don't care /
unknown / doesn't exist" - it's just _some_ glyph so the example has
some chance of lining up under annoying proportional fonts :-(

Say the first character matches:

0x00
0xyzavvv

And the second and third:

0xyz
0xyzavvv

But the 4th doesn't:

0xyzZ000
0xyzavvv

Is there any possibility of a match if we shift the needle one to the
right? No! If we tried, the last character of u would line up with the
"x" in the haystack, and we already know "x" is not a suffix of u.

How about if we shifted two to the right? No again! And for the same
reason: then the last two characters of u would line up with "xy" in
the haystack, but we already know "xy" is not a suffix of u.

And so on.  In general, if we find a mismatch while trying the k'th
(1-based) character of v, we can shift the needle k places to the
right before there's any possibility of matching.  In this case, k=4,
so we can jump to:

0xyzA000
0xyzavvv

Note that no "extra" storage is needed to exploit this. No character
lookups, no extra expenses in time or space of any kind.  Just "if we
mismatch on the k'th try, we can jump ahead k positions".  Nothing
special about "xyza" either - this works with any characters at all!

"xyza" isn't by any stretch a hard case regardless.  But it goes
_some_ way toward hinting at why potentially hard cases are rendered
toothless too.  For example, the needle "x" * 100 is split into

u = "" # yup, empty!
v = needle

Now, e.g., if we find a mismatch at the 83'rd character of v, we can
leap ahead 83 positions immediately.

What about needle "x" * 100 + "y"?  I picked that to crush your
premature optimism ;-) Now the splitting is

u = "x" * 100
v = "y"

and the wonderful trick above only applies to a trivial 1-character
string. The "hard part" is all in matching the u piece now, which I'll
leave as an exercise for the reader ;-)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HPMYTQSJ4IXUYUCZE2EYCIF34RTQDW7O/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Modified parser does not seem to work. What am I doing wrong?

[Thomas Wouters]

On Fri, 16 Oct 2020, 23:04 Stefano Borini,  wrote:

> Hello,
>
> I am trying to implement PEP-637, and I started modifying the parser
> and the grammar, but I don't know what I am missing.
>
> The PR is here
>
>
> https://github.com/python/cpython/compare/master...stefanoborini:PEP-637-implementation-attempt-1?expand=1
>
> It includes other stuff but the core is that I extended the Subscript
> in the asdl to accept the keyword args
>
> | Subscript(expr value, expr slice, keyword* keywords, expr_context ctx)
>
> which seems to work:
>
> >>> ast.parse('a[3]').body[0].value.keywords
> []
>
> I also added a few "productions" (I believe they are called like this,
> my compiler theory is very approximated), one of which is now
>
> primary[expr_ty]:
> 
> | a=primary '[' b=slices c=[',' k=kwargs {k}]']' {
> _Py_Subscript(a, b, c, Load, EXTRA) }
>
> I also tried with additional formats like
>
> | a=primary '[' b=slices c=kwargs ']' { _Py_Subscript(a, b, c,
> Load, EXTRA) }
>
> or even just
>
> | a=primary '[' c=kwargs ']' { _Py_Subscript(a, NULL, c, Load, EXTRA) }
>
> but I always get a SyntaxError:
>
> >>> ast.parse('a[k=3]').body[0].value.keywords
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "/Users/sbo/Work/Projects/stefanoborini/cpython/Lib/ast.py",
> line 50, in parse
> return compile(source, filename, mode, flags,
>   File "", line 1
> a[k=3]
>^
> SyntaxError: invalid syntax
>
> Note that I always recreated ast parser and pegen.c with 'make
> regen-all' and recompiled with make.
> I tried to enable debug and print the pegen.c debug, but it's a bit
> heavy and I could not immediately spot the issue. I suspect I am
> missing something somewhere.
>

Have you manually checked that the generated source in pegen.c contains
your changes? You should be able to see by just looking for calls to
_Py_Subscript, or other code snippets from the modified grammar. Or, you
can introduce an error in the grammar's code snippets and check that you
get the expected error. FWIW, I found I had to explicitly regenerate
pegen.c with 'make regen-pegen'; 'make regen-all' wasn't enough.


> Thanks
>
>
> --
> Kind regards,
>
> Stefano Borini
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/RZYDHYZPTPGLBX4VCR6HTYGJ3GL2CWIX/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DGQKPV2OHXG6RPZGM3NKZBMFI3IDWMA7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 638: Syntactic macros

[Dan Stromberg]

On Sat, Sep 26, 2020 at 5:11 AM Mark Shannon  wrote:

> Hi everyone,
>
> I've submitted my PEP on syntactic macros as PEP 638.
> https://www.python.org/dev/peps/pep-0638/
>

Speaking as a former C developer, why do "We need to let the community
develop their own extensions"?  What's insufficient about Python's current
extensibility?

The complexity of a language varies with the square of its feature count,
and adding macros intended for creation of domain-specific language
features  balloons the feature count. Granted, they don't much increase the
complexity of CPython, the language implementation, but the de facto
language then becomes more than CPython - potentially a lot more.

That is, making it easier to extend python means a proliferation of
extensions with little thought given to their long term viability or
cross-domain compatibility.

It's kind of like zsh vs. bash.  zsh has a smaller implementation, but a
larger _language_.  For this reason, I'm not terribly interested in zsh,
but I like bash.  On the other hand, bash has seen a proliferation of
unnecessary extensions in recent years, so I may jump ship to something
else someday - something with a smaller language, that isn't afraid of
fork+exec.

I've used m4 before as a macro system for Python+Cython, but I would never
consider suggesting that m4 should become part of Python itself.

IMO Perl is dying because of its exuberant design.  One of the most
important things a language designer has to do is say "no" sometimes.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AKKHSCE6IS3IW7MBJ3TP5253EXG5NWZO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Modified parser does not seem to work. What am I doing wrong?

[Pablo Galindo Salgado]

Hi Stefano,

One of the problems you have is that the rule for slices has a negative
lookahead for the comma:

 slices[expr_ty]:
| a=slice !',' { a }

IIRC the reason that is there is to allow "x[3,]" to be parsed.

Also, to allow "a[k=3]" you need to to create a rule that allows skipping
the "slices" part of the subscript and allow the 2nd argument of
_Py_Subscript  to be NULL. For example:

| a=t_primary '[' k=kwargs ']' !t_lookahead { _Py_Subscript(a, NULL, k,
Store, EXTRA) }

Regards,
Pablo

On Fri, 16 Oct 2020 at 22:07, Stefano Borini 
wrote:

> Hello,
>
> I am trying to implement PEP-637, and I started modifying the parser
> and the grammar, but I don't know what I am missing.
>
> The PR is here
>
>
> https://github.com/python/cpython/compare/master...stefanoborini:PEP-637-implementation-attempt-1?expand=1
>
> It includes other stuff but the core is that I extended the Subscript
> in the asdl to accept the keyword args
>
> | Subscript(expr value, expr slice, keyword* keywords, expr_context ctx)
>
> which seems to work:
>
> >>> ast.parse('a[3]').body[0].value.keywords
> []
>
> I also added a few "productions" (I believe they are called like this,
> my compiler theory is very approximated), one of which is now
>
> primary[expr_ty]:
> 
> | a=primary '[' b=slices c=[',' k=kwargs {k}]']' {
> _Py_Subscript(a, b, c, Load, EXTRA) }
>
> I also tried with additional formats like
>
> | a=primary '[' b=slices c=kwargs ']' { _Py_Subscript(a, b, c,
> Load, EXTRA) }
>
> or even just
>
> | a=primary '[' c=kwargs ']' { _Py_Subscript(a, NULL, c, Load, EXTRA) }
>
> but I always get a SyntaxError:
>
> >>> ast.parse('a[k=3]').body[0].value.keywords
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "/Users/sbo/Work/Projects/stefanoborini/cpython/Lib/ast.py",
> line 50, in parse
> return compile(source, filename, mode, flags,
>   File "", line 1
> a[k=3]
>^
> SyntaxError: invalid syntax
>
> Note that I always recreated ast parser and pegen.c with 'make
> regen-all' and recompiled with make.
> I tried to enable debug and print the pegen.c debug, but it's a bit
> heavy and I could not immediately spot the issue. I suspect I am
> missing something somewhere.
>
> Thanks
>
>
> --
> Kind regards,
>
> Stefano Borini
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/RZYDHYZPTPGLBX4VCR6HTYGJ3GL2CWIX/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JKEGBZW7WVBTNSLEM44AYIV6CWLSK5GO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Ideas for improving the contribution experience

[Tal Einat]

(Context: Continuing to prepare for the core dev sprint next week. Since
the sprint is near, *I'd greatly appreciate any quick comments, feedback
and ideas!*)

Following up my collection of past beginning contributor experiences, I've
collected these experiences in a dedicated GitHub repo[1] and written a
(subjective!) summary of main themes that I recognize in the stories, which
I've also included in the repo[2].

A "TL;DR" bullet list of those main themes:
* Slow/no responsiveness
* Long, slow process
* Hard to find where to contribute
* Mentorship helps a lot, but is scarce
* A lot to learn to get started
* It's intimidating

More specifically, something that has come up often is that maintaining
momentum for new contributors is crucial for them to become long-term
contributors. Most often, this comes up in relation to the first two
points: Suggestions or PRs are completely receive no attention at all
("ignored") or stop receiving attention at some point ("lost to the void").
Unfortunately, the probability of this is pretty high for any issue/PR, so
for a new contributor this is almost guaranteed to happen while working on
one of their first few contributions. I've seen this happen many times, and
have found that I have to personally follow promising contributors' work to
ensure that this doesn't happen to them. I've also seen contributors learn
to actively seek out core devs when these situations arise, which is often
a successful tactic, but shouldn't be necessary so often.

Now, this is in large part a result of the fact that us core devs are not a
very large group, made up almost entirely of volunteers working on this in
their spare time. Last I checked, the total amount of paid development time
dedicated to developing Python is less than 3 full-time (i.e. ~100 hours a
week).

The situation being problematic is clear enough that the PSF had concrete
plans to hire paid developers to review issues and PRs. However, those
plans have been put on hold indefinitely, since the PSF's funding has
shrunk dramatically since the COVID-19 outbreak (no PyCon!).

So, what can be done? Besides raising more funds (see a note on this
below), I think we can find ways to reduce how often issues/PRs become
"stalled". Here are some ideas:

1. *Generate reminders for reviewers when an issue or PR becomes "stalled'
due to them.* Personally, I've found that both b.p.o. and GitHub make it
relatively hard to remember to follow up on all of the many issues/PRs
you've taken part in reviewing. It takes considerable attention and
discipline to do so consistently, and reminders like these would have
helped me. Many (many!) times, all it took to get an issue/PR moving
forward (or closed) was a simple "ping?" comment.

2. *Generate reminders for contributors when an issue or PR becomes
"stalled" due to them.* Similar to the above, but I consider these separate.

3. *Advertise something like a "2-for-1" standing offer for reviews.* This
would give contributors an "official", acceptable way to get attention for
their issue/PR, other than "begging" for attention on a mailing list. There
are good ways for new contributors to be of significant help despite being
new to the project, such as checking whether old bugs are still relevant,
searching for duplicate issues, or applying old patches to the current code
and creating a PR. (This would be similar to Martin v. Löwis's 5-for-1
offer in 2012[3], which had little success but lead to some interesting
followup discussion[4]).

4. *Encourage core devs to dedicate some of their time to working through
issues/PRs which are "ignored" or "stalled".* This would require first
generating reliable lists of issues/PRs in such states. This could be in
various forms, such as predefined GitHub/b.p.o. queries, a dedicated
web-page, a periodic message similar to b.p.o.'s "weekly summary" email, or
dedicated tags/labels for issues/PRs. (Perhaps prioritize "stalled" over
"ignored".)

- Tal Einat


[1]: https://github.com/taleinat/python-contribution-feedback
[2]:
https://github.com/taleinat/python-contribution-feedback/blob/master/Takeaways%20-%20October%202020.md
[3]:
https://mail.python.org/archives/list/python-dev@python.org/message/7DLUN4Y7P77BSDW5YRWQQGVB3KVOY2M3/
[4]:
https://mail.python.org/archives/list/python-dev@python.org/thread/N4MMHXXOAL77UJLO264PCTCRJLAIKBC6/#N4MMHXXOAL77UJLO264PCTCRJLAIKBC6
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AYSE65DUJFMFY5F7GJZSZDD252UWQENE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 638: Syntactic macros

[Guido van Rossum]

Dima,

Do you have a link to "babel macros"? Searching for that brought up several
different things; not being a frequent JS user I don't know how to filter
these.

--Guido

On Wed, Oct 14, 2020 at 11:55 PM Dima Tisnek  wrote:

> My 2c as a Python user (mostly) and someone who dabbled in ES2020:
>
> The shouting syntax! does not sit well with me.
> The $hygenic is also cumbersome.
>
> To contrast, babel macros:
> * looks like regular code, without special syntax: existing tooling
> works, less mental strain
> * have access to call site environment, so not strictly hygienic(?):
> allow for greater expressive power
>
> I these the two points above really helped adopt babel macros in the
> js community and should, at the very least be seriously considered by
> the py community.
>
> Cheers,
> d.
>
> On Sat, 26 Sep 2020 at 21:16, Mark Shannon  wrote:
> >
> > Hi everyone,
> >
> > I've submitted my PEP on syntactic macros as PEP 638.
> > https://www.python.org/dev/peps/pep-0638/
> >
> > All comments and suggestions are welcome.
> >
> > Cheers,
> > Mark
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/U4C4XHNRC4SHS3TPZWCTY4SN4QU3TT6V/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/VEC7VWY5TJJGBXWFQUX3XO43SQAZ7FMR/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FF42I5Q3HZVOKOZC4NEEKJK5DXKJYJKS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 11: Drop support for AIX releases without dlopen

[Kevin Adler]

Interesting. Given that, shouldn't PEP 11 be updated with that change? Seems to 
me that PEP 11 only documents platforms with *official support*, so is AIX 
officially supported? The comment in the issue would indicate it is not 
officially supported, but it _is_ listed here: 
https://pythondev.readthedocs.io/platforms.html#python-platforms

Batuhan Taskaya wrote:
> As far as I am aware, we already dropped support for AIX 5.3<=.  See 
> https://bugs.python.org/issue40680 for details.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KD4FVVQAAT5GCF6R3UXGPAEURWN3QUN6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 11: Drop support for AIX releases without dlopen

[Batuhan Taskaya]

As far as I am aware, we already dropped support for AIX 5.3<=.  See 
https://bugs.python.org/issue40680 for details.


On 16.10.2020 23:15, Kevin Adler wrote:

Python has supported using dynload_shlib (using dlopen) on AIX since 
https://github.com/python/cpython/commit/c19c5a62aef7dce0e8147655b0d2f087965fae75
 in 2003. This is also about the time that AIX 4.3 went out of support, which 
is believed to be the AIX release that added support for dlopen. Considering it 
is now 20ish years later and in this time, every supported AIX release has had 
dlopen support, I suspect nobody has used or tested this code path in quite 
some time. I propose removing this support under PEP 11.

For reference, I opened https://bugs.python.org/issue42030 and changes were 
merged in https://github.com/python/cpython/pull/22717. I opened a PR to update 
PEP 11 in https://github.com/python/peps/issues/1653, but was told this needed 
discussion here. Sorry if this didn't quite follow the process correctly, I 
thought I had things in the right order, but I guess not. Also apologies if 
this is a duplicate, but my first message appears to have been lost/eaten 
(possibly a first-time moderation queue?).
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7QVLHW36FBZBEHSEPNVXAKXGUQHP3WX5/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FYZOH3K7C5KK4CTSAFO5FEWEZRJADHMG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Modified parser does not seem to work. What am I doing wrong?

[Stefano Borini]

Hello,

I am trying to implement PEP-637, and I started modifying the parser
and the grammar, but I don't know what I am missing.

The PR is here

https://github.com/python/cpython/compare/master...stefanoborini:PEP-637-implementation-attempt-1?expand=1

It includes other stuff but the core is that I extended the Subscript
in the asdl to accept the keyword args

| Subscript(expr value, expr slice, keyword* keywords, expr_context ctx)

which seems to work:

>>> ast.parse('a[3]').body[0].value.keywords
[]

I also added a few "productions" (I believe they are called like this,
my compiler theory is very approximated), one of which is now

primary[expr_ty]:

| a=primary '[' b=slices c=[',' k=kwargs {k}]']' {
_Py_Subscript(a, b, c, Load, EXTRA) }

I also tried with additional formats like

| a=primary '[' b=slices c=kwargs ']' { _Py_Subscript(a, b, c,
Load, EXTRA) }

or even just

| a=primary '[' c=kwargs ']' { _Py_Subscript(a, NULL, c, Load, EXTRA) }

but I always get a SyntaxError:

>>> ast.parse('a[k=3]').body[0].value.keywords
Traceback (most recent call last):
  File "", line 1, in 
  File "/Users/sbo/Work/Projects/stefanoborini/cpython/Lib/ast.py",
line 50, in parse
return compile(source, filename, mode, flags,
  File "", line 1
a[k=3]
   ^
SyntaxError: invalid syntax

Note that I always recreated ast parser and pegen.c with 'make
regen-all' and recompiled with make.
I tried to enable debug and print the pegen.c debug, but it's a bit
heavy and I could not immediately spot the issue. I suspect I am
missing something somewhere.

Thanks


-- 
Kind regards,

Stefano Borini
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RZYDHYZPTPGLBX4VCR6HTYGJ3GL2CWIX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] PEP 11: Drop support for AIX releases without dlopen

[Kevin Adler]

Python has supported using dynload_shlib (using dlopen) on AIX since 
https://github.com/python/cpython/commit/c19c5a62aef7dce0e8147655b0d2f087965fae75
 in 2003. This is also about the time that AIX 4.3 went out of support, which 
is believed to be the AIX release that added support for dlopen. Considering it 
is now 20ish years later and in this time, every supported AIX release has had 
dlopen support, I suspect nobody has used or tested this code path in quite 
some time. I propose removing this support under PEP 11.

For reference, I opened https://bugs.python.org/issue42030 and changes were 
merged in https://github.com/python/cpython/pull/22717. I opened a PR to update 
PEP 11 in https://github.com/python/peps/issues/1653, but was told this needed 
discussion here. Sorry if this didn't quite follow the process correctly, I 
thought I had things in the right order, but I guess not. Also apologies if 
this is a duplicate, but my first message appears to have been lost/eaten 
(possibly a first-time moderation queue?).
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7QVLHW36FBZBEHSEPNVXAKXGUQHP3WX5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Drop support for AIX releases without dlopen

[kadler]

Python has supported using dynload_shlib (using dlopen) on AIX since 
https://github.com/python/cpython/commit/c19c5a62aef7dce0e8147655b0d2f087965fae75
 in 2003. Considering this is now 20 years later and all supported AIX versions 
support dlopen, I suspect nobody has used or tested this code path in quite 
some time. I propose removing this support under PEP 11.

For reference, I opened https://bugs.python.org/issue42030 and changes were 
merged in https://github.com/python/cpython/pull/22717. I opened a PR to update 
PEP 11 in https://github.com/python/peps/issues/1653, but was told this needed 
discussion here. Sorry if this didn't quite follow the process correctly, I 
thought I had things in the right order, but I guess not.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5TQQVERJDL6GU2OZBOBJM7RPWY7MAJBJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Remove module's __version__ attributes in the stdlib

[Brett Cannon]

On Fri, Oct 16, 2020 at 3:08 AM Steve Holden  wrote:

> Since some code clearly accesses __version__, would it make sense to equip
> all stdlib modules with a __version__ property that returned the Python
> version, suitably prefixed?
>

That's another way to go, but I don't think that really provides more
accurate information specifically. If we were going to go that way then I
would advocate making __version__ a PEP-ified thing to get the whole
community to buy in (but I don't know if that's useful to begin with).

-Brett


>
> Kind regards,
> Steve
>
>
> On Fri, Oct 16, 2020 at 5:28 AM Karthikeyan  wrote:
>
>> On Fri, Oct 16, 2020, 12:45 AM Serhiy Storchaka 
>> wrote:
>>
>>> 14.10.20 20:56, Brett Cannon пише:
>>> > I think if the project is not maintained externally and thus synced
>>> into
>>> > the stdlib we can drop the attributes.
>>>
>>> I have found only one exception. decimal.__version__ refers to the
>>> version of the external specification which was not changed since 2009.
>>> I think it should be kept, although it might be better to use different
>>> name for it (like "spec_version").
>>>
>>> I do not know about any current projects maintained externally and
>>> synced into the stdlib. simplejson and ElementTree are too different now
>>> from the stdlib versions. Some features flow in both directions, but
>>> selectively on case by case basis, not as full sync. External argparse
>>> is outdated now.
>>>
>> I guess zipp that is maintained externally has code adopted into
>> zipfile.ZipPath regularly : https://github.com/jaraco/zipp
>>
>> __version__ was removed from mock and it broke a package in fedora. The
>> PR has a discussion and also links to the bpo to remove __version__ from
>> all of stdlib : https://github.com/python/cpython/pull/17977
>>
>> I am also in favor of removing since it causes confusion when the package
>> is not maintained externally n synced into stdlib.
>>
>> Thanks
>>
>> ___
>>> Python-Dev mailing list -- python-dev@python.org
>>> To unsubscribe send an email to python-dev-le...@python.org
>>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/python-dev@python.org/message/MIEMWWC5W2WKV25WTARXACQOIUBUUSLS/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/3NY5JIFUP5674Q3FR2DOMLXGBE6D4XJD/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/JZ4JVMYROUAONIZG2VQGKI6XCE4MJBDQ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JNZHANDMQCYNF65RUOE3XKEEAPWYE54X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2020-10-09 - 2020-10-16)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open7707 (+17)
  closed 46082 (+53)
  total  53789 (+70)

Open issues with patches: 3125 


Issues opened (46)
==

#41987: singledispatchmethod raises an error when relying on a forward
https://bugs.python.org/issue41987  opened by glyph

#41988: No hyphen in broken up word in documentation
https://bugs.python.org/issue41988  opened by cryvate

#41989: htmlparser unclosed script tag causes data loss
https://bugs.python.org/issue41989  opened by waylan

#41990: venv module clashes with pip --user ... improve coordination.
https://bugs.python.org/issue41990  opened by bernd.wechner

#41994: Refcount issues in import
https://bugs.python.org/issue41994  opened by serhiy.storchaka

#41995: five possible Null Pointer Dereference bugs.
https://bugs.python.org/issue41995  opened by brightest3379

#41999: imaplib Time2Internaldate crashing with time.struct_time suppl
https://bugs.python.org/issue41999  opened by sajicek

#42001: Deprecate `typing.io` Wrapper Namespace
https://bugs.python.org/issue42001  opened by rohitkg98

#42004: Allow uploading files with SimpleHTTPRequestHandler
https://bugs.python.org/issue42004  opened by lufte

#42005: profile/cProfile CLI should catch BrokenPipeError
https://bugs.python.org/issue42005  opened by zmwangx

#42006: Stop using PyDict_GetItem, PyDict_GetItemString and _PyDict_Ge
https://bugs.python.org/issue42006  opened by serhiy.storchaka

#42008: Internal Random class calling seed() with incorrect argument
https://bugs.python.org/issue42008  opened by rhettinger

#42009: Unable to compile with message compiler due to source order
https://bugs.python.org/issue42009  opened by chrisburr

#42010: Generic types accept indexing/subscripting, causing confusion
https://bugs.python.org/issue42010  opened by kj

#42012: typing support in wsgiref
https://bugs.python.org/issue42012  opened by srittau

#42013: venv on Windows with symlinks is broken if invoked with -I
https://bugs.python.org/issue42013  opened by gaborjbernat

#42014: shutil.rmtree calls onerror with different function than faile
https://bugs.python.org/issue42014  opened by nijel

#42016: Add ksh to venv examples
https://bugs.python.org/issue42016  opened by fibonacci

#42018: winreg SetValue(Ex) should mention integer as an acceptable va
https://bugs.python.org/issue42018  opened by kwojniak_box

#42019: Override MagicMock special methods
https://bugs.python.org/issue42019  opened by Indy

#42022: Allow ensurepip to work without bundled wheels
https://bugs.python.org/issue42022  opened by FFY00

#42023: Argparse: Add a "display" arg
https://bugs.python.org/issue42023  opened by ThatXliner

#42024: Exception ignored in: https://bugs.python.org/issue42024  opened by jayesh007

#42025: zoneinfo: wrong time difference across transition when tzinfo 
https://bugs.python.org/issue42025  opened by d.grellscheid

#42027: /passive run of Windows installer fail silently on Win7
https://bugs.python.org/issue42027  opened by colin-b

#42028: Regression in mimetypes for image/bmp
https://bugs.python.org/issue42028  opened by xpdseth

#42032: Setting PYTHONPYCACHEPREFIX using ~ (tilde) creates a "~" fold
https://bugs.python.org/issue42032  opened by vkbo

#42033: Seemingly unnecessary complexification of foo(**kw)
https://bugs.python.org/issue42033  opened by xmorel

#42034: Unchecked return in Objects/typeobject.c and possible uninitia
https://bugs.python.org/issue42034  opened by monocle-ai

#42035: [C API] PyType_GetSlot cannot get tp_name
https://bugs.python.org/issue42035  opened by fancitron

#42036: Unchecked return in Modules/posixmodule.c from multiple functi
https://bugs.python.org/issue42036  opened by monocle-ai

#42037: Documentation confusion in CookieJar functions
https://bugs.python.org/issue42037  opened by markus

#42038: Tracemalloc's format() doc contradictory
https://bugs.python.org/issue42038  opened by lisroach

#42040: Change IDLE 'more information' splash line, change doc name
https://bugs.python.org/issue42040  opened by terry.reedy

#42041: venv subprocess call to python resolves to wrong interpreter
https://bugs.python.org/issue42041  opened by keller00

#42042: docs.python.org for 3.10 missing the new refererence record.
https://bugs.python.org/issue42042  opened by corona10

#42043: zipfile.Path should support inheritance
https://bugs.python.org/issue42043  opened by conchylicultor

#42044: Running Python in unbuffered mode may not write all contents t
https://bugs.python.org/issue42044  opened by fabioz

#42045: Add support to async code to pdb
https://bugs.python.org/issue42045  opened by hack.augusto

#42046: Unable to write to file without elevated privileges
https://bugs.python.org/issue42046  opened by john_miller

#42047: DragonFlyBSD thread native id suppor

[Python-Dev] Re: Changing Python's string search algorithms

[Tim Peters]

[Marco Sulla]
> Excuse me if I intrude in an algorithm that I have not understood, but
> the new optimization can be applied to regexps too?

The algorithm is limited to searching for fixed strings.

However, _part_ of our regexp implementation (the bit that looks ahead
for a fixed string) will inherit it by magic.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Z4R6VUAOXMH2D5NDGWZX52BLU6F7XNXE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Changing Python's string search algorithms

[Marco Sulla]

Excuse me if I intrude in an algorithm that I have not understood, but
the new optimization can be applied to regexps too?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UAYHXDYVG5IW4NDV3TSHTMSBKJKRFVEM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Remove module's __version__ attributes in the stdlib

[Steve Holden]

Since some code clearly accesses __version__, would it make sense to equip
all stdlib modules with a __version__ property that returned the Python
version, suitably prefixed?

Kind regards,
Steve


On Fri, Oct 16, 2020 at 5:28 AM Karthikeyan  wrote:

> On Fri, Oct 16, 2020, 12:45 AM Serhiy Storchaka 
> wrote:
>
>> 14.10.20 20:56, Brett Cannon пише:
>> > I think if the project is not maintained externally and thus synced into
>> > the stdlib we can drop the attributes.
>>
>> I have found only one exception. decimal.__version__ refers to the
>> version of the external specification which was not changed since 2009.
>> I think it should be kept, although it might be better to use different
>> name for it (like "spec_version").
>>
>> I do not know about any current projects maintained externally and
>> synced into the stdlib. simplejson and ElementTree are too different now
>> from the stdlib versions. Some features flow in both directions, but
>> selectively on case by case basis, not as full sync. External argparse
>> is outdated now.
>>
> I guess zipp that is maintained externally has code adopted into
> zipfile.ZipPath regularly : https://github.com/jaraco/zipp
>
> __version__ was removed from mock and it broke a package in fedora. The PR
> has a discussion and also links to the bpo to remove __version__ from all
> of stdlib : https://github.com/python/cpython/pull/17977
>
> I am also in favor of removing since it causes confusion when the package
> is not maintained externally n synced into stdlib.
>
> Thanks
>
> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/MIEMWWC5W2WKV25WTARXACQOIUBUUSLS/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/3NY5JIFUP5674Q3FR2DOMLXGBE6D4XJD/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JZ4JVMYROUAONIZG2VQGKI6XCE4MJBDQ/
Code of Conduct: http://python.org/psf/codeofconduct/