Re: [Python-ideas] Built-in parsing library

2019-04-09 Thread Barry Scott
Nam, I'm not so sure that a "universal parsing library" is possible for the stdlib. I think one way you could find out what the requirements are is to refactor at least 2 of the existing stdlib modules that you have identified as needing a better parser. Did you find that you could use the

Re: [Python-ideas] Built-in parsing library

2019-04-09 Thread Nam Nguyen
On Mon, Apr 8, 2019 at 7:59 AM Christopher Barker wrote: > > > On Mon, Apr 8, 2019 at 12:02 AM Paul Moore wrote: > >> I would expect that the only reasonable way of getting a parsing >> library in the stdlib would be to propose an established one from PyPI >> to be moved into the stdlib > > >

Re: [Python-ideas] Built-in parsing library

2019-04-08 Thread Ai mu
@DavidMertz Each one of them takes a dramatically different approach to the defining a grammar they work more towards implementing well known standards like the BNF. well internally they might work different to parse etc. Abdur-Rahmaan Janhangeer Mauritius

Re: [Python-ideas] Built-in parsing library

2019-04-08 Thread Christopher Barker
On Mon, Apr 8, 2019 at 12:02 AM Paul Moore wrote: > I would expect that the only reasonable way of getting a parsing > library in the stdlib would be to propose an established one from PyPI > to be moved into the stdlib Absolutely -- unlike some proposals, a stand-alone parsing lib could very

Re: [Python-ideas] Built-in parsing library

2019-04-08 Thread Paul Moore
On Mon, 8 Apr 2019 at 02:54, Nam Nguyen wrote: > Back to my original goal, I've gathered that there is some interest in having > a more general parser library in the stdlib. "Some", but not "much". Should I > start out with a straw proposal so that we can hash it out further? I would expect

Re: [Python-ideas] Built-in parsing library

2019-04-07 Thread Nam Nguyen
On Mon, Apr 1, 2019 at 3:13 PM Terry Reedy wrote: > On 4/1/2019 1:14 AM, Guido van Rossum wrote: > > We do have a parser generator in the standard library: > > https://github.com/python/cpython/tree/master/Lib/lib2to3/pgen2 > > It is effectively undocumented and by inference discouraged from

Re: [Python-ideas] Built-in parsing library

2019-04-01 Thread Terry Reedy
On 4/1/2019 1:14 AM, Guido van Rossum wrote: We do have a parser generator in the standard library: https://github.com/python/cpython/tree/master/Lib/lib2to3/pgen2 It is effectively undocumented and by inference discouraged from use. The entry for lib2to3 in the 2to3 doc:

Re: [Python-ideas] Built-in parsing library

2019-04-01 Thread Nam Nguyen
Sure! Same examples mentioned in Victor's https://vstinner.github.io/tag/security.html could have been fixed by having a more proper parser. This one that I helped author was also a parsing issue. https://python-security.readthedocs.io/vuln/bpo-30500_urllib_connects_to_a_wrong_host.html Thanks

Re: [Python-ideas] Built-in parsing library

2019-04-01 Thread Nathaniel Smith
On Sun, Mar 31, 2019 at 9:17 PM Nam Nguyen wrote: > Installing a package out of stdlib does not solve the problem that motivated > this thread. The libraries included in the stdlib can't use those parsers. Can you be more specific about exactly which code in the stdlib you think should be

Re: [Python-ideas] Built-in parsing library

2019-04-01 Thread Stephen J. Turnbull
David Mertz writes: > OK, I'll acknowledge my comment might have overstated the bar to overcome. > A parser added to the standard library doesn't need to be perfect for > everyone. But adding to stdlib *does* provide a kind of endorsement of the > right default way to go about things.

Re: [Python-ideas] Built-in parsing library

2019-04-01 Thread Stephen J. Turnbull
David Mertz writes: > While I can imagine proposing one for inclusion in the standard > library, you'd have to choose one (or write a new one) and explain > why that one is better for everyone (or at least a better starting > point) than all the others are. In principle, no, one just needs

Re: [Python-ideas] Built-in parsing library

2019-03-31 Thread James Lu
Stack-based LL(1) push down automata can be implemented by hand, indeed isn’t that that a textmateLanguage file is? There’s also the option of using Iro to generate a tmLanguage. ___ Python-ideas mailing list Python-ideas@python.org

Re: [Python-ideas] Built-in parsing library

2019-03-31 Thread Guido van Rossum
We do have a parser generator in the standard library: https://github.com/python/cpython/tree/master/Lib/lib2to3/pgen2 On Sun, Mar 31, 2019 at 9:17 PM Nam Nguyen wrote: > On Sun, Mar 31, 2019 at 12:13 PM David Mertz wrote: > >> I just found this nice summary. It's not complete, but it looks

Re: [Python-ideas] Built-in parsing library

2019-03-31 Thread Nam Nguyen
On Sun, Mar 31, 2019 at 12:13 PM David Mertz wrote: > I just found this nice summary. It's not complete, but it looks well > written. https://tomassetti.me/parsing-in-python/ > > On Sun, Mar 31, 2019, 3:09 PM David Mertz wrote: > >> There are about a half dozen widely used parsing libraries for

Re: [Python-ideas] Built-in parsing library

2019-03-31 Thread David Mertz
I just found this nice summary. It's not complete, but it looks well written. https://tomassetti.me/parsing-in-python/ On Sun, Mar 31, 2019, 3:09 PM David Mertz wrote: > There are about a half dozen widely used parsing libraries for Python. > Each one of them takes a dramatically different

Re: [Python-ideas] Built-in parsing library

2019-03-31 Thread Nick Timkovich
What does it mean to be a universal parser? In my mind, to be universal you should be able to parse anything, so you'd need something as versatile as any Turing language, so one could stick with the one we already have (Python). I'm vaguely aware of levels of grammar (regular, context-free?,

[Python-ideas] Built-in parsing library

2019-03-30 Thread Nam Nguyen
Hello list, What do you think of a universal parsing library in the stdlib mainly for use by other libraries in the stdlib? Through out the years we have had many issues with protocol parsing. Some have even introduced security bugs. The main cause of these issues is the use of simple regular