Re: [Python-Dev] C99

2016-06-04 Thread Martin Panter
On 4 June 2016 at 06:11, Benjamin Peterson  wrote:
> PEP 7 requires CPython to use C code conforming to the venerable C89
> standard. Traditionally, we've been stuck with C89 due to poor C support
> in MSVC. However, MSVC 2013 and 2015 implement the key features of C99.
> C99 does not offer anything earth-shattering; here are the features I
> think we'd find most interesting:
> - Variable declarations can be on any line: removes possibly the most
> annoying limitation of C89.
> - Inline functions: We can make Py_DECREF and Py_INCREF inline functions
> rather than unpleasant macros.
> - C++-style line comments: Not an killer feature but commonly used.
> - Booleans

My most-missed C99 feature would be designated initializers. Does MSVC
support them? It might allow you to do away with those giant pasted
slot tables, and just write the slots you need:

PyTypeObject PyUnicodeIter_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
.tp_name = "str_iterator",
.tp_basicsize = sizeof(unicodeiterobject),
.tp_dealloc = unicodeiter_dealloc,
.tp_getattro = PyObject_GenericGetAttr,
.tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,
.tp_traverse = unicodeiter_traverse,
.tp_iter = PyObject_SelfIter,
.tp_iternext = unicodeiter_next,
.tp_methods = unicodeiter_methods,
};

> So, what say you to updating PEP 7 to allow C99 features for Python 3.6
> (in so much as GCC and MSVC support them)?

Sounds good for features that are well-supported by compilers that
people use. (Are there other compilers used than just GCC and MSVC?)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Improving the bytecode

2016-06-04 Thread Serhiy Storchaka
Following the converting 8-bit bytecode to 16-bit bytecode (wordcode), 
there are other issues for improving the bytecode.


1. http://bugs.python.org/issue27129
Make the bytecode more 16-bit oriented.

2. http://bugs.python.org/issue27140
Add new opcode BUILD_CONST_KEY_MAP for building a dict with constant 
keys. This optimize the common case and especially helpful for two 
following issues (creating and calling functions).


3. http://bugs.python.org/issue27095
Simplify MAKE_FUNCTION/MAKE_CLOSURE. Instead packing three numbers in 
oparg the new MAKE_FUNCTION takes built tuples and dicts from the stack. 
MAKE_FUNCTION and MAKE_CLOSURE are merged in the single opcode.


4. http://bugs.python.org/issue27213
Rework CALL_FUNCTION* opcodes. Replace four existing opcodes with three 
simpler and more efficient opcodes.


5. http://bugs.python.org/issue27127
Rework the for loop implementation.

6. http://bugs.python.org/issue17611
Move unwinding of stack for "pseudo exceptions" from interpreter to 
compiler.


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C99

2016-06-04 Thread Sebastian Krause
Martin Panter  wrote:
>> So, what say you to updating PEP 7 to allow C99 features for Python 3.6
>> (in so much as GCC and MSVC support them)?
>
> Sounds good for features that are well-supported by compilers that
> people use. (Are there other compilers used than just GCC and MSVC?)

clang on OS X, but it supports pretty much everything that GCC
supports as well.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Improving the bytecode

2016-06-04 Thread Brett Cannon
It's not on the list but I'm hoping to convince Dino to work on END_FINALLY
to be a bit more sane.

On Sat, Jun 4, 2016, 01:17 Serhiy Storchaka  wrote:

> Following the converting 8-bit bytecode to 16-bit bytecode (wordcode),
> there are other issues for improving the bytecode.
>
> 1. http://bugs.python.org/issue27129
> Make the bytecode more 16-bit oriented.
>
> 2. http://bugs.python.org/issue27140
> Add new opcode BUILD_CONST_KEY_MAP for building a dict with constant
> keys. This optimize the common case and especially helpful for two
> following issues (creating and calling functions).
>
> 3. http://bugs.python.org/issue27095
> Simplify MAKE_FUNCTION/MAKE_CLOSURE. Instead packing three numbers in
> oparg the new MAKE_FUNCTION takes built tuples and dicts from the stack.
> MAKE_FUNCTION and MAKE_CLOSURE are merged in the single opcode.
>
> 4. http://bugs.python.org/issue27213
> Rework CALL_FUNCTION* opcodes. Replace four existing opcodes with three
> simpler and more efficient opcodes.
>
> 5. http://bugs.python.org/issue27127
> Rework the for loop implementation.
>
> 6. http://bugs.python.org/issue17611
> Move unwinding of stack for "pseudo exceptions" from interpreter to
> compiler.
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Improving the bytecode

2016-06-04 Thread Eric Snow
You should get in touch with Mark Shannon, while you're working on
ceval.  He has some definite improvements that can be made to the eval
loop.

-eric

On Sat, Jun 4, 2016 at 2:08 AM, Serhiy Storchaka  wrote:
> Following the converting 8-bit bytecode to 16-bit bytecode (wordcode), there
> are other issues for improving the bytecode.
>
> 1. http://bugs.python.org/issue27129
> Make the bytecode more 16-bit oriented.
>
> 2. http://bugs.python.org/issue27140
> Add new opcode BUILD_CONST_KEY_MAP for building a dict with constant keys.
> This optimize the common case and especially helpful for two following
> issues (creating and calling functions).
>
> 3. http://bugs.python.org/issue27095
> Simplify MAKE_FUNCTION/MAKE_CLOSURE. Instead packing three numbers in oparg
> the new MAKE_FUNCTION takes built tuples and dicts from the stack.
> MAKE_FUNCTION and MAKE_CLOSURE are merged in the single opcode.
>
> 4. http://bugs.python.org/issue27213
> Rework CALL_FUNCTION* opcodes. Replace four existing opcodes with three
> simpler and more efficient opcodes.
>
> 5. http://bugs.python.org/issue27127
> Rework the for loop implementation.
>
> 6. http://bugs.python.org/issue17611
> Move unwinding of stack for "pseudo exceptions" from interpreter to
> compiler.
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail.com
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C99

2016-06-04 Thread Christian Heimes
On 2016-06-03 23:11, Benjamin Peterson wrote:
> PEP 7 requires CPython to use C code conforming to the venerable C89
> standard. Traditionally, we've been stuck with C89 due to poor C support
> in MSVC. However, MSVC 2013 and 2015 implement the key features of C99.
> C99 does not offer anything earth-shattering; here are the features I
> think we'd find most interesting:
> - Variable declarations can be on any line: removes possibly the most
> annoying limitation of C89.
> - Inline functions: We can make Py_DECREF and Py_INCREF inline functions
> rather than unpleasant macros.
> - C++-style line comments: Not an killer feature but commonly used.
> - Booleans
> In summary, some niceties that would make CPython hacking a little more
> fun.
> 
> So, what say you to updating PEP 7 to allow C99 features for Python 3.6
> (in so much as GCC and MSVC support them)?

+1

- We never officially deprecated C89 platforms withou 64 bit integers in
PEP 7. Victor's changes to pytime.h implies support for uint64_t and
int64_t. C99 has mandatory long long int support.

- If we also drop Solaris Studio C compiler support, we can replace
header guards (e.g. #ifndef Py_PYTHON_H) with #pragma once

Christian



___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C99

2016-06-04 Thread Guido van Rossum
Funny. Just two weeks ago I was helping someone who discovered a
compiler that doesn't support the new relaxed variable declaration
rules. I think it was on Windows. Maybe this move is a little too
aggressively deprecating older Windows compilers?

On Sat, Jun 4, 2016 at 10:27 AM, Christian Heimes  wrote:
> On 2016-06-03 23:11, Benjamin Peterson wrote:
>> PEP 7 requires CPython to use C code conforming to the venerable C89
>> standard. Traditionally, we've been stuck with C89 due to poor C support
>> in MSVC. However, MSVC 2013 and 2015 implement the key features of C99.
>> C99 does not offer anything earth-shattering; here are the features I
>> think we'd find most interesting:
>> - Variable declarations can be on any line: removes possibly the most
>> annoying limitation of C89.
>> - Inline functions: We can make Py_DECREF and Py_INCREF inline functions
>> rather than unpleasant macros.
>> - C++-style line comments: Not an killer feature but commonly used.
>> - Booleans
>> In summary, some niceties that would make CPython hacking a little more
>> fun.
>>
>> So, what say you to updating PEP 7 to allow C99 features for Python 3.6
>> (in so much as GCC and MSVC support them)?
>
> +1
>
> - We never officially deprecated C89 platforms withou 64 bit integers in
> PEP 7. Victor's changes to pytime.h implies support for uint64_t and
> int64_t. C99 has mandatory long long int support.
>
> - If we also drop Solaris Studio C compiler support, we can replace
> header guards (e.g. #ifndef Py_PYTHON_H) with #pragma once
>
> Christian
>
>
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/guido%40python.org



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C99

2016-06-04 Thread Dino Viehland via Python-Dev


Martin wrote:
> On 4 June 2016 at 06:11, Benjamin Peterson  wrote:
> > PEP 7 requires CPython to use C code conforming to the venerable C89
> > standard. Traditionally, we've been stuck with C89 due to poor C
> > support in MSVC. However, MSVC 2013 and 2015 implement the key
> features of C99.
> > C99 does not offer anything earth-shattering; here are the features I
> > think we'd find most interesting:
> > - Variable declarations can be on any line: removes possibly the most
> > annoying limitation of C89.
> > - Inline functions: We can make Py_DECREF and Py_INCREF inline
> > functions rather than unpleasant macros.
> > - C++-style line comments: Not an killer feature but commonly used.
> > - Booleans
> 
> My most-missed C99 feature would be designated initializers. Does MSVC
> support them? It might allow you to do away with those giant pasted slot
> tables, and just write the slots you need:
> 
> PyTypeObject PyUnicodeIter_Type = {
> PyVarObject_HEAD_INIT(&PyType_Type, 0)
> .tp_name = "str_iterator",
> .tp_basicsize = sizeof(unicodeiterobject),
> .tp_dealloc = unicodeiter_dealloc,
> .tp_getattro = PyObject_GenericGetAttr,
> .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,
> .tp_traverse = unicodeiter_traverse,
> .tp_iter = PyObject_SelfIter,
> .tp_iternext = unicodeiter_next,
> .tp_methods = unicodeiter_methods,
> };

I checked and VC++ does actually support this, and it looks like they support 
// comments  as well.  I don't think it fully supports all of the C99 features 
- it appears 
They just cherry picked some stuff.  The C99 standard library does appear to be 
fully
supported with the exception of tgmath.h.


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C99

2016-06-04 Thread Benjamin Peterson


On Sat, Jun 4, 2016, at 11:32, Dino Viehland wrote:
> 
> 
> Martin wrote:
> > On 4 June 2016 at 06:11, Benjamin Peterson  wrote:
> > > PEP 7 requires CPython to use C code conforming to the venerable C89
> > > standard. Traditionally, we've been stuck with C89 due to poor C
> > > support in MSVC. However, MSVC 2013 and 2015 implement the key
> > features of C99.
> > > C99 does not offer anything earth-shattering; here are the features I
> > > think we'd find most interesting:
> > > - Variable declarations can be on any line: removes possibly the most
> > > annoying limitation of C89.
> > > - Inline functions: We can make Py_DECREF and Py_INCREF inline
> > > functions rather than unpleasant macros.
> > > - C++-style line comments: Not an killer feature but commonly used.
> > > - Booleans
> > 
> > My most-missed C99 feature would be designated initializers. Does MSVC
> > support them? It might allow you to do away with those giant pasted slot
> > tables, and just write the slots you need:
> > 
> > PyTypeObject PyUnicodeIter_Type = {
> > PyVarObject_HEAD_INIT(&PyType_Type, 0)
> > .tp_name = "str_iterator",
> > .tp_basicsize = sizeof(unicodeiterobject),
> > .tp_dealloc = unicodeiter_dealloc,
> > .tp_getattro = PyObject_GenericGetAttr,
> > .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,
> > .tp_traverse = unicodeiter_traverse,
> > .tp_iter = PyObject_SelfIter,
> > .tp_iternext = unicodeiter_next,
> > .tp_methods = unicodeiter_methods,
> > };
> 
> I checked and VC++ does actually support this, and it looks like they
> support 
> // comments  as well.  I don't think it fully supports all of the C99
> features - it appears 
> They just cherry picked some stuff.  The C99 standard library does appear
> to be fully
> supported with the exception of tgmath.h.

Are the C99 features VS++ supports documented anywhere? I couldn't find
any list.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C99

2016-06-04 Thread Christian Heimes
On 2016-06-04 10:47, Guido van Rossum wrote:
> Funny. Just two weeks ago I was helping someone who discovered a
> compiler that doesn't support the new relaxed variable declaration
> rules. I think it was on Windows. Maybe this move is a little too
> aggressively deprecating older Windows compilers?

Yes, it's not support in VS 2012 and 2008 for Python 3.4 and older. New
C99 features are available in VS 2013,
https://blogs.msdn.microsoft.com/vcblog/2013/06/28/c1114-stl-features-fixes-and-breaking-changes-in-vs-2013/


Python 3.5+ requires VS 2015 anyway. Traditionally we tried to keep
backwards compatibility with older compiler versions. The new features
are tempting enough to deprecate compiler versions that have been
released more than five years ago.

Christian
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C99

2016-06-04 Thread Guido van Rossum
As long as we don't require extension module authors to use them --
they may have their own compatibility requirements.

On Sat, Jun 4, 2016 at 11:50 AM, Christian Heimes  wrote:
> On 2016-06-04 10:47, Guido van Rossum wrote:
>> Funny. Just two weeks ago I was helping someone who discovered a
>> compiler that doesn't support the new relaxed variable declaration
>> rules. I think it was on Windows. Maybe this move is a little too
>> aggressively deprecating older Windows compilers?
>
> Yes, it's not support in VS 2012 and 2008 for Python 3.4 and older. New
> C99 features are available in VS 2013,
> https://blogs.msdn.microsoft.com/vcblog/2013/06/28/c1114-stl-features-fixes-and-breaking-changes-in-vs-2013/
>
>
> Python 3.5+ requires VS 2015 anyway. Traditionally we tried to keep
> backwards compatibility with older compiler versions. The new features
> are tempting enough to deprecate compiler versions that have been
> released more than five years ago.
>
> Christian



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C99

2016-06-04 Thread Christian Heimes
On 2016-06-04 11:59, Guido van Rossum wrote:
> As long as we don't require extension module authors to use them --
> they may have their own compatibility requirements.

On Windows extension modules must be compiled with a specific version of
MSVC any way. For Python 3.6 VS 2015 or newer is a hard requirement.

We kept the old compiler directories around for embedders.

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C99

2016-06-04 Thread Guido van Rossum
I'm talking about 3rd party extensions. Those may require source
compatibility with older Python versions. All I'm asking for is to not
require source-level use of C99 features. Of course requiring a
specific compiler to work with specific CPython versions is fine.

On Sat, Jun 4, 2016 at 12:05 PM, Christian Heimes  wrote:
> On 2016-06-04 11:59, Guido van Rossum wrote:
>> As long as we don't require extension module authors to use them --
>> they may have their own compatibility requirements.
>
> On Windows extension modules must be compiled with a specific version of
> MSVC any way. For Python 3.6 VS 2015 or newer is a hard requirement.
>
> We kept the old compiler directories around for embedders.
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C99

2016-06-04 Thread Christian Heimes
On 2016-06-04 12:07, Guido van Rossum wrote:
> I'm talking about 3rd party extensions. Those may require source
> compatibility with older Python versions. All I'm asking for is to not
> require source-level use of C99 features. Of course requiring a
> specific compiler to work with specific CPython versions is fine.

Ah, the other way around. Yes, that makes a lot of sense.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C99

2016-06-04 Thread Larry Hastings



On 06/03/2016 11:11 PM, Benjamin Peterson wrote:

So, what say you to updating PEP 7 to allow C99 features for Python 3.6
(in so much as GCC and MSVC support them)?


+1

Clearly it'll be 3.5+ only, and clearly it'll be a specific list of 
features ("C89 but also permitting //-comments, variadic macros, 
variable declarations on any line, inline functions, and designated 
initializers").  But I'm looking forward to it!


We already had macros for inline (e.g. Py_LOCAL_INLINE), maybe we can 
remove those.



//arry/
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C99

2016-06-04 Thread Meador Inge
On Sat, Jun 4, 2016 at 1:11 AM, Benjamin Peterson 
wrote:


> So, what say you to updating PEP 7 to allow C99 features for Python 3.6
> (in so much as GCC and MSVC support them)?
>

+1

# Meador
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] cpython: replace custom validation logic in the parse module with a simple DFA validator

2016-06-04 Thread Christian Heimes
On 2016-06-02 11:32, benjamin.peterson wrote:
> https://hg.python.org/cpython/rev/4a9159ea2536
> changeset:   101601:4a9159ea2536
> user:Benjamin Peterson 
> date:Thu Jun 02 11:30:18 2016 -0700
> summary:
>   replace custom validation logic in the parse module with a simple DFA 
> validator (closes #26526)
> 
> Patch from A. Skrobov.
> 
> files:
>   Misc/NEWS  | 3 +
>   Modules/parsermodule.c |  2545 +--
>   2 files changed, 96 insertions(+), 2452 deletions(-)
> 
> 
> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -22,6 +22,9 @@
>  Library
>  ---
>  
> +- Issue #26526: Replace custom parse tree validation in the parser
> +  module with a simple DFA validator.
> +
>  - Issue #27114: Fix SSLContext._load_windows_store_certs fails with
>PermissionError
>  
> diff --git a/Modules/parsermodule.c b/Modules/parsermodule.c
> --- a/Modules/parsermodule.c
> +++ b/Modules/parsermodule.c
> @@ -670,9 +670,75 @@
>  
>  
>  static node* build_node_tree(PyObject *tuple);
> -static int   validate_expr_tree(node *tree);
> -static int   validate_file_input(node *tree);
> -static int   validate_encoding_decl(node *tree);
> +
> +static int
> +validate_node(node *tree)
> +{
> +int type = TYPE(tree);
> +int nch = NCH(tree);
> +dfa *nt_dfa;
> +state *dfa_state;
> +int pos, arc;
> +
> +assert(ISNONTERMINAL(type));
> +type -= NT_OFFSET;
> +if (type >= _PyParser_Grammar.g_ndfas) {
> +PyErr_Format(parser_error, "Unrecognized node type %d.", TYPE(tree));
> +return 0;
> +}
> +nt_dfa = &_PyParser_Grammar.g_dfa[type];
> +REQ(tree, nt_dfa->d_type);
> +
> +/* Run the DFA for this nonterminal. */
> +dfa_state = &nt_dfa->d_state[nt_dfa->d_initial];
> +for (pos = 0; pos < nch; ++pos) {
> +node *ch = CHILD(tree, pos);
> +int ch_type = TYPE(ch);
> +for (arc = 0; arc < dfa_state->s_narcs; ++arc) {
> +short a_label = dfa_state->s_arc[arc].a_lbl;
> +assert(a_label < _PyParser_Grammar.g_ll.ll_nlabels);
> +if (_PyParser_Grammar.g_ll.ll_label[a_label].lb_type == ch_type) 
> {
> + /* The child is acceptable; if non-terminal, validate 
> it recursively. */
> +if (ISNONTERMINAL(ch_type) && !validate_node(ch))
> +return 0;
> +
> +/* Update the state, and move on to the next child. */
> +dfa_state = &nt_dfa->d_state[dfa_state->s_arc[arc].a_arrow];
> +goto arc_found;
> +}
> +}
> +/* What would this state have accepted? */
> +{
> +short a_label = dfa_state->s_arc->a_lbl;
> +int next_type;
> +if (!a_label) /* Wouldn't accept any more children */
> +goto illegal_num_children;
> +
> +next_type = _PyParser_Grammar.g_ll.ll_label[a_label].lb_type;
> +if (ISNONTERMINAL(next_type))
> +PyErr_Format(parser_error, "Expected node type %d, got %d.",
> + next_type, ch_type);
> +else
> +PyErr_Format(parser_error, "Illegal terminal: expected %s.",
> + _PyParser_TokenNames[next_type]);

Coverity doesn't that line:

CID 1362505 (#1 of 1): Out-of-bounds read (OVERRUN)
20. overrun-local: Overrunning array _PyParser_TokenNames of 58 8-byte
elements at element index 255 (byte offset 2040) using index next_type
(which evaluates to 255).

Can you add a check to verify, that next_type is not out-of-bounds, e.g.

+else if (next_type > N_TOKENS)
+PyErr_Format(parser_error, "Illegal node type %d",
next_type);



> +return 0;
> +}
> +
> +arc_found:
> +continue;
> +}
> +/* Are we in a final state? If so, return 1 for successful validation. */
> +for (arc = 0; arc < dfa_state->s_narcs; ++arc) {
> +if (!dfa_state->s_arc[arc].a_lbl) {
> +return 1;
> +}
> +}
> +
> +illegal_num_children:
> +PyErr_Format(parser_error,
> + "Illegal number of children for %s node.", nt_dfa->d_name);
> +return 0;
> +}



___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Improving the bytecode

2016-06-04 Thread Mark Shannon

On 04/06/16 10:02, Eric Snow wrote:

You should get in touch with Mark Shannon, while you're working on
ceval.  He has some definite improvements that can be made to the eval
loop.


See http://bugs.python.org/issue17611 for my suggested improvements.
I've made a new comment there.

Cheers,
Mark.



-eric

On Sat, Jun 4, 2016 at 2:08 AM, Serhiy Storchaka  wrote:

Following the converting 8-bit bytecode to 16-bit bytecode (wordcode), there
are other issues for improving the bytecode.

1. http://bugs.python.org/issue27129
Make the bytecode more 16-bit oriented.

2. http://bugs.python.org/issue27140
Add new opcode BUILD_CONST_KEY_MAP for building a dict with constant keys.
This optimize the common case and especially helpful for two following
issues (creating and calling functions).

3. http://bugs.python.org/issue27095
Simplify MAKE_FUNCTION/MAKE_CLOSURE. Instead packing three numbers in oparg
the new MAKE_FUNCTION takes built tuples and dicts from the stack.
MAKE_FUNCTION and MAKE_CLOSURE are merged in the single opcode.

4. http://bugs.python.org/issue27213
Rework CALL_FUNCTION* opcodes. Replace four existing opcodes with three
simpler and more efficient opcodes.

5. http://bugs.python.org/issue27127
Rework the for loop implementation.

6. http://bugs.python.org/issue17611
Move unwinding of stack for "pseudo exceptions" from interpreter to
compiler.

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail.com

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com