Re: [Python-Dev] AST branch update
Hi Jeremy, On Thu, Oct 13, 2005 at 04:52:14PM -0400, Jeremy Hylton wrote: I don't think the current test suite covers all of the possible syntax errors that can be raised. I'd like to add a new test suite that covers all of the remaining cases, perhaps moving some existing tests into this module as well. You might be interested in PyPy's test suite here. In particular, http://codespeak.net/svn/pypy/dist/pypy/interpreter/test/test_syntax.py contains a list of syntactically valid and invalid corner cases. If you are willing to check out the whole of PyPy (i.e. http://codespeak.net/svn/pypy/dist) you should also be able to run the whole test suite, or at least the following tests: python test_all.py pypy/interpreter/test/test_compiler.py python test_all.py pypy/interpreter/pyparser/ which compare CPython's builtin compiler with our own compilers; as of PyPy revision 18722 these tests pass on all CPython versions (2.3.5, 2.4.2, HEAD). A bientot, Armin. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] AST branch update
[Jeremy] Neil and I have been working on the AST branch for the last week. We're nearly ready to merge the changes to the head. [Raymond] Nice work. Indeed. I should've threatened to kill the AST branch long ago! :) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] AST branch update
On 10/13/05, Guido van Rossum [EMAIL PROTECTED] wrote: Indeed. I should've threatened to kill the AST branch long ago! :) :-) I decreased a lot of the memory leaks. Here are some more to work on. I doubt this list is complete, but it's a start: PyObject_Malloc (obmalloc.c:717) _PyObject_DebugMalloc (obmalloc.c:1014) compiler_enter_scope (newcompile.c:1204) compiler_mod (newcompile.c:1894) PyAST_Compile (newcompile.c:471) Py_CompileStringFlags (pythonrun.c:1240) builtin_compile (bltinmodule.c:391) Tuple (Python-ast.c:907) ast_for_testlist (ast.c:1782) ast_for_classdef (ast.c:2677) ast_for_stmt (ast.c:2758) PyAST_FromNode (ast.c:233) PyParser_ASTFromFile (pythonrun.c:1291) parse_source_module (import.c:762) load_source_module (import.c:886) new_arena (obmalloc.c:500) PyObject_Malloc (obmalloc.c:699) PyObject_Realloc (obmalloc.c:837) _PyObject_DebugRealloc (obmalloc.c:1077) PyNode_AddChild (node.c:95) shift (parser.c:112) PyParser_AddToken (parser.c:244) parsetok (parsetok.c:165) PyParser_ParseFileFlags (parsetok.c:89) PyParser_ASTFromFile (pythonrun.c:1288) parse_source_module (import.c:762) load_source_module (import.c:886) Lambda (Python-ast.c:610) ast_for_lambdef (ast.c:859) ast_for_expr (ast.c:1443) ast_for_testlist (ast.c:1776) ast_for_expr_stmt (ast.c:1845) ast_for_stmt (ast.c:2716) PyAST_FromNode (ast.c:233) PyParser_ASTFromString (pythonrun.c:1271) Py_CompileStringFlags (pythonrun.c:1237) builtin_compile (bltinmodule.c:391) BinOp (Python-ast.c:557) ast_for_binop (ast.c:1389) ast_for_expr (ast.c:1531) ast_for_testlist (ast.c:1776) ast_for_expr_stmt (ast.c:1845) ast_for_stmt (ast.c:2716) PyAST_FromNode (ast.c:233) PyParser_ASTFromString (pythonrun.c:1271) Py_CompileStringFlags (pythonrun.c:1237) builtin_compile (bltinmodule.c:391) Name (Python-ast.c:865) ast_for_atom (ast.c:1201) ast_for_expr (ast.c:1555) ast_for_testlist (ast.c:1776) ast_for_expr_stmt (ast.c:1798) ast_for_stmt (ast.c:2716) PyAST_FromNode (ast.c:233) PyParser_ASTFromString (pythonrun.c:1271) Py_CompileStringFlags (pythonrun.c:1237) builtin_compile (bltinmodule.c:391) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] AST branch update
On 10/14/05, Neal Norwitz [EMAIL PROTECTED] wrote: I decreased a lot of the memory leaks. Here are some more to work on. I doubt this list is complete, but it's a start: Oh and since I fixed the memory leaks in a generated file Python/Python-ast.c, the changes still need to be implemented in the right place (ie, Parser/asdl_c.py). Valgrind didn't report any invalid uses of memory, though there is also a lot potentially leaked memory. It seemed a lot higher than what I remembered, so I'm not sure if it's an issue or not. I'll look into that after we get the definite memory leaks plugged. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] AST branch update
I'm excited to see work continuing (resuming?) on the AST tree. I don't know how many machines you've been able to test the AST branch on. I have a linux/amd64 machine handy and I've tried to run the test suite with a fresh copy of the ast-branch. test_trace segfaults consistently, even when run alone. You didn't give me the impression that the failure was a segfault, so I'll include more information about it below. With '-x test_trace -x test_codecencodings_kr', I get through the testsuite run. Compared to a build of HEAD, also from today, I get additional failures in test_genexps test_grp test_pwd test_symtable and additional unexpected skips of: test_email test_email_codecs The 'pwd' and 'grp' failures look like they're due to a change not merged from HEAD. I'm not sure what to make of the 'genexps' failure. Is it just a harmless output difference? I didn't see you mention that in your message. Here is some of the relevant-looking output: $ ./python -E -tt ./Lib/test/regrtest.py [...] ** File /usr/src/python-ast/Lib/test/test_genexps.py, line ?, in test.test_genexps.__test__.doctests Failed example: (y for y in (1,2)) = 10 Expected: Traceback (most recent call last): ... SyntaxError: assign to generator expression not possible Got: Traceback (most recent call last): File /usr/src/python-ast/Lib/doctest.py, line 1243, in __run compileflags, 1) in test.globs File doctest test.test_genexps.__test__.doctests[38], line 1 SyntaxError: assignment to generator expression not possible (doctest test.test_genexps.__test__.doctests[38], line 1) ** File /usr/src/python-ast/Lib/test/test_genexps.py, line ?, in test.test_genexps.__test__.doctests Failed example: (y for y in (1,2)) += 10 Expected: Traceback (most recent call last): ... SyntaxError: augmented assign to tuple literal or generator expression not possible Got: Traceback (most recent call last): File /usr/src/python-ast/Lib/doctest.py, line 1243, in __run compileflags, 1) in test.globs File doctest test.test_genexps.__test__.doctests[39], line 1 SyntaxError: augmented assignment to generator expression not possible (doctest test.test_genexps.__test__.doctests[39], line 1) ** [...] test test_grp failed -- Traceback (most recent call last): File /usr/src/python-ast/Lib/test/test_grp.py, line 29, in test_values e2 = grp.getgrgid(e.gr_gid) OverflowError: signed integer is greater than maximum [...] test test_pwd failed -- Traceback (most recent call last): File /usr/src/python-ast/Lib/test/test_pwd.py, line 42, in test_values self.assert_(pwd.getpwuid(e.pw_uid) in entriesbyuid[e.pw_uid]) OverflowError: signed integer is greater than maximum The segfault in test_trace looks like this: $ gdb ./python (gdb) source Misc/gdbinit (gdb) run Lib/test/test_trace.py [...] test_10_no_jump_to_except_1 (__main__.JumpTestCase) ... FAIL Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 46912496260768 (LWP 11945)] PyEval_EvalFrame (f=0x652c30) at Python/ceval.c:1994 [1967 case COMPARE_OP:) 1994Py_DECREF(v); (gdb) print oparg $1 = 10 [PyCmp_EXC_MATCH?] (gdb) pyo v NULL $2 = void #0 PyEval_EvalFrame (f=0x652c30) at Python/ceval.c:1994 #1 0x00475800 in PyEval_EvalFrame (f=0x697390) at Python/ceval.c:3618 #2 0x00475800 in PyEval_EvalFrame (f=0x694f10) at Python/ceval.c:3618 #3 0x00475800 in PyEval_EvalFrame (f=0x649fa0) at Python/ceval.c:3618 [...] #50 0x004113bb in Py_Main (argc=Variable argc is not available.) at Modules/main.c:484 (gdb) pystack Lib/test/test_trace.py (447): no_jump_to_except_2 Lib/test/test_trace.py (447): run_test Lib/test/test_trace.py (557): test_11_no_jump_to_except_2 /usr/src/python-ast/Lib/unittest.py (581): run /usr/src/python-ast/Lib/unittest.py (280): __call__ /usr/src/python-ast/Lib/unittest.py (420): run /usr/src/python-ast/Lib/unittest.py (427): __call__ /usr/src/python-ast/Lib/unittest.py (420): run /usr/src/python-ast/Lib/unittest.py (427): __call__ /usr/src/python-ast/Lib/unittest.py (692): run /usr/src/python-ast/Lib/test/test_support.py (692): run_suite /usr/src/python-ast/Lib/test/test_support.py (278): run_unittest Lib/test/test_trace.py (600): test_main Lib/test/test_trace.py (600): module I'm not sure what other information from gdb to furnish. Jeff pgpwR8FC2UMjQ.pgp
Re: [Python-Dev] AST branch update
On Thu, Oct 13, 2005 at 05:08:41PM -0500, [EMAIL PROTECTED] wrote: test_trace segfaults consistently, even when run alone. That's a bug in frame_setlineno(), IMO. It's failing to detect an invalid jump because the lnotab generated by the new compiler is slightly different (DUP_TOP opcode corresponds to a different line). I'm not sure what to make of the 'genexps' failure. Is it just a harmless output difference? I didn't see you mention that in your message. It's a bug in the traceback.py module, IMO. See bug 1326077. Neil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] AST branch update
On Fri, Oct 14, 2005 at 01:03:28AM -0400, Raymond Hettinger wrote: Do the AST branch generate a syntax error for: foo(a = i for i in range(10)) No. It generates the same broken code as the current compiler. Neil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
