[pypy-dev] PyPy 1.6 not working on Windows XP
Hey guys, I upgraded to PyPy 1.6 on my 2 Windows XP 32 bit machines. It crashes on both system when running the GarlicSim test suite. It shows a Windows error dialog saying pypy.exe has encountered a problem and needs to close. We are sorry for the inconvenience. and giving this data: AppName: pypy.exe AppVer: 0.0.0.0 ModName: kernel32.dll ModVer: 5.1.2600.5512 Offset: 00040b0d I can also open a dialog with a lot of data on the error (don't know how useful it is) but Windows won't let me Ctrl-C it. What can I do? Thanks, Ram. ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
1. Switch to linux. It helps. 2. To get a meaningful error log try to run the pypy from terminal. To do copy-paste you will need 3-rd party terminal, i.e. power shell. Then you'll be able to copy the error messages. Without them it is pretty much impossible to identify the problem. 3. Another option is to use debugger to run the suite step-by-step and see what happens. Choose your poison. Alex. On Wednesday 05 October 2011 13:44:14 Ram Rachum wrote: Hey guys, I upgraded to PyPy 1.6 on my 2 Windows XP 32 bit machines. It crashes on both system when running the GarlicSim test suite. It shows a Windows error dialog saying pypy.exe has encountered a problem and needs to close. We are sorry for the inconvenience. and giving this data: AppName: pypy.exe AppVer: 0.0.0.0 ModName: kernel32.dll ModVer: 5.1.2600.5512 Offset: 00040b0d I can also open a dialog with a lot of data on the error (don't know how useful it is) but Windows won't let me Ctrl-C it. What can I do? Thanks, Ram. ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
2011/10/5 Ram Rachum r...@rachum.com: Hey guys, I upgraded to PyPy 1.6 on my 2 Windows XP 32 bit machines. It crashes on both system when running the GarlicSim test suite. It shows a Windows error dialog saying pypy.exe has encountered a problem and needs to close. We are sorry for the inconvenience. and giving this data: AppName: pypy.exe AppVer: 0.0.0.0 ModName: kernel32.dll ModVer: 5.1.2600.5512 Offset: 00040b0d I can also open a dialog with a lot of data on the error (don't know how useful it is) but Windows won't let me Ctrl-C it. What can I do? pypy is a console application, i.e. it opens a Console window. You should be able to Select all and Copy all the content (click on the top-left icon) -- Amaury Forgeot d'Arc ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
On Wed, Oct 5, 2011 at 1:49 PM, Alex Pyattaev alex.pyatt...@gmail.comwrote: 1. Switch to linux. It helps. Not funny. 2. To get a meaningful error log try to run the pypy from terminal. To do copy-paste you will need 3-rd party terminal, i.e. power shell. Then you'll be able to copy the error messages. Without them it is pretty much impossible to identify the problem. I am running PyPy from terminal, bash provided by msys. The error still comes up in a dialog and the shell contains only the output from `nose` up to the failure, with no word on the failure. 3. Another option is to use debugger to run the suite step-by-step and see what happens. I'll give that a try. Choose your poison. Alex. On Wednesday 05 October 2011 13:44:14 Ram Rachum wrote: Hey guys, I upgraded to PyPy 1.6 on my 2 Windows XP 32 bit machines. It crashes on both system when running the GarlicSim test suite. It shows a Windows error dialog saying pypy.exe has encountered a problem and needs to close. We are sorry for the inconvenience. and giving this data: AppName: pypy.exe AppVer: 0.0.0.0 ModName: kernel32.dll ModVer: 5.1.2600.5512 Offset: 00040b0d I can also open a dialog with a lot of data on the error (don't know how useful it is) but Windows won't let me Ctrl-C it. What can I do? Thanks, Ram. ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
On Wed, Oct 5, 2011 at 2:07 PM, Amaury Forgeot d'Arc amaur...@gmail.comwrote: 2011/10/5 Ram Rachum r...@rachum.com: I am running PyPy from terminal, bash provided by msys. The error still comes up in a dialog and the shell contains only the output from `nose` up to the failure, with no word on the failure. Can you still see which test fails? and then add print statements to determine the exact location of the crash? -- Amaury Forgeot d'Arc I have hundreds of tests, and PyPy fails before a single one begins. It seems that PyPy crashes somewhere in nose's initialization. Isn't there a way to find the last Python line run before the crash without stepping with a finer granularity every time? Ram. ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
2011/10/5 Ram Rachum r...@rachum.com: I have hundreds of tests, and PyPy fails before a single one begins. It seems that PyPy crashes somewhere in nose's initialization. Isn't there a way to find the last Python line run before the crash without stepping with a finer granularity every time? Not without a debugger, I'm afraid -- Amaury Forgeot d'Arc ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
On Wed, Oct 5, 2011 at 2:11 PM, Amaury Forgeot d'Arc amaur...@gmail.comwrote: 2011/10/5 Ram Rachum r...@rachum.com: I have hundreds of tests, and PyPy fails before a single one begins. It seems that PyPy crashes somewhere in nose's initialization. Isn't there a way to find the last Python line run before the crash without stepping with a finer granularity every time? Not without a debugger, I'm afraid -- Amaury Forgeot d'Arc How do I run the Nose test suite on Pypy with a debugger? I usually use Wing IDE, but it doesn't support PyPy. I'm also aware of Nose's `--pdb` flag which drops you into the debugger after an error, but it doesn't work here because this crash seems to be happening at a lower level. So I don't know how to start this in a debugger. Ram. ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
2011/10/5 Ram Rachum r...@rachum.com: How do I run the Nose test suite on Pypy with a debugger? I usually use Wing IDE, but it doesn't support PyPy. I'm also aware of Nose's `--pdb` flag which drops you into the debugger after an error, but it doesn't work here because this crash seems to be happening at a lower level. So I don't know how to start this in a debugger. A Python debugger won't help, since it runs in the same (segfaulting) process. You need a C-level debugger, i.e. Visual Studio. -- Amaury Forgeot d'Arc ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
Generally, any binary-level debugger such as gdb or MSVC should work with pypy. At the very least you will find which operation crashed. If it is something really specific, for example sin/log/sign, it might be quite easy to map it back to python code. If it is not, it will be nearly impossible to find the original source line (at least I've failed when I tried). Another option is to edit the sources of the test suite adding print statements incrementally until you spot the place where it crashes. It is a slow, but very reliable way. That is of course if it is a particular segment of python code that crashes it. Also, could you send your exact environtment specs? I'll try to replicate it on a VM and see if it crashes for me too. I have an XP VM somewhere. PS: Sorry for my stupid joke about switching to linux. It was meant to cheer you up a bit. Alex. On Wednesday 05 October 2011 14:18:08 Ram Rachum wrote: On Wed, Oct 5, 2011 at 2:11 PM, Amaury Forgeot d'Arc amaur...@gmail.comwrote: 2011/10/5 Ram Rachum r...@rachum.com: I have hundreds of tests, and PyPy fails before a single one begins. It seems that PyPy crashes somewhere in nose's initialization. Isn't there a way to find the last Python line run before the crash without stepping with a finer granularity every time? Not without a debugger, I'm afraid -- Amaury Forgeot d'Arc How do I run the Nose test suite on Pypy with a debugger? I usually use Wing IDE, but it doesn't support PyPy. I'm also aware of Nose's `--pdb` flag which drops you into the debugger after an error, but it doesn't work here because this crash seems to be happening at a lower level. So I don't know how to start this in a debugger. Ram. ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
On Wed, Oct 5, 2011 at 2:22 PM, Amaury Forgeot d'Arc amaur...@gmail.comwrote: 2011/10/5 Ram Rachum r...@rachum.com: How do I run the Nose test suite on Pypy with a debugger? I usually use Wing IDE, but it doesn't support PyPy. I'm also aware of Nose's `--pdb` flag which drops you into the debugger after an error, but it doesn't work here because this crash seems to be happening at a lower level. So I don't know how to start this in a debugger. A Python debugger won't help, since it runs in the same (segfaulting) process. You need a C-level debugger, i.e. Visual Studio. -- Amaury Forgeot d'Arc I don't know how to use that... I don't program C at all, only Python. If I use a Python debugger, can't I just step forward line by line, see where I get the crash, and then isolate the offending line? Ram. ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
If I use a Python debugger, can't I just step forward line by line, see where I get the crash, and then isolate the offending line? The way pypy works - no you can not really do that. In Cpython it works somewhat better, but not in PYPY. Basically you have to use C debugger to locate the crash point, because the program that crashes can not really identify where it crashed precisely (well it can but pypy detects crash points very unprecisely). So in order to figure where exactly it crashes you have to use C debugger. OR, as I have said, add print statements until you localize the bug. *This may not work in some cases, as adding print's actually modifies the program= it might not crash at all* Alex. On Wednesday 05 October 2011 14:27:45 Ram Rachum wrote: ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
2011/10/5 Alex Pyattaev alex.pyatt...@gmail.com: In Cpython it works somewhat better, but not in PYPY How is CPython behavior better with segfaults? -- Amaury Forgeot d'Arc ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
2011/10/5 Michael Foord fuzzy...@gmail.com: Well, in recent versions it gained the faulthandler module... :-) http://pypi.python.org/pypi/faulthandler/ Yes, and this was a *great* improvement! I hope we will be able to adapt it to pypy. -- Amaury Forgeot d'Arc ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
[pypy-dev] strange error in urlunparse
Hello all. I make some experiments with pypy in our twisted project and get strange error in standard urlparse.urlunparse function def urlunparse(data): Put a parsed URL back together again. This may result in a slightly different, but equivalent URL, if the URL that was parsed originally had redundant delimiters, e.g. a ? with an empty query (the draft states that these are equivalent). scheme, netloc, url, params, query, fragment = data if params: url = %s;%s % (url, params) return urlunsplit((scheme, netloc, url, query, fragment)) If i try using apache benchmark , after ~500 successfull responses i am starting to get error on line if params: (216 line in real urlparse.py file) with message local variable 'params' referenced before assignment. How it's possible? My pypy version is Python 2.7.1 (7acf2b8fcafd, Sep 26 2011, 11:38:29) [PyPy 1.6.1-dev0 with GCC 4.6.1] on linux2 Best regards, Max Lavrenov ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
On Wed, Oct 5, 2011 at 08:22, Amaury Forgeot d'Arc amaur...@gmail.com wrote: 2011/10/5 Michael Foord fuzzy...@gmail.com: Well, in recent versions it gained the faulthandler module... :-) http://pypi.python.org/pypi/faulthandler/ Yes, and this was a *great* improvement! I hope we will be able to adapt it to pypy. It may be early to start bringing this up, but I recently created an extension to catch crashes in C code and write minidumps: https://bitbucket.org/briancurtin/minidumper. It's 3.x only at the moment, although I'll be backporting within the next few days. I'll have a look at how/if it works on PyPy once I get there. (minidumper will likely be absorbed into faulthandler once it becomes more complete, which is why I mention it) ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
On Wed, Oct 5, 2011 at 2:23 PM, Alex Pyattaev alex.pyatt...@gmail.comwrote: Generally, any binary-level debugger such as gdb or MSVC should work with pypy. At the very least you will find which operation crashed. As I said to Amaury, I don't know how to use those... Python is the only language I program in. If it is something really specific, for example sin/log/sign, it might be quite easy to map it back to python code. If it is not, it will be nearly impossible to find the original source line (at least I've failed when I tried). Another option is to edit the sources of the test suite adding print statements incrementally until you spot the place where it crashes. It is a slow, but very reliable way. That is of course if it is a particular segment of python code that crashes it. I'll try, thanks. Also, could you send your exact environtment specs? I'll try to replicate it on a VM and see if it crashes for me too. I have an XP VM somewhere. What specs do you mean? It's just the recent PyPy 1.6 on a Windows XP SP3 32 bit machine with minimal packages installed. (distribute, pip, nosetests.) Let me know if you need any more data. PS: Sorry for my stupid joke about switching to linux. It was meant to cheer you up a bit. Forgiven :) Alex. On Wednesday 05 October 2011 14:18:08 Ram Rachum wrote: On Wed, Oct 5, 2011 at 2:11 PM, Amaury Forgeot d'Arc amaur...@gmail.comwrote: 2011/10/5 Ram Rachum r...@rachum.com: I have hundreds of tests, and PyPy fails before a single one begins. It seems that PyPy crashes somewhere in nose's initialization. Isn't there a way to find the last Python line run before the crash without stepping with a finer granularity every time? Not without a debugger, I'm afraid -- Amaury Forgeot d'Arc How do I run the Nose test suite on Pypy with a debugger? I usually use Wing IDE, but it doesn't support PyPy. I'm also aware of Nose's `--pdb` flag which drops you into the debugger after an error, but it doesn't work here because this crash seems to be happening at a lower level. So I don't know how to start this in a debugger. Ram. ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
2011/10/5 Brian Curtin brian.cur...@gmail.com: On Wed, Oct 5, 2011 at 08:22, Amaury Forgeot d'Arc amaur...@gmail.com wrote: 2011/10/5 Michael Foord fuzzy...@gmail.com: Well, in recent versions it gained the faulthandler module... :-) http://pypi.python.org/pypi/faulthandler/ Yes, and this was a *great* improvement! I hope we will be able to adapt it to pypy. It may be early to start bringing this up, but I recently created an extension to catch crashes in C code and write minidumps: https://bitbucket.org/briancurtin/minidumper. It's 3.x only at the moment, although I'll be backporting within the next few days. I'll have a look at how/if it works on PyPy once I get there. The code is quite simple, and should probably compile as is with pypy (except for the PyModuleDef of course) but it would be as simple to rewrite it in RPython. btw there is a suspect thing in your code: app_name = pyname will store the char* buffer into a static variable, but nothing guarantees that the PyObject passed will stay alive! -- Amaury Forgeot d'Arc ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] strange error in urlunparse
2011/10/5 Max Lavrenov max.lavre...@gmail.com: after ~500 successfull responses i am starting to get error on line if params Looks like a JIT error to me, which has a default threshold of 1000 iterations. Can you try again with pypy --jit threshold=-1 to completely disable the JIT? -- Amaury Forgeot d'Arc ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
On Wed, Oct 5, 2011 at 4:11 PM, Ram Rachum r...@rachum.com wrote: On Wed, Oct 5, 2011 at 2:23 PM, Alex Pyattaev alex.pyatt...@gmail.comwrote: Another option is to edit the sources of the test suite adding print statements incrementally until you spot the place where it crashes. It is a slow, but very reliable way. That is of course if it is a particular segment of python code that crashes it. I'll try, thanks. Okay, I've spent a few hours print-debugging, and I think I've almost got it. The crash happens on a line: st = os.stat(s) inside `os.path.isdir`, where `s` is a string 'C:\\Documents and Settings\\User\\My Documents\\Python Projects\\GarlicSim\\garlicsim\\src' This is a directory that happens not to exist, but of course this is not a good reason to crash. I have tried running `os.stat(s)` in the PyPy shell with that same `s`, but didn't get a crash there. I don't know why it's crashing in Nose but not in the shell. Does anyone have a clue? Ram. ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
2011/10/5 Ram Rachum r...@rachum.com: Okay, I've spent a few hours print-debugging, and I think I've almost got it. The crash happens on a line: st = os.stat(s) inside `os.path.isdir`, where `s` is a string 'C:\\Documents and Settings\\User\\My Documents\\Python Projects\\GarlicSim\\garlicsim\\src' This is a directory that happens not to exist, but of course this is not a good reason to crash. I have tried running `os.stat(s)` in the PyPy shell with that same `s`, but didn't get a crash there. I don't know why it's crashing in Nose but not in the shell. Does anyone have a clue? it's possible that it's a RPython-level exception, or a bad handle because too many files wait for the garbage collector to close them. Can you give more information about the crash itself? - What are the last lines printed in the console? Try to disable stdout capture in Nose, by passing the -s option. - after the pypy process has exited, type echo %ERRORLEVEL% in the same console, to print the exit code of the last process. Which number is it? -- Amaury Forgeot d'Arc ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
On Wed, Oct 5, 2011 at 6:51 PM, Amaury Forgeot d'Arc amaur...@gmail.comwrote: 2011/10/5 Ram Rachum r...@rachum.com: Okay, I've spent a few hours print-debugging, and I think I've almost got it. The crash happens on a line: st = os.stat(s) inside `os.path.isdir`, where `s` is a string 'C:\\Documents and Settings\\User\\My Documents\\Python Projects\\GarlicSim\\garlicsim\\src' This is a directory that happens not to exist, but of course this is not a good reason to crash. I have tried running `os.stat(s)` in the PyPy shell with that same `s`, but didn't get a crash there. I don't know why it's crashing in Nose but not in the shell. Does anyone have a clue? it's possible that it's a RPython-level exception, or a bad handle because too many files wait for the garbage collector to close them. Can you give more information about the crash itself? - What are the last lines printed in the console? Try to disable stdout capture in Nose, by passing the -s option. This is the entire output: Preparing to run tests using Python 2.7.1 (080f42d5c4b4, Aug 23 2011, 11:41:11) [PyPy 1.6.0 with MSC v.1500 32 bit] Running tests directly from GarlicSim repo. Pypy doesn't have wxPython, not loading `garlicsim_wx` tests. nose.config: INFO: Set working dir to C:\Documents and Settings\User\My Documents\Python Projects\GarlicSim nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$'] nose.plugins.cover: INFO: Coverage report will include only packages: ['garlicsim', 'garlicsim_lib', 'garlicsim_wx', 'test_garlicsim', 'test_garlicsim_lib', 'test_garlicsim_wx', 'garlicsim', 'garlicsim_lib', 'garlicsim_wx', 'test_garlicsim', 'test_garlicsim_lib', 'test_garlicsim_wx', 'garlicsim', 'garlicsim_lib', 'garlicsim_wx', 'test_garlicsim', 'test_garlicsim_lib', 'test_garlicsim_wx'] - after the pypy process has exited, type echo %ERRORLEVEL% in the same console, to print the exit code of the last process. Which number is it? -1073741819 -- Amaury Forgeot d'Arc ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] gettrace?
Hi Amaury, On Wed, Oct 5, 2011 at 21:54, Amaury Forgeot d'Arc amaur...@gmail.com wrote: Is there any other way to do the equivalent of `sys.gettrace` in PyPy? Hum, it was probably ovelooked. Seems trivial to implement, will give a try. Indeed, sys.gettrace() and sys.getprofile() were added in Python 2.6, with proper unit tests in test_sys_settrace.py. We should try to understand why this test didn't fail so far on PyPy... Do we run it at all? If not, it's kind of bad. Armin ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] gettrace?
2011/10/5 Armin Rigo ar...@tunes.org: Indeed, sys.gettrace() and sys.getprofile() were added in Python 2.6, with proper unit tests in test_sys_settrace.py. We should try to understand why this test didn't fail so far on PyPy... Do we run it at all? If not, it's kind of bad. test_sys_settrace.py runs correctly, but the function is named set_and_retrieve_func... without the test_ prefix! Fixed in revision 383ca802ba07, will fix in cpython as well. -- Amaury Forgeot d'Arc ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPQ: a dbapi 2 PostgreSQL driver working with pypy
On Wed, Oct 5, 2011 at 12:59 PM, Igor Katson igor.kat...@gmail.com wrote:
So I've launched a few tests for all PostgreSQL python drivers
out there, and optimized PyPq a bit along the way.
*IntergerInsert* test does this 3 times (i wanted more, but pg8000
bloats
postgres memory usage to 1,5 gigabytes on this somewhy, so i lowered the
amount
of queries to 3)
cursor.executemany('insert into test_table values(%s, %s, %s, %s)',
[(1,2,3,4)] * 3)
*IntegerSelect* selects this data back into python
*VariableDataInsert* does the same as IntegerInsert, but inserts a string, a
datetime, a date and a timestamp into the database (except for pg8000, which
told
me that it did not support timestamps)
*VariableDataSelect* selects this data back into python
cPython 2.7.2 (32-bit, archlinux latest build), 3 inserts
Psycopg2IntegerInsert.test_insert took 1.78s
.Psycopg2IntegerSelect.test_select took 0.06s
.Psycopg2VariableDataInsert.test_insert took 2.57s
.Psycopg2VariableDataSelect.test_select took 0.25s
.Psycopg2ctIntegerInsert.test_insert took 4.46s
.Psycopg2ctIntegerSelect.test_select took 1.62s
.Psycopg2ctVariableDataInsert.test_insert took 6.00s
.Psycopg2ctVariableDataSelect.test_select took 3.31s
.PyPQIntegerInsertTest.test_insert took 3.41s
.PyPQIntegerSelectTest.test_select took 0.84s
.PyPQVariableDataInsertTest.test_insert took 4.07s
.PyPQVariableDataSelectTest.test_select took 3.70s
pg8000IntegerInsert.test_insert took 16.20s
.pg8000IntegerSelect.test_select took 1.43s
.pg8000VariableDataInsert.test_insert took 18.00s
.pg8000VariableDataSelect.test_select took 2.17s
PyPy 1.6.0 (32-bit, archlinux latest build), 3 inserts
Psycopg2ctIntegerInsert.test_insert took 2.69s
.Psycopg2ctIntegerSelect.test_select took 0.63s
.Psycopg2ctVariableDataInsert.test_insert took 4.53s
.Psycopg2ctVariableDataSelect.test_select took 1.36s
.PyPQIntegerInsertTest.test_insert took 4.61s
.PyPQIntegerSelectTest.test_select took 0.37s
.PyPQVariableDataInsertTest.test_insert took 4.48s
.PyPQVariableDataSelectTest.test_select took 1.58s
pg8000IntegerInsert.test_insert took 8.34s
.pg8000IntegerSelect.test_select took 0.60s
.pg8000VariableDataInsert.test_insert took 9.15s
.pg8000VariableDataSelect.test_select took 1.64s
As we can see, pg8000 is slow on inserts, and as i've said, it does some
strange
things to my postgres, bloating the postgres memory usage to 1.5 gigabytes
(i tried
to insert 10 records with executemany)
On cPython, pypq is faster than psycopg2ct and pg8000, except for
VariableDataSelect
test.
On PyPy, all of them get faster, except pypq, though it is still a bit
faster
than psycopg2ct in 2 tests.
Next, I tested pypq side by side to see the difference more clearly.
Here are the results.
cPython 2.7.2 (32-bit, archlinux latest build), 20 inserts
Psycopg2IntegerInsert.test_insert took 12.22s
.Psycopg2IntegerSelect.test_select took 0.39s
.Psycopg2VariableDataInsert.test_insert took 17.30s
.Psycopg2VariableDataSelect.test_select took 1.71s
.Psycopg2ctIntegerInsert.test_insert took 28.56s
.Psycopg2ctIntegerSelect.test_select took 10.48s
.Psycopg2ctVariableDataInsert.test_insert took 38.53s
.Psycopg2ctVariableDataSelect.test_select took 21.67s
.PyPQIntegerInsertTest.test_insert took 22.53s
.PyPQIntegerSelectTest.test_select took 5.59s
.PyPQVariableDataInsertTest.test_insert took 26.86s
.PyPQVariableDataSelectTest.test_select took 24.84s
PyPy 1.6.0 (32-bit, archlinux latest build), 20 inserts
Psycopg2ctIntegerInsert.test_insert took 14.11s
.Psycopg2ctIntegerSelect.test_select took 3.18s
.Psycopg2ctVariableDataInsert.test_insert took 29.36s
.Psycopg2ctVariableDataSelect.test_select took 7.78s
.PyPQIntegerInsertTest.test_insert took 25.91s
.PyPQIntegerSelectTest.test_select took 1.92s
.PyPQVariableDataInsertTest.test_insert took 30.31s
.PyPQVariableDataSelectTest.test_select took 8.73s
On 10/05/2011 01:38 AM, Maciej Fijalkowski wrote:
On Tue, Oct 4, 2011 at 4:51 PM, Igor Katsonigor.kat...@gmail.com wrote:
Hi, Dan,
before answering I'll describe the situation a bit.
there was a question today, if I know about pg8000 or psycopg2ct.
As for pg8000, pure python should be slower than ctypes, anyway, so I
don't
think these two should be compared.
[citation needed]
Great data!
It probably does make sense to run each of pypy tests twice to see how
much time is spent warming up the JIT, although definitely the data is
very interesting already.
Cheers,
fijal
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.6 not working on Windows XP
Can you fill in something for #Do something that uses lots of RAM? Because I'm not sure I'll get it right. On Thu, Oct 6, 2011 at 12:02 AM, Alex Pyattaev alex.pyatt...@gmail.comwrote: I've had a very similar stuff, as in something crashing only when run many times when I had a bug in a container type implemented in C. Basically, I had wrong refcount for the objets, which caused them to be freed by garbage collectore while they have been still used. Maybe something similar happens in the code that wraps windows API that handles file opening. That would explain why the bug never happens on linux. A good candidate would be incorrect refcount for the return value if the file does not exist. Try something like this: s=some_file rets=[] for i in range(1000): rets.append(os.stat(s)) gc.collect() #Do something that uses lots of RAM (but a random amount, preferably in small blocks) print rets if it crashes then you have exactly that issue. 1000 might be not enough to toggle the crash though, as you need the OS to actually allocate different segments of memory for this to work. The more RAM you have the more cycles you need to toggle the crash. At least this approach helped me to debug extension modules written in C. BTW, for me on VM the test case does not crash. But I have SP2 windows there. On Wednesday 05 October 2011 19:37:07 Ram Rachum wrote: On Wed, Oct 5, 2011 at 6:51 PM, Amaury Forgeot d'Arc amaur...@gmail.comwrote: 2011/10/5 Ram Rachum r...@rachum.com: Okay, I've spent a few hours print-debugging, and I think I've almost got it. The crash happens on a line: st = os.stat(s) inside `os.path.isdir`, where `s` is a string 'C:\\Documents and Settings\\User\\My Documents\\Python Projects\\GarlicSim\\garlicsim\\src' This is a directory that happens not to exist, but of course this is not a good reason to crash. I have tried running `os.stat(s)` in the PyPy shell with that same `s`, but didn't get a crash there. I don't know why it's crashing in Nose but not in the shell. Does anyone have a clue? it's possible that it's a RPython-level exception, or a bad handle because too many files wait for the garbage collector to close them. Can you give more information about the crash itself? - What are the last lines printed in the console? Try to disable stdout capture in Nose, by passing the -s option. This is the entire output: Preparing to run tests using Python 2.7.1 (080f42d5c4b4, Aug 23 2011, 11:41:11) [PyPy 1.6.0 with MSC v.1500 32 bit] Running tests directly from GarlicSim repo. Pypy doesn't have wxPython, not loading `garlicsim_wx` tests. nose.config: INFO: Set working dir to C:\Documents and Settings\User\My Documents\Python Projects\GarlicSim nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$'] nose.plugins.cover: INFO: Coverage report will include only packages: ['garlicsim', 'garlicsim_lib', 'garlicsim_wx', 'test_garlicsim', 'test_garlicsim_lib', 'test_garlicsim_wx', 'garlicsim', 'garlicsim_lib', 'garlicsim_wx', 'test_garlicsim', 'test_garlicsim_lib', 'test_garlicsim_wx', 'garlicsim', 'garlicsim_lib', 'garlicsim_wx', 'test_garlicsim', 'test_garlicsim_lib', 'test_garlicsim_wx'] - after the pypy process has exited, type echo %ERRORLEVEL% in the same console, to print the exit code of the last process. Which number is it? -1073741819 -- Amaury Forgeot d'Arc ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
