The books Learning Python in chapter Execution Optimization Tools pag.30
...Execution Optimization Tools
CPython, Jython, and IronPython all implement the Python language in
similar ways:
by compiling source code to byte code and executing the byte code on an
appropriate
virtual machine. Still other systems, including the Psyco just-in-time
compiler and the
Shedskin C++ translator, instead attempt to optimize the basic execution
model. These
systems are not required knowledge at this point in your Python career, but
a quick
look at their place in the execution model might help demystify the model
in general.
The Psyco just-in-time compiler
The Psyco system is not another Python implementation, but rather a
component that
extends the byte code execution model to make programs run faster. In terms
of
Figure 2-2, Psyco is an enhancement to the PVM that collects and uses type
information
while the program runs to translate portions of the program’s byte code all
the way
down to real binary machine code for faster execution. Psyco accomplishes
this
† Jython and IronPython are completely independent implementations of
Python that compile Python source
for different runtime architectures. It is also possible to access Java and
.NET software from standard CPython
programs: JPype and Python for .NET systems, for example, allow CPython
code to call out to Java and .NET
components.
Translation without requiring changes to the code or a separate compilation
step during
development.
Roughly, while your program runs, Psyco collects information about the
kinds of objects
being passed around; that information can be used to generate highly
efficient
machine code tailored for those object types. Once generated, the machine
code then
replaces the corresponding part of the original byte code to speed your
program’s overall
execution. The net effect is that, with Psyco, your program becomes much
quicker
over time and as it is running. In ideal cases, some Python code may become
as fast as
compiled C code under Psyco.
Because this translation from byte code happens at program runtime, Psyco
is generally
known as a just-in-time (JIT) compiler. Psyco is actually a bit different
from the JIT
compilers some readers may have seen for the Java language, though. Really,
Psyco is
a specializing JIT compiler—it generates machine code tailored to the data
types that
your program actually uses. For example, if a part of your program uses
different data
types at different times, Psyco may generate a different version of machine
code to
support each different type combination.
Psyco has been shown to speed Python code dramatically. According to its
web page,
Psyco provides “2x to 100x speed-ups, typically 4x, with an unmodified
Python interpreter
and unmodified source code, just a dynamically loadable C extension module.”
Of equal significance, the largest speedups are realized for algorithmic
code written in
pure Python—exactly the sort of code you might normally migrate to C to
optimize.
With Psyco, such migrations become even less important.
Psyco is not yet a standard part of Python; you will have to fetch and
install it separately.
It is also still something of a research project, so you’ll have to track
its evolution online.
In fact, at this writing, although Psyco can still be fetched and installed
by itself, it
appears that much of the system may eventually be absorbed into the newer
“PyPy”
project—an attempt to reimplement Python’s PVM in Python code, to better
support
optimizations like Psyco.
Perhaps the largest downside of Psyco is that it currently only generates
machine code
for Intel x86 architecture chips, though this includes Windows and Linux
boxes and
recent Macs. For more details on the Psyco extension, and other JIT efforts
that may
arise, consult http://www.python.org; you can also check out Psyco’s home
page, which
currently resides at http://psyco.sourceforge.net. ...
I think this is enough to say that psyco is faster.by.
Ovidio Marinho Falcao Neto
Web Developer
[email protected]
83 8826 9088 - Oi
83 9336 3782 - Claro
Brasil
2013/6/1 Massimo Di Pierro <[email protected]>
> Looks like I am wrong and there is no problem with pg8000.
>
>
> On Saturday, 1 June 2013 09:09:54 UTC-5, Mariano Reingart wrote:
>
>> I don't get errors nor any difference:
>>
>> db = DAL('postgres:pg8000://**reingart:1234@localhost/**
>> pg8000',pool_size=1,check_**reserved=['all'])
>>
>> db.define_table('thing',Field(**'name'))
>>
>> def test1():
>> value = r"\'"
>> id = db.thing.insert(name=value)
>> value = db(db.thing.id==id).select().**first().name
>> return dict(id=id, value=value, lenght=len(value),
>> adapter=db._adapter.__version_**_)
>>
>> def test2():
>> id = db.thing.insert(name='%')
>> value = db(db.thing.id==id).select().**first().name
>> return dict(id=id, value=value, lenght=len(value),
>> adapter=db._adapter.__version_**_)
>>
>> def test3():
>> id = db.thing.insert(name='%%')
>> value = db(db.thing.id==id).select().**first().name
>> return dict(id=id, value=value, lenght=len(value),
>> adapter=db._adapter.__version_**_)
>>
>>
>> Test1
>>
>> adapter:gluon.contrib.pg8000.**dbapi 1.10
>> id:14L
>> lenght:2
>> value:\'
>>
>> Test2
>>
>> adapter:gluon.contrib.pg8000.**dbapi 1.10
>> id:15L
>> lenght:1
>> value:%
>>
>> Test3
>>
>> adapter:gluon.contrib.pg8000.**dbapi 1.10
>> id:16L
>> lenght:2
>> value:%%
>>
>> I'm missing something?
>>
>> Regards
>>
>> Mariano Reingart
>> http://www.sistemasagiles.com.**ar <http://www.sistemasagiles.com.ar>
>> http://reingart.blogspot.com
>>
>>
>> On Sat, Jun 1, 2013 at 1:39 AM, Massimo Di Pierro
>> <[email protected]> wrote:
>> > Can you try this? With postgres and pg8000
>> >
>> > db.define_table('thing',Field(**'name'))
>> > value = r"\'"
>> > db.thing.insert(name=value)
>> >
>> > It should insert the thing but I suspect you will get an error
>> >
>> > You can also try:
>> >
>> > id = db.thing.insert(name='%')
>> > print db.thing[id].name
>> >
>> > do you get '%' or '%%'?
>> >
>> > Massimo
>> >
>> >
>> >
>> >
>> > On Thursday, 30 May 2013 17:05:30 UTC-5, Mariano Reingart wrote:
>> >>
>> >> Hi Massimo, do you have a link to the SQL injection issue?
>> >>
>> >> I couldn't reproduce it, nor the communication problem (there were an
>> >> out of sync statement issue under high loads, IIRC)
>> >>
>> >> BTW, I was given access to the pg8000 official repository (now it is
>> >> being maintained again), so I'm planning to merge my version with the
>> >> latest updates (including some performance enhancements).
>> >>
>> >> Joe: I attended the pypy tutorial at PyCon US 2012, seeking to speed
>> >> up pg8000 without luck. Not only there was no improvement, also I got
>> >> stuck by a pypy unsuported feature in Windows. Maybe pypy has better
>> >> support now, maybe the new enhancements in pg8000 are better for its
>> >> JIT compiler.
>> >>
>> >> If you just have to upload a CSV file, see the COPY statement, it is
>> >> unbeatable.
>> >>
>> >> Best regards,
>> >>
>> >> Mariano Reingart
>> >> http://www.sistemasagiles.com.**ar <http://www.sistemasagiles.com.ar>
>> >> http://reingart.blogspot.com
>> >>
>> >>
>> >> On Thu, May 30, 2013 at 6:33 PM, Massimo Di Pierro
>> >> <[email protected]> wrote:
>> >> > Mind I have security concern about pg8000. It is vulnerable to SQL
>> >> > injections in web2py.
>> >> >
>> >> >
>> >> > On Thursday, 30 May 2013 14:41:55 UTC-5, Joe Barnhart wrote:
>> >> >>
>> >> >> I have just tried both drivers -- but in an apples-and-oranges
>> >> >> comparison.
>> >> >> I used pg8000 with pypy and web2py because it is pure Python and
>> can be
>> >> >> used
>> >> >> with pypy. I used psycopg2 with python 2.7 on the same database
>> and
>> >> >> application.
>> >> >>
>> >> >> My application begins with a bulk-load of a CSV file. The file has
>> >> >> about
>> >> >> 450,000 records of about 10 fields each. Inserting the file using
>> >> >> psycopg2
>> >> >> and python 2.7 took about 4-5 minutes on a quad-core i7 iMac. The
>> >> >> memory
>> >> >> used was about 20M for postgres (largest thread) and about an equal
>> >> >> amount
>> >> >> for python. The task was handled by the web2py scheduler.
>> >> >>
>> >> >> The pypy-pg8000 version of the file load took almost an hour, but
>> that
>> >> >> is
>> >> >> deceptive. The problem is that it overwhelmed the 12GB of memory
>> in
>> >> >> the
>> >> >> computer. Both the pypy task and the postgres task ran amok with
>> >> >> memory
>> >> >> requirements. The postgres task took >8GB and forced the computer
>> into
>> >> >> swapping, killing the response time.
>> >> >>
>> >> >> Pypy is known for being somewhat of a memory hog (I was trying
>> version
>> >> >> 2.0.2). It worked darned well in web2py, with this being the only
>> >> >> problem I
>> >> >> encountered. Since my code heavily relies on modules, the speedup
>> was
>> >> >> noticible using pypy. Some of my longer tasks include creating pdf
>> >> >> files
>> >> >> and this took about 1/3 to 1/5 the time under pypy as compared to
>> >> >> cpython
>> >> >> 2.7.1.
>> >> >>
>> >> >> I know this is not an accurate comparison (because of the pypy
>> >> >> component),
>> >> >> but the runaway memory use of postgres under pg8000 concerned me so
>> I
>> >> >> thought I'd mention it.
>> >> >>
>> >> >> -- Joe B.
>> >> >>
>> >> >> On Wednesday, May 1, 2013 4:59:26 PM UTC-7, Marco Tulio wrote:
>> >> >>>
>> >> >>> Are there any advantages on one or another or are they basically
>> the
>> >> >>> same
>> >> >>> thing?
>> >> >>> I'm using psycopg2 atm.
>> >> >>>
>> >> >>> --
>> >> >>> []'s
>> >> >>> Marco Tulio
>> >> >
>> >> > --
>> >> >
>> >> > ---
>> >> > You received this message because you are subscribed to the Google
>> >> > Groups
>> >> > "web2py-users" group.
>> >> > To unsubscribe from this group and stop receiving emails from it,
>> send
>> >> > an
>> >> > email to [email protected].
>> >> > For more options, visit
>> >> > https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>.
>>
>> >> >
>> >> >
>> >
>> > --
>> >
>> > ---
>> > You received this message because you are subscribed to the Google
>> Groups
>> > "web2py-users" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an
>> > email to web2py+un...@**googlegroups.com.
>> > For more options, visit
>> > https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>.
>>
>> >
>> >
>>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
--
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
