Curt Hagenlocher wrote:
The" .NET people" might say they already have an answer which involves
precompiling the IL with NGEN. And in fact, using DLR precompilation
and NGEN with IronPython results in some pretty nice performance
improvements.
But using IronPython directly against .py files incurs several
expenses that aren't part of normal .NET startup. We have to parse
the .py file, build DLR-level expression trees from the language tree
and build IL from the expression tree. Only then are we at the point
where a C# starts, and can JIT the IL.
It's worth noting that CPython has to do similar work when it first
loads a .py file. It, too, has to parse into a tree structure and
generate byte codes for the tree. And even though the CPython byte
code is at a higher level than the IL generated by the DLR, CPython
still has a performance issue -- which is addressed by storing the
byte codes as a .pyc file.
The difference is still orders of magnitude smaller though...
We're looking forward to moving to precompiled binaries with IronPython
2 at Resolver Systems - but there we will still have situations where we
need to import from py files.
Michael
I'm hoping that over the next year, we'll come up with a convention
(like .pyc) that allows subsequent runs of a program to benefit from
the work done in previous runs -- without having to go through the
manual step of DLR precompilation.
On Mon, Oct 13, 2008 at 9:57 AM, Vernon Cole <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>> wrote:
Help me out here. I have seen many references to schemes to
optimize import speeds on Iron Python. Isn't this an attempt to
make up for the snail-like startup speeds suffered by *all* .net
applications? It seems to me that when Iron Python finally starts
running, it imports and performs very well. The slowdown is in
getting the .net engine started. Try comparing startup times for
"Hello World" in C Python and Iron.
It is my experience that I can identify .net programs in
general by the fact that I have double-clicked its desktop icon
three times (thinking I had missed it somehow) before the first
instance displays its splash screen, and then the next two display
an "already running" dialog.
Any chance of getting the .net people to help out with this?
--
Vernon Cole
On Mon, Oct 13, 2008 at 10:00 AM, Dino Viehland
<[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote:
This is all still on 1.x, right? It looks like #1 is fixed in
2.0 (we are locking but on the wrong object in 1.x).
#2 is still broken in 2.x though as well.
-----Original Message-----
From: [EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>
[mailto:[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>] On Behalf Of
Kamil Dworakowski
Sent: Monday, October 13, 2008 4:45 AM
To: users@lists.ironpython.com <mailto:users@lists.ironpython.com>
Subject: [IronPython] parallel importing
Importing time is such a pain that I wanted to do it in
parallel on
many threads. I figured that if the modules where imported in
such a
way that:
no two threads import the same module at the same time
everything would be fine. To ensure that condition, it is
enough to
build a dependency graph and import based on that.
I did it: Resolver One start up time improved by 20% on a two-core
machine. But it crashes, surprisingly on single core machines
it is
more often (6 crashes on 200 starts).
So far I have identified two causes for crashes:
1. One thread imports a module with class B inside while
another is
importing a module with class C inside. If B and C are
subclasses of
A, it can result in IndexOutOfRangeException being raised,
when, under
the hood, IronPython.Runtime.Types.DynamicType.AddSubclass is
being
executed.
2. Attributes on .NET modules are loaded lazily, so importing
namespaces only is not enough. Attribute getting from reflected
packages is not thread safe. Looks like I would have to import
every
class explicitly (would that be enough?).
Second cause would be pretty easy to address, but I'm not so sure
about the first one. Are there any more potential points of
problems?
I am beginning to think I was to optimistic about all of this
importing on multiple cores, but if these are the only ones it
could
probably be still fixed.
If anyone is interested the code for it is on github:
http://github.com/luntain/ipy-parallel-import.
--
Kamil Dworakowski
Resolver Systems Ltd.
_______________________________________________
Users mailing list
Users@lists.ironpython.com <mailto:Users@lists.ironpython.com>
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
_______________________________________________
Users mailing list
Users@lists.ironpython.com <mailto:Users@lists.ironpython.com>
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
_______________________________________________
Users mailing list
Users@lists.ironpython.com <mailto:Users@lists.ironpython.com>
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
------------------------------------------------------------------------
_______________________________________________
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.resolverhacks.net/
http://wwww.theotherdelia.co.uk/
_______________________________________________
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
