Re: [IronPython] How do I test if a variable is defined?
If you're running some sort of checker (pylint, pychecker) over this, you might get a warning about the name line not doing anything. To avoid that, you'll sometimes see a variant like this: try: unused = name except NameError: # variable 'name' is not defined. Except then PyLint will complain that 'unused' is unused, plus you code no longer accurately conveys its intent. Well pychecker ignores variables called unused (and _, but that clashes with the gettext usage). I figured pylint did too, but that was only a guess. IMO this code does accurately convey its intent - I have seen this idiom in a lot of Python code. Cheers, Tony ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] IronPython 2: Bug in Indexing (Blocker)
Dino Viehland wrote:
Thanks for the report. I've opened CodePlex bug #19350 to track this
(http://www.codeplex.com/IronPython/WorkItem/View.aspx?WorkItemId=19350).
I believe the fix is actually pretty simple and we'll discuss whether we'll
take it for 2.0 final at our Friday team meeting. If you want to try out the
fix either to unblock yourself or just see if there's other issues lurking
behind it you can do this change:
Thanks Dino - that fix certainly fixes the specific issue and means I
have less failing tests in my Worksheet test (and no new ones as far as
I can tell).
Although Resolver One now runs on IronPython 2 I'm still in a position
where there are a stack of failing tests (we have 6000 tests in total
and I'm gradually moving through them working out whether failures are
because we are testing against the old API, failures that indicate bugs
in IronPython, failures due to moving to Python 2.5, or anything else...)
Although aspects of Resolver One on IronPython 2 are just as fast as
they are on IronPython 1 there are still some specific performance
issues. We anticipate that the hardest part of the port will be
identifying and fixing those issues (thankfully we have a bunch of
performance tests to help us). If we find anything specific we will
report it back of course.
We would appreciate this fix being in 2.0 final as it is our goal to do
a new Resolver One release based on it as soon as humanly possible, and
we would prefer not to have to rely on a patched version.
Anyway - thanks for identifying a fix so quickly.
All the best,
Michael
return new MetaObject[] {
arguments[0],
new MetaObject(
Ast.Call(
typeof(PythonOps).GetMethod(MakeTuple),
Ast.NewArrayInit(typeof(object), tupleArgs)
),
- Restrictions.Empty
+ Restrictions.Combine(arguments)
),
arguments[arguments.Length-1]
};
If you just search for return new MetaObject[] in PythonProtocol.Operations.cs
you'll get to the right spot. There's a 2nd spot about 20 lines later for the getter case
which has the same issue fix.
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Michael Foord
Sent: Wednesday, November 05, 2008 8:03 AM
To: Discussion of IronPython
Subject: [IronPython] IronPython 2: Bug in Indexing (Blocker)
Hello Guys,
Discovered a bug in indexing Python objects in IronPython 2.
The following code:
class X(object):
def __setitem__(self, key, value):
print repr(key)
def f(a, b):
X()[a, b] = object()
f(1, 2)
f('one', 'two')
Produces the following exception:
Traceback (most recent call last):
File bugtest.py, line 9, in bugtest.py
File bugtest.py, line 6, in f
TypeError: Specified cast is not valid.
It looks like IronPython has a call site that is now expecting integers
the second time round.
Switching to using explicit tuples for indexing works around the problem:
X()[(a, b)] = object()
However this kind of indexing is an *extremely* common way to use our
spreadsheet API from user code, so we couldn't release with this bug in
place.
(Doing the indexing outside of the function also avoids the problem
interestingly enough.)
All the best,
Michael Foord
--
Michael Foord
Senior Software Engineer, Resolver Systems Ltd.
[EMAIL PROTECTED]
+44 (0) 20 7253 6372
Try out Resolver One! http://www.resolversystems.com/get-it/
17a Clerkenwell Road, London EC1M 5RD, UK
VAT No.: GB 893 5643 79 Registered in England and Wales as company number
5467329.
Registered address: 843 Finchley Road, London NW11 8NA, UK
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
--
http://www.ironpythoninaction.com/
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Ironclad problem, with which someone here may be able to help
Thanks Dino -- it seems that if I use ipy with -X:MTA, I can no longer
reproduce the problem. However, this bothers me a bit: I'm not competent
to follow all the consequences here, but this situation seems to imply
that Ironclad won't be usable safely from any STA thread. Is this an
intended restriction or a bug?
Incidentally, tracking down the call stacks proved to be hard work: the
timing changed enough that I only got a single failure over dozens of
runs, and it turned out I'd got the logging wrong, so I didn't find out
anything useful :(.
Dino Viehland wrote:
I would suggest getting a snap shot of the call stacks when this is happening
if that's possible. I can't pin anything down but I wonder if you could have
an STA object or something that otherwise requires message pumping. That
message pumping could happen while you're doing a Monitor.Enter call. If that
was happening maybe there is some subtle CLR bug or a surprise feature? It is
surprising that Monitor.Enter can go re-entrant but it can...
So it'd be interesting to get thread snapshots and see if
EnsureGIL (443) 2
EnsureGIL (443) 1 - omg, wtf, bbq, etc.
Could be happening because Thread 1 experiences contention, blocks and pumps
messages, causing the finalizer thread (2) to run, the lock is acquired and ...
?
Only other thing I could think of is does it repro on other machines? Maybe
it's a hardware bug as unlikely as that seems?
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of William Reade
Sent: Wednesday, November 05, 2008 10:01 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Ironclad problem, with which someone here may be able
to help
The log starts in the middle (after many lock/unlocks, some from each
thread); I'm running on x86; and I have no additional AppDomains.
I don't think it would be safe for me to entirely avoid locking during
finalization, but I could probably cut it down to a quick lock, on a
separate object, to enqueue pointers for cleanup and deallocation on the
main thread. However, I'm reluctant to do that until I'm sure that the
problem is specifically related to GC, otherwise it'll just come back as
soon as anyone tries any serious multithreading :).
Curt Hagenlocher wrote:
Locking during finalization is often considered to be a bad idea. In
particular, locking without a timeout introduces the possibility that
you will hang the finalization thread, preventing further objects from
being finalized. But clearly, that's not what's happening here.
Other questions that probably don't matter but might be interesting to
know:
Can we assume that the finalization thread isn't the first place where
this lock is required? That your log starts somewhere in the middle?
Is this under x86 or x64 or both?
Are you creating any additional AppDomains in the process?
On Wed, Nov 5, 2008 at 10:15 AM, William Reade
[EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote:
Hi Curt
I am indeed; that's how I know thread 2 is the GC thread. Is
locking during GC forbidden?
William
Curt Hagenlocher wrote:
...or, for that matter, any __del__ methods from within Python
-- which ultimately are handled by finalization.
On Wed, Nov 5, 2008 at 9:37 AM, Curt Hagenlocher
[EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
wrote:
So, the obvious question for me is whether or not you're
using any
finalizers.
On Wed, Nov 5, 2008 at 5:57 AM, William Reade
[EMAIL PROTECTED]
mailto:[EMAIL PROTECTED]
mailto:[EMAIL PROTECTED]
mailto:[EMAIL PROTECTED]
wrote:
Hi all
While running the numpy tests, I've come across a situation
which, to the best of my knowledge, is simply
impossible. I'm
hoping that one of the local .NET gurus will be able to
tell
me what I'm missing, or point me somewhere I can get
more insight.
The 4 methods involved are as follows:
---
public int GetThreadId()
{
return Thread.CurrentThread.ManagedThreadId;
}
public void WriteFlush(string info)
{
Console.WriteLine(info);
Console.Out.Flush();
}
public void EnsureGIL()
{
Monitor.Enter(this.dispatcherLock);
this.WriteFlush(String.Format(
EnsureGIL ({1}) {0}, this.GetThreadId(),
Builtin.id(this.dispatcherLock)));
}
public void ReleaseGIL()
Re: [IronPython] Latest Hosting Documentation
http://compilerlab.members.winisp.net/dlr-spec-hosting.pdf (PDF) or http://compilerlab.members.winisp.net/dlr-spec-hosting.doc (Word) On Thu, Nov 6, 2008 at 5:41 AM, Slide [EMAIL PROTECTED] wrote: Can someone point me to the latest hosting docs? Thanks, slide -- slide-o-blog http://slide-o-blog.blogspot.com/ ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Ironclad problem, with which someone here may be able to help
When you run with -X:MTA, is the object now being finalized on a thread
other than thread #2?
On Thu, Nov 6, 2008 at 7:01 AM, William Reade
[EMAIL PROTECTED]wrote:
Thanks Dino -- it seems that if I use ipy with -X:MTA, I can no longer
reproduce the problem. However, this bothers me a bit: I'm not competent to
follow all the consequences here, but this situation seems to imply that
Ironclad won't be usable safely from any STA thread. Is this an intended
restriction or a bug?
Incidentally, tracking down the call stacks proved to be hard work: the
timing changed enough that I only got a single failure over dozens of runs,
and it turned out I'd got the logging wrong, so I didn't find out anything
useful :(.
Dino Viehland wrote:
I would suggest getting a snap shot of the call stacks when this is
happening if that's possible. I can't pin anything down but I wonder if you
could have an STA object or something that otherwise requires message
pumping. That message pumping could happen while you're doing a
Monitor.Enter call. If that was happening maybe there is some subtle CLR
bug or a surprise feature? It is surprising that Monitor.Enter can go
re-entrant but it can...
So it'd be interesting to get thread snapshots and see if
EnsureGIL (443) 2
EnsureGIL (443) 1 - omg, wtf, bbq, etc.
Could be happening because Thread 1 experiences contention, blocks and
pumps messages, causing the finalizer thread (2) to run, the lock is
acquired and ... ?
Only other thing I could think of is does it repro on other machines?
Maybe it's a hardware bug as unlikely as that seems?
-Original Message-
From: [EMAIL PROTECTED] [mailto:
[EMAIL PROTECTED] On Behalf Of William Reade
Sent: Wednesday, November 05, 2008 10:01 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Ironclad problem, with which someone here may be
able to help
The log starts in the middle (after many lock/unlocks, some from each
thread); I'm running on x86; and I have no additional AppDomains.
I don't think it would be safe for me to entirely avoid locking during
finalization, but I could probably cut it down to a quick lock, on a
separate object, to enqueue pointers for cleanup and deallocation on the
main thread. However, I'm reluctant to do that until I'm sure that the
problem is specifically related to GC, otherwise it'll just come back as
soon as anyone tries any serious multithreading :).
Curt Hagenlocher wrote:
Locking during finalization is often considered to be a bad idea. In
particular, locking without a timeout introduces the possibility that
you will hang the finalization thread, preventing further objects from
being finalized. But clearly, that's not what's happening here.
Other questions that probably don't matter but might be interesting to
know:
Can we assume that the finalization thread isn't the first place where
this lock is required? That your log starts somewhere in the middle?
Is this under x86 or x64 or both?
Are you creating any additional AppDomains in the process?
On Wed, Nov 5, 2008 at 10:15 AM, William Reade
[EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
wrote:
Hi Curt
I am indeed; that's how I know thread 2 is the GC thread. Is
locking during GC forbidden?
William
Curt Hagenlocher wrote:
...or, for that matter, any __del__ methods from within Python
-- which ultimately are handled by finalization.
On Wed, Nov 5, 2008 at 9:37 AM, Curt Hagenlocher
[EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
wrote:
So, the obvious question for me is whether or not you're
using any
finalizers.
On Wed, Nov 5, 2008 at 5:57 AM, William Reade
[EMAIL PROTECTED]
mailto:[EMAIL PROTECTED]
mailto:[EMAIL PROTECTED]
mailto:[EMAIL PROTECTED]
wrote:
Hi all
While running the numpy tests, I've come across a situation
which, to the best of my knowledge, is simply
impossible. I'm
hoping that one of the local .NET gurus will be able to
tell
me what I'm missing, or point me somewhere I can get
more insight.
The 4 methods involved are as follows:
---
public int GetThreadId()
{
return Thread.CurrentThread.ManagedThreadId;
}
public void WriteFlush(string info)
{
Console.WriteLine(info);
Console.Out.Flush();
}
public void EnsureGIL()
{
Monitor.Enter(this.dispatcherLock);
this.WriteFlush(String.Format(
[IronPython] Latest Hosting Documentation
Can someone point me to the latest hosting docs? Thanks, slide -- slide-o-blog http://slide-o-blog.blogspot.com/ ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Ironclad problem, with which someone here may be able to help
Finalization still happens on thread 2, and I see no evidence of thread 1 any more; my main thread now appears to have id 3. Curt Hagenlocher wrote: When you run with -X:MTA, is the object now being finalized on a thread other than thread #2? On Thu, Nov 6, 2008 at 7:01 AM, William Reade [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Thanks Dino -- it seems that if I use ipy with -X:MTA, I can no longer reproduce the problem. However, this bothers me a bit: I'm not competent to follow all the consequences here, but this situation seems to imply that Ironclad won't be usable safely from any STA thread. Is this an intended restriction or a bug? Incidentally, tracking down the call stacks proved to be hard work: the timing changed enough that I only got a single failure over dozens of runs, and it turned out I'd got the logging wrong, so I didn't find out anything useful :(. Dino Viehland wrote: I would suggest getting a snap shot of the call stacks when this is happening if that's possible. I can't pin anything down but I wonder if you could have an STA object or something that otherwise requires message pumping. That message pumping could happen while you're doing a Monitor.Enter call. If that was happening maybe there is some subtle CLR bug or a surprise feature? It is surprising that Monitor.Enter can go re-entrant but it can... So it'd be interesting to get thread snapshots and see if EnsureGIL (443) 2 EnsureGIL (443) 1 - omg, wtf, bbq, etc. Could be happening because Thread 1 experiences contention, blocks and pumps messages, causing the finalizer thread (2) to run, the lock is acquired and ... ? Only other thing I could think of is does it repro on other machines? Maybe it's a hardware bug as unlikely as that seems? -Original Message- From: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED]] On Behalf Of William Reade Sent: Wednesday, November 05, 2008 10:01 AM To: Discussion of IronPython Subject: Re: [IronPython] Ironclad problem, with which someone here may be able to help The log starts in the middle (after many lock/unlocks, some from each thread); I'm running on x86; and I have no additional AppDomains. I don't think it would be safe for me to entirely avoid locking during finalization, but I could probably cut it down to a quick lock, on a separate object, to enqueue pointers for cleanup and deallocation on the main thread. However, I'm reluctant to do that until I'm sure that the problem is specifically related to GC, otherwise it'll just come back as soon as anyone tries any serious multithreading :). Curt Hagenlocher wrote: Locking during finalization is often considered to be a bad idea. In particular, locking without a timeout introduces the possibility that you will hang the finalization thread, preventing further objects from being finalized. But clearly, that's not what's happening here. Other questions that probably don't matter but might be interesting to know: Can we assume that the finalization thread isn't the first place where this lock is required? That your log starts somewhere in the middle? Is this under x86 or x64 or both? Are you creating any additional AppDomains in the process? On Wed, Nov 5, 2008 at 10:15 AM, William Reade [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Hi Curt I am indeed; that's how I know thread 2 is the GC thread. Is locking during GC forbidden? William Curt Hagenlocher wrote: ...or, for that matter, any __del__ methods from within Python -- which ultimately are handled by finalization. On Wed, Nov 5, 2008 at 9:37 AM, Curt Hagenlocher [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: So, the obvious question for me is whether or not you're using any
[IronPython] trivial long() bug
CPy:
long('')
Traceback (most recent call last):
File stdin, line 1, in module
ValueError: invalid literal for long() with base 10: ''
IPy:
long('')
0L
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Ironclad problem, with which someone here may be able to help
In MTA mode we'll create a new thread which is thread 3 - thread 1 will just be hanging around waiting for it to exit. Unfortunately w/o a separate EXE we can't avoid the separate thread. I've pinged the CLR dev who owns interop these days to see if he's ever seen anything like this. Unfortunately he's OOF until the 10th. My best suggestion for the time being is to avoid using Monitor for this lock so that you don't pump messages. That's probably going to mean writing your own lock and then P/Invoking to WaitForSingleObject when you experience contention and need to block the thread. Using an AutoResetEvent or ManualResetEvent directly won't work because we'll pump messages there too! I would also suggest looking at what CPython does here. Dumpbin /imports python.exe shows no imports from ole32 so I'm guessing they don't do anything to spin up COM - which makes sense. I would therefore assume when acquiring the GIL CPython will never pump messages and therefore programs don't expect possible re-entrancy - it'd probably be a good idea to mimic that behavior :). Unfortunately there are other places where we use Monitor in IronPython - list.append comes to mind. It'll be interesting to see if that turns out to be a problem eventually as well. Anyway, I'll let you know when I hear back. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of William Reade Sent: Thursday, November 06, 2008 7:29 AM To: Discussion of IronPython Subject: Re: [IronPython] Ironclad problem, with which someone here may be able to help Finalization still happens on thread 2, and I see no evidence of thread 1 any more; my main thread now appears to have id 3. Curt Hagenlocher wrote: When you run with -X:MTA, is the object now being finalized on a thread other than thread #2? On Thu, Nov 6, 2008 at 7:01 AM, William Reade [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Thanks Dino -- it seems that if I use ipy with -X:MTA, I can no longer reproduce the problem. However, this bothers me a bit: I'm not competent to follow all the consequences here, but this situation seems to imply that Ironclad won't be usable safely from any STA thread. Is this an intended restriction or a bug? Incidentally, tracking down the call stacks proved to be hard work: the timing changed enough that I only got a single failure over dozens of runs, and it turned out I'd got the logging wrong, so I didn't find out anything useful :(. Dino Viehland wrote: I would suggest getting a snap shot of the call stacks when this is happening if that's possible. I can't pin anything down but I wonder if you could have an STA object or something that otherwise requires message pumping. That message pumping could happen while you're doing a Monitor.Enter call. If that was happening maybe there is some subtle CLR bug or a surprise feature? It is surprising that Monitor.Enter can go re-entrant but it can... So it'd be interesting to get thread snapshots and see if EnsureGIL (443) 2 EnsureGIL (443) 1 - omg, wtf, bbq, etc. Could be happening because Thread 1 experiences contention, blocks and pumps messages, causing the finalizer thread (2) to run, the lock is acquired and ... ? Only other thing I could think of is does it repro on other machines? Maybe it's a hardware bug as unlikely as that seems? -Original Message- From: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED]] On Behalf Of William Reade Sent: Wednesday, November 05, 2008 10:01 AM To: Discussion of IronPython Subject: Re: [IronPython] Ironclad problem, with which someone here may be able to help The log starts in the middle (after many lock/unlocks, some from each thread); I'm running on x86; and I have no additional AppDomains. I don't think it would be safe for me to entirely avoid locking during finalization, but I could probably cut it down to a quick lock, on a separate object, to enqueue pointers for cleanup and deallocation on the main thread. However, I'm reluctant to do that until I'm sure that the problem is specifically related to GC, otherwise it'll just come back as soon as anyone tries any serious multithreading :). Curt Hagenlocher wrote: Locking during finalization is often considered to be a bad idea. In particular, locking without a timeout introduces the possibility that you will hang the finalization thread, preventing further
Re: [IronPython] trivial long() bug
Thanks, I've filed this as CodePlex #19363
(http://www.codeplex.com/IronPython/WorkItem/View.aspx?WorkItemId=19363)
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of William Reade
Sent: Thursday, November 06, 2008 8:58 AM
To: Discussion of IronPython
Subject: [IronPython] trivial long() bug
CPy:
long('')
Traceback (most recent call last):
File stdin, line 1, in module
ValueError: invalid literal for long() with base 10: ''
IPy:
long('')
0L
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
___
Users mailing list
Users@lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
[IronPython] Porting from CPython: Figuring out the calling code's directory
Hi, I'm porting Testoob to IronPython and have a module function that needs to know the filename where the code calling it is defined. In CPython I get the current frame with sys._getframe(), then climb the stack with frame.f_back until I find a different file: def _first_external_frame(): import sys # find the first frame with a filename different than this one frame = sys._getframe() while frame.f_globals[__file__] == __file__: frame = frame.f_back return frame def _calling_module_directory(): from os.path import dirname, normpath return normpath(dirname(_first_external_frame().f_globals[__file__])) I know I can't climb the stack in IronPython, but is there another way to do it? Maybe some available .NET assembly metadata? BTW - the use case is for building test suites: you can have a subpackage of tests and define __init__.py like this: def suite(): import testoob return testoob.collecting.collect_from_files(test_*.py) Thanks, orip. -- Check out my blog: http://orip.org ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Porting from CPython: Figuring out the calling code's directory
The short answer is no – we realize frames is a common request and I do have a prototype implementation of them. We’re still working on our 2.1 planning but it might be available there w/ a command line option or it might be something we could port to 2.0.1. The longer answer would be potentially with a lot of work and it could be quite slow. You could walk every module, every function, every class and attempt to gather up all the function objects in the world. Then you could call PythonOps.FunctionGetTarget on each one of them. Finally you could do a full .NET stack trace (new System.Diagnostics.StackTrace()) and attempt to figure your caller by filtering out any IronPython/DLR frames. Finally once you have that method you could see if that method is == the .Method property of any of the delegates you got from FunctionGetTarget calls. And I’m not even 100% sure if that’d work – the delegates for a PythonFunction might have a different MethodInfo then the one in the stack trace but I’m not certain (if this is the case it’d be due to the way CLR hides DynamicMethod MethodInfo’s so only the creator has access to them – you might be able to use some hack like a name comparison heuristic or private reflection or something to bring it all together). From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Ori Peleg Sent: Thursday, November 06, 2008 2:27 PM To: users@lists.ironpython.com Subject: [IronPython] Porting from CPython: Figuring out the calling code's directory Hi, I'm porting Testoob to IronPython and have a module function that needs to know the filename where the code calling it is defined. In CPython I get the current frame with sys._getframe(), then climb the stack with frame.f_back until I find a different file: def _first_external_frame(): import sys # find the first frame with a filename different than this one frame = sys._getframe() while frame.f_globals[__file__] == __file__: frame = frame.f_back return frame def _calling_module_directory(): from os.path import dirname, normpath return normpath(dirname(_first_external_frame().f_globals[__file__])) I know I can't climb the stack in IronPython, but is there another way to do it? Maybe some available .NET assembly metadata? BTW - the use case is for building test suites: you can have a subpackage of tests and define __init__.py like this: def suite(): import testoob return testoob.collecting.collect_from_files(test_*.py) Thanks, orip. -- Check out my blog: http://orip.org ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Ironclad problem, with which someone here may be able to help
Do you know if numpy is using COM anywhere? Or does Ironclad use COM for any
of its interop?
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of William Reade
Sent: Thursday, November 06, 2008 6:01 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Ironclad problem, with which someone here may be able
to help
Thanks Dino -- it seems that if I use ipy with -X:MTA, I can no longer
reproduce the problem. However, this bothers me a bit: I'm not competent
to follow all the consequences here, but this situation seems to imply
that Ironclad won't be usable safely from any STA thread. Is this an
intended restriction or a bug?
Incidentally, tracking down the call stacks proved to be hard work: the
timing changed enough that I only got a single failure over dozens of
runs, and it turned out I'd got the logging wrong, so I didn't find out
anything useful :(.
Dino Viehland wrote:
I would suggest getting a snap shot of the call stacks when this is happening
if that's possible. I can't pin anything down but I wonder if you could have
an STA object or something that otherwise requires message pumping. That
message pumping could happen while you're doing a Monitor.Enter call. If
that was happening maybe there is some subtle CLR bug or a surprise feature?
It is surprising that Monitor.Enter can go re-entrant but it can...
So it'd be interesting to get thread snapshots and see if
EnsureGIL (443) 2
EnsureGIL (443) 1 - omg, wtf, bbq, etc.
Could be happening because Thread 1 experiences contention, blocks and pumps
messages, causing the finalizer thread (2) to run, the lock is acquired and
... ?
Only other thing I could think of is does it repro on other machines? Maybe
it's a hardware bug as unlikely as that seems?
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of William Reade
Sent: Wednesday, November 05, 2008 10:01 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Ironclad problem, with which someone here may be
able to help
The log starts in the middle (after many lock/unlocks, some from each
thread); I'm running on x86; and I have no additional AppDomains.
I don't think it would be safe for me to entirely avoid locking during
finalization, but I could probably cut it down to a quick lock, on a
separate object, to enqueue pointers for cleanup and deallocation on the
main thread. However, I'm reluctant to do that until I'm sure that the
problem is specifically related to GC, otherwise it'll just come back as
soon as anyone tries any serious multithreading :).
Curt Hagenlocher wrote:
Locking during finalization is often considered to be a bad idea. In
particular, locking without a timeout introduces the possibility that
you will hang the finalization thread, preventing further objects from
being finalized. But clearly, that's not what's happening here.
Other questions that probably don't matter but might be interesting to
know:
Can we assume that the finalization thread isn't the first place where
this lock is required? That your log starts somewhere in the middle?
Is this under x86 or x64 or both?
Are you creating any additional AppDomains in the process?
On Wed, Nov 5, 2008 at 10:15 AM, William Reade
[EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote:
Hi Curt
I am indeed; that's how I know thread 2 is the GC thread. Is
locking during GC forbidden?
William
Curt Hagenlocher wrote:
...or, for that matter, any __del__ methods from within Python
-- which ultimately are handled by finalization.
On Wed, Nov 5, 2008 at 9:37 AM, Curt Hagenlocher
[EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
wrote:
So, the obvious question for me is whether or not you're
using any
finalizers.
On Wed, Nov 5, 2008 at 5:57 AM, William Reade
[EMAIL PROTECTED]
mailto:[EMAIL PROTECTED]
mailto:[EMAIL PROTECTED]
mailto:[EMAIL PROTECTED]
wrote:
Hi all
While running the numpy tests, I've come across a situation
which, to the best of my knowledge, is simply
impossible. I'm
hoping that one of the local .NET gurus will be able to
tell
me what I'm missing, or point me somewhere I can get
more insight.
The 4 methods involved are as follows:
---
public int GetThreadId()
{
return Thread.CurrentThread.ManagedThreadId;
}
public void WriteFlush(string info)
{
Console.WriteLine(info);
Re: [IronPython] Extension methods in python
By custom class you mean a class you've written? And then you want it to light up with dynamic languages? From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marty Nelson Sent: Thursday, November 06, 2008 7:18 PM To: Discussion of IronPython Subject: Re: [IronPython] Extension methods in python I actually want to use an object instance of a .NET custom class that I pass in before the script is invoked from c#. Our current scripts look something like: row.PropertySet[Foo][Bar].Value = x and it would be much better if script authors could write: row.Foo.Bar = x So I'm looking to have an on the fly function read the metadata of expected keys and then code generated python class wrapper for the script author to consume. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Miha Valencic Sent: Thursday, November 06, 2008 9:08 AM To: Discussion of IronPython Subject: Re: [IronPython] Extension methods in python Marty, from what I understand you are passing a variable into python engine and want extension methods there? You can use something like that: import clr import System class MyString(System.String): pass; def ToFoo(self): return self.upper()+' FooBarBaz' a = MyString(Hello world!); print a.ToFoo(); That would print HELLO WORLD! FooBarBaz. You can't put methods on System.String though, but you can create new object from System.String... Miha 2008/11/5 Marty Nelson [EMAIL PROTECTED]mailto:[EMAIL PROTECTED] Is there the equivalent of extension method in python? I want to put a variable into the script scope and create extension methods for it. Does this make sense and is it possible? === Notice: This e-mail message, together with any attachments, contains information of Symyx Technologies, Inc. or any of its affiliates or subsidiaries that may be confidential, proprietary, copyrighted, privileged and/or protected work product, and is meant solely for the intended recipient. If you are not the intended recipient, and have received this message in error, please contact the sender immediately, permanently delete the original and any copies of this email and any attachments thereto. ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Extension methods in python
Like a Christmas tree. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dino Viehland Sent: Thursday, November 06, 2008 7:20 PM To: Discussion of IronPython Subject: Re: [IronPython] Extension methods in python By custom class you mean a class you've written? And then you want it to light up with dynamic languages? From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marty Nelson Sent: Thursday, November 06, 2008 7:18 PM To: Discussion of IronPython Subject: Re: [IronPython] Extension methods in python I actually want to use an object instance of a .NET custom class that I pass in before the script is invoked from c#. Our current scripts look something like: row.PropertySet[Foo][Bar].Value = x and it would be much better if script authors could write: row.Foo.Bar = x So I'm looking to have an on the fly function read the metadata of expected keys and then code generated python class wrapper for the script author to consume. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Miha Valencic Sent: Thursday, November 06, 2008 9:08 AM To: Discussion of IronPython Subject: Re: [IronPython] Extension methods in python Marty, from what I understand you are passing a variable into python engine and want extension methods there? You can use something like that: import clr import System class MyString(System.String): pass; def ToFoo(self): return self.upper()+' FooBarBaz' a = MyString(Hello world!); print a.ToFoo(); That would print HELLO WORLD! FooBarBaz. You can't put methods on System.String though, but you can create new object from System.String... Miha 2008/11/5 Marty Nelson [EMAIL PROTECTED] Is there the equivalent of extension method in python? I want to put a variable into the script scope and create extension methods for it. Does this make sense and is it possible? === Notice: This e-mail message, together with any attachments, contains information of Symyx Technologies, Inc. or any of its affiliates or subsidiaries that may be confidential, proprietary, copyrighted, privileged and/or protected work product, and is meant solely for the intended recipient. If you are not the intended recipient, and have received this message in error, please contact the sender immediately, permanently delete the original and any copies of this email and any attachments thereto. ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Extension methods in python
Ok, you don’t want extension methods, you want a dynamic object, and the good
and bad news is there are a few options ☺
Let's start with the correct options first. If you can change the base class
the easiest thing to do would be to subclass
Microsoft.Scripting.Actions.Dynamic. Then you can override GetMember and
return another subclass of Dynamic which overrides SetMember and enables .Bar =
value to work.
If you can’t change the subclass then you need to implement IDynamicObject.
There’s only one method to implement and all it does is return a new
MetaObject. That MetaObject is a subclass of MetaObject that you define for
your type and it can then override methods like GetMember or SetMember (there's
many more) and instead of returning the values it returns a new MetaObject.
It’s hard to give you exact sample code because I don’t know if PropertySet is
an indexed property defined in VB, or if it returns a Dictionarystring, ??,
etc… But in general you just want to look at the Expression class and use its
factories - and if you're familiar with .NET reflection you might find it
straight forward. But the general idea is you'll want to do something like:
return new MetaObject(
Expression.Call(
Expression.Property(Expression,
Value.GetType().GetProperty(PropertySet)),
typeof(Dictionarystring, Dictionarystring,
object).GetMethod(get_Item),
Expression.Constant(action.Name)
),
Restrictions.TypeRestriction(Expression, Value.GetType())
);
And something similar for SetMember exception for you'll have an extra
MetaObject coming in as value there, so you'd end up with something similar
like:
return new MetaObject(
Expression.AssignProperty(
Expression.Call(Expression,
Value.GetType().GetMethod(get_Item)),
typeof(Whatever).GetProperty(Value),
value.Expression
),
Restrictions.TypeRestriction(Expression, Value.GetType())
);
The non-correct way (long-term) is that you can make it work with IronPython
but not necessarily other languages (e.g. C# dynamic) but the flip side is it's
really easy and straight forward. There you can define a couple of methods
like this:
[SpecialName]
public object GetCustomMember(string name) {
return row.PropertySet[name];
}
And in whatever type row.PropertySet[Foo] returns:
[SpecialName]
public void SetMember(string name, object value) {
this[name] = value;
}
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marty Nelson
Sent: Thursday, November 06, 2008 8:02 PM
To: Discussion of IronPython
Subject: Re: [IronPython] Extension methods in python
Like a Christmas tree.
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dino Viehland
Sent: Thursday, November 06, 2008 7:20 PM
To: Discussion of IronPython
Subject: Re: [IronPython] Extension methods in python
By custom class you mean a class you’ve written? And then you want it to
“light up” with dynamic languages?
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marty Nelson
Sent: Thursday, November 06, 2008 7:18 PM
To: Discussion of IronPython
Subject: Re: [IronPython] Extension methods in python
I actually want to use an object instance of a .NET custom class that I pass in
before the script is invoked from c#. Our current scripts look something like:
row.PropertySet[“Foo”][“Bar”].Value = x
and it would be much better if script authors could write:
row.Foo.Bar = x
So I’m looking to have an on the fly function read the metadata of expected
keys and then code generated python class wrapper for the script author to
consume.
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Miha Valencic
Sent: Thursday, November 06, 2008 9:08 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Extension methods in python
Marty, from what I understand you are passing a variable into python engine and
want extension methods there? You can use something like that:
import clr
import System
class MyString(System.String):
pass;
def ToFoo(self):
return self.upper()+' FooBarBaz'
a = MyString(Hello world!);
print a.ToFoo();
That would print HELLO WORLD! FooBarBaz.
You can't put methods on System.String though, but you can create new object
from System.String...
Miha
2008/11/5 Marty Nelson [EMAIL PROTECTED]
Is there the equivalent of extension method in python? I want to put a
variable into the script scope and create extension methods for it. Does this
make sense and is it possible?
===
Notice: This e-mail message, together with any attachments, contains
information of Symyx Technologies, Inc. or any of its affiliates or
