Re: [python-win32] Memory access violation using pywin32 as WSH
We have a glimmer of hope. Just got a fresh build of xmetal.exe from the vendor, and it's not failing in my initial tests. Much more testing to do, but this looks very promising. I've asked them to tell me about the fix, but they've dodged that question so far. So let me stop here and tell you how grateful I am for your patient assistance. And (of course) for creating this package in the first place. Many thanks, Bob ___ python-win32 mailing list python-win32@python.org https://mail.python.org/mailman/listinfo/python-win32
Re: [python-win32] Memory access violation using pywin32 as WSH
On Tue, Nov 29, 2022 at 6:19 PM Mark Hammond wrote: > > On 30/11/2022 3:06 am, Bob Kline wrote: > >> As a side note, imp.new_module() [3] was deprecated back in Python 3.4. > > Yep, you are looking at a very old version. True, but I checked HEAD for the main branch, and the code [1] is still calling the deprecated method. Cheers, Bob [1] https://github.com/mhammond/pywin32/blob/main/com/win32comext/axscript/client/pyscript.py#L217 ___ python-win32 mailing list python-win32@python.org https://mail.python.org/mailman/listinfo/python-win32
Re: [python-win32] Memory access violation using pywin32 as WSH
On Tue, Nov 29, 2022 at 6:23 PM Mark Hammond wrote: > The close method is Microsoft is not helping much here. The documentation to which you linked has no information about this method other than that it returns a 32-bit integer for which no semantics are given, and that it lives in the Microsoft.VisualStudio.Debugger.Interop namespace. I'm having a little difficulty figuring out how that namespace would be appropriate for their general scripting language interface. > It's worth pointing out that any "obvious" bug in pywin32 > here would probably be able to be reproduced in, and > reported by, the various other hosts, including cscript.exe > and wscript.exe (which themselves are just active script > hosts, just like your problematic host) I had that thought, too. But then I realized that cscript.exe and wscript.exe use a model of a separate process for each invocation of a single script. That script can be very complicated, but it's loaded, parsed, and executed once. I can run a pretty complicated Python macro once per XMetaL session (by which I mean separate process) without any problem. If I want to run a script twice using cscript.exe I have to launch two separate cscript.exe processes. It occurred to me that a better comparison might be using Internet Explorer, which can presumably run multiple scripts multiple times in a single process. So I grabbed form.htm from axscript/Demos/client/ie (I promise I'm looking at the correct repository now ), thinking I'd see if clicking the Submit button multiple times caused any problems. Unfortunately, the scripts didn't fire at all, and the browser's console window didn't provide any explanation as to why. (By the way, I had to fix the third INPUT element, which was malformed, as its tag was missing its right angle bracket delimiter. IE may have been tolerant of that broken syntax at some time in the past, but it doesn't accept it now.) Cheers, Bob ___ python-win32 mailing list python-win32@python.org https://mail.python.org/mailman/listinfo/python-win32
Re: [python-win32] Memory access violation using pywin32 as WSH
On Tue, Nov 29, 2022 at 6:19 PM Mark Hammond wrote: > ... the ax object isn't going to have a Reset - so whatever that's trying to > do isn't getting done. How would we go about finding out what that missing method was supposed to do, so that we can know whether the failure to do it is related to the problem? One of the frustrations here is that while the documentation for the host application says "You can use any scripting language that conforms to the Microsoft Scripting Language Interface" trying to find the specification for that interface on Microsoft's web site turns up nothing. Has Microsoft published the spec and then gone to the trouble of extirpating every trace of it? Do you have a copy? > ... Yep, you are looking at a very old version. Sorry for the old link. Don't have any idea how I ended up in that repository. Obviously the copy I was hacking to get my debug logging was the real thing, installed by pip. I'm suitably embarrassed that I didn't notice that they didn't match. > ... when I say I'm hoping to see a stack trace, I mean a native stack trace I'll see what I can do to produce one. Thanks, Bob ___ python-win32 mailing list python-win32@python.org https://mail.python.org/mailman/listinfo/python-win32
Re: [python-win32] Memory access violation using pywin32 as WSH
On 30/11/2022 9:09 am, Bob Kline wrote: So at least part of that puzzle is cleared up, though it's still somewhat unsettling that only seven calls to ScriptItem.Close() are made, regardless of how many ScriptItem objects have been instantiated during a session. The close method is https://learn.microsoft.com/en-us/dotnet/api/microsoft.visualstudio.debugger.interop.iactivescript.close?view=visualstudiosdk-2019, so it's the responsibility of the host to call that. I agree it's a bit of a red flag, and may point to confusion in the host about object lifetimes, which could certainly end up causing what you are seeing. It's worth pointing out that any "obvious" bug in pywin32 here would probably be able to be reproduced in, and reported by, the various other hosts, including cscript.exe and wscript.exe (which themselves are just active script hosts, just like your problematic host) Mark ___ python-win32 mailing list python-win32@python.org https://mail.python.org/mailman/listinfo/python-win32
Re: [python-win32] Memory access violation using pywin32 as WSH
On 30/11/2022 3:06 am, Bob Kline wrote:
Cranking up the microscope (or spelunking deeper, whichever metaphor
you prefer), the first thing I'll say is that this code is devilishly
complex.
Here's what I've found in the latest pass.
First, there's a minor bug at line 353 of pyscript.py [1], where
globalNameSpaceModule is set to None when it's
self.globalNameSpaceModule which should be set to None. The bug is
currently innocuous, as the only invocation of the method in which
that assignment is made in this version of the code is from
PyScript.Close(), and that method makes the correct assignment:
Oops, yeah, that looks like a bug (but note that the links you are using
are from sublime text, not from pywin32 - it looks quite old - the
correct link is
https://github.com/mhammond/pywin32/blob/main/com/win32comext/axscript/client/pyscript.py
where it is at line 416.
self.globalNameSpaceModule = None
It seems to me that there's a more serious problem at line 350 of that
module [2], where that same method appears to be invoking a method
which does not exist, at least as far as I can determine (bearing in
mind that I might not completely understand all the nuances of the
tricky code here).
self.globalNameSpaceModule.ax._Reset_()
Working backward, to see if I can figure out where _Reset_ would be
defined, I found this code in the PyScript class:
def InitNew(self):
framework.COMScript.InitNew(self)
import imp
self.scriptDispatch = None
self.globalNameSpaceModule = imp.new_module("__ax_main__")
self.globalNameSpaceModule.__dict__['ax'] = AXScriptAttribute(self)
It's not completely clear how this method gets invoked (I don't see
any calls to it, but I do see that "InitNew" appears as one of the
strings in the _public_methods_ array attribute of the base class),
Yes, that's a COM method, so called by the host - see
https://learn.microsoft.com/en-us/dotnet/api/microsoft.visualstudio.debugger.interop.iactivescriptparse64.
That try/except block was probably just handling the case where Reset is
called twice without an InitNew call in between (in which case
self.globalNameSpaceModule is None, so will not have a .ax attribute) -
but I agree with your analysis that even if it *did* have that
attribute, the ax object isn't going to have a Reset - so whatever
that's trying to do isn't getting done.
As a side note, imp.new_module() [3] was deprecated back in Python 3.4.
Yep, you are looking at a very old version.
I'm attaching the logs I captured. One set is for a session in which
the macro was only invoked once (and therefore didn't result in any
observable failures) and the other for the session in which the macro
was invoked twice, with the symptoms described earlier in this thread.
Please let me know if anything else stands out as potentially useful
Nothing stands out there :( But that's not particularly surprising as it
really should be impossible for bugs in that .py code to crash the
process - strangeness could happen, but an access violation shouldn't be
possible. Something like a COM reference counting bug in (either in
axscript or in the host) could explain this.
Re your other message, when I say I'm hoping to see a stack trace, I
mean a native stack trace, as should be shown in the event viewer or in
a JIT debugger, which will show the symbols in the various DLLs rather
than the Python functions. Eg, it showing the crash happening in, say,
IUnknown::Release would imply such a reference counting bug, or if it
instead shows it in some other COM method it would help narrow down the
problem area. Unfortunately, without knowing your environment I can't
offer specific advice about how to get it - but running the crashing
program under any Visual Studio version would certainly offer it.
HTH,
Mark
___
python-win32 mailing list
python-win32@python.org
https://mail.python.org/mailman/listinfo/python-win32
Re: [python-win32] Memory access violation using pywin32 as WSH
One more data point. I added a bit more to my logging and learned that the seven calls to the ScriptItem constructor each time the macro is invoked are for the seven global objects which the host application (XMetaL) makes available to the scripts: * FormDriver * ActiveDocument * Selection * Application * FormFuncs * ResourceManager So at least part of that puzzle is cleared up, though it's still somewhat unsettling that only seven calls to ScriptItem.Close() are made, regardless of how many ScriptItem objects have been instantiated during a session. Cheers, Bob ___ python-win32 mailing list python-win32@python.org https://mail.python.org/mailman/listinfo/python-win32
Re: [python-win32] Memory access violation using pywin32 as WSH
Cranking up the microscope (or spelunking deeper, whichever metaphor
you prefer), the first thing I'll say is that this code is devilishly
complex.
Here's what I've found in the latest pass.
First, there's a minor bug at line 353 of pyscript.py [1], where
globalNameSpaceModule is set to None when it's
self.globalNameSpaceModule which should be set to None. The bug is
currently innocuous, as the only invocation of the method in which
that assignment is made in this version of the code is from
PyScript.Close(), and that method makes the correct assignment:
self.globalNameSpaceModule = None
It seems to me that there's a more serious problem at line 350 of that
module [2], where that same method appears to be invoking a method
which does not exist, at least as far as I can determine (bearing in
mind that I might not completely understand all the nuances of the
tricky code here).
self.globalNameSpaceModule.ax._Reset_()
Working backward, to see if I can figure out where _Reset_ would be
defined, I found this code in the PyScript class:
def InitNew(self):
framework.COMScript.InitNew(self)
import imp
self.scriptDispatch = None
self.globalNameSpaceModule = imp.new_module("__ax_main__")
self.globalNameSpaceModule.__dict__['ax'] = AXScriptAttribute(self)
It's not completely clear how this method gets invoked (I don't see
any calls to it, but I do see that "InitNew" appears as one of the
strings in the _public_methods_ array attribute of the base class),
but I can tell from my logging that it does get called. However it
gets invoked, that means that the _Reset_() call being made above is
on an instance of the AXScriptAttribute class, which—as far as I can
tell—has no _Reset_() method (and no base class from which it could
inherit that method). This is confirmed by an exception which is
caught at shutdown time in the PyScript.ResetNamespace() method. The
handler for that exception looks like this:
except AttributeError:
pass # ???
The obvious questions would be
* what was the non-existent method expected to do?
* could the fact that it didn't get done be causing my problem?
The ResetNamespace method has no documentation, so all I would have
for answers to those questions would be wild guesses.
As a side note, imp.new_module() [3] was deprecated back in Python 3.4.
I'm attaching the logs I captured. One set is for a session in which
the macro was only invoked once (and therefore didn't result in any
observable failures) and the other for the session in which the macro
was invoked twice, with the symptoms described earlier in this thread.
Please let me know if anything else stands out as potentially useful
in the logs.
Cheers,
Bob
[1]
https://github.com/SublimeText/Pywin32/blob/master/lib/x64/win32comext/axscript/client/pyscript.py#L353
[2]
https://github.com/SublimeText/Pywin32/blob/master/lib/x64/win32comext/axscript/client/pyscript.py#L350
[3] https://docs.python.org/3/library/imp.html#imp.new_module
pyscript-logs.tar
Description: Unix tar archive
___
python-win32 mailing list
python-win32@python.org
https://mail.python.org/mailman/listinfo/python-win32
Re: [python-win32] Memory access violation using pywin32 as WSH
On Mon, Nov 28, 2022 at 7:24 PM Mark Hammond wrote:
> ...
> The absolute best thing for us would be to reproduce the crash in the
> test code at
> https://github.com/mhammond/pywin32/tree/main/com/win32comext/axscript/test.
I presume that would depend on getting the vendor to open the kimono
and reveal which calls to the scripting engine the application is
making at what times. One handicap I'm dealing with is the fact that I
don't have a good understanding of what the "Microsoft Scripting
Language Interface" is. Searching for it in MSDN turns up only an
article ("Using COM Objects in Windows Script Host") which describes
running scripts using WScript.exe or CScript.exe. It seems unlikely
that the XMetaL application would be launching a fresh CScript.exe or
WScript.exe process every time a macro is invoked (with the massive
performance hit that would introduce), so presumably they have a copy
of this elusive interface specification which they're following to
emulate what those applications are doing.
> The next best thing would probably be a stack trace of the crash.
Any tips for how I would go about obtaining such a thing? I suppose
it's possible that the crash happens while some Python code is
running, and I'd be able to wrap that code in a try block and dump a
stack track to a file (assuming I can guess what code that would be).
But it seems at least as likely that when the crash occurs (that is,
when the dialog window appears at logoff time reporting the memory
access violation) we are no longer anywhere near any of the Python
code. Perhaps the Python code (or C code underneath it) has done
something naughty (for example, trashing a pointer somewhere) well
before the actual crash happens and it only manifests itself at logoff
time when the OS forces the application to let go and really shut
down.
Any advice you can provide to help me get a better understanding of
how the moving parts interact and what I should try next would be
immensely appreciated.
Many thanks,
Bob
___
python-win32 mailing list
python-win32@python.org
https://mail.python.org/mailman/listinfo/python-win32
Re: [python-win32] Memory access violation using pywin32 as WSH
It's been years since I've looked at this, but IIRC, a "script item" is whatever the host wants it to be - eg, in a HTML page, it would be anything between tags. I believe WSH would just use one for a script file. The Close() method will generally be called by the host - not calling this could cause a memory leak (particularly if the COM reference count also didn't hit zero), but it shouldn't cause a crash. That said though, I certainly would not rule out a bug in pywin32's implementation. The absolute best thing for us would be to reproduce the crash in the test code at https://github.com/mhammond/pywin32/tree/main/com/win32comext/axscript/test. The next best thing would probably be a stack trace of the crash. Cheers, Mark On 29/11/2022 10:38 am, Bob Kline wrote: On Mon, Nov 28, 2022 at 4:12 PM Bob Kline wrote: Perhaps, for example, I'll be able to find something which is supposed to get released at application shutdown time but which isn't. I've added some debug logging to the top of most methods in pyscript.py, mostly just announcing that the method was invoked. I was a little surprised to find that for each invocation of the macro seven instances of the ScriptItem class were created, instead of just one. So clearly I have yet to figure out exactly what a ScriptItem is. The really interesting thing is that if I invoke the macro once for a given run of the host application, the ScriptItem.Close() method is called seven times, matching the seven times the ScriptItem constructor is hit. However, if the macro is invoked twice for a run of the host application, I still see only seven calls to ScriptItem.Close(), even though there are now fourteen calls to the ScriptItem constructor. Can I assume that this is a problem? Sure smells like one. Where does the responsibility for calling the Close() method ultimately fall? ___ python-win32 mailing list python-win32@python.org https://mail.python.org/mailman/listinfo/python-win32
Re: [python-win32] Memory access violation using pywin32 as WSH
On Mon, Nov 28, 2022 at 4:12 PM Bob Kline wrote: > > Perhaps, for example, I'll be able to find something which is supposed > to get released at application shutdown time but which isn't. I've added some debug logging to the top of most methods in pyscript.py, mostly just announcing that the method was invoked. I was a little surprised to find that for each invocation of the macro seven instances of the ScriptItem class were created, instead of just one. So clearly I have yet to figure out exactly what a ScriptItem is. The really interesting thing is that if I invoke the macro once for a given run of the host application, the ScriptItem.Close() method is called seven times, matching the seven times the ScriptItem constructor is hit. However, if the macro is invoked twice for a run of the host application, I still see only seven calls to ScriptItem.Close(), even though there are now fourteen calls to the ScriptItem constructor. Can I assume that this is a problem? Sure smells like one. Where does the responsibility for calling the Close() method ultimately fall? -- Bob Kline https://www.rksystems.com mailto:bkl...@rksystems.com ___ python-win32 mailing list python-win32@python.org https://mail.python.org/mailman/listinfo/python-win32
[python-win32] Memory access violation using pywin32 as WSH
We've got a project we've supported for many years which involves
using XMetaL (a commercial XML editor) customized with macros written
in JScript. The vendor claims that the scripting "can use any
scripting language that conforms to the Microsoft Scripting Language
Interface." Since JScript is much more restricted in what it can do,
and since our team is much more familiar with Python than with
JScript, we would like to switch to Python for our scripts. However, a
dirt-simple proof of concept fails with a memory access violation. We
created a macro file with a single macro which displays an alert box
with a simple string. If the lang attribute for the macro element (the
macros are stored in an XML file, with the code for each macro stored
in a MACRO element) is set to "JScript" the macro can be run an
unlimited number of times without any problems. If the lang attribute
is set to "Python" (the macro is so simple that the syntax is
identical for the two languages) the macro can be run once without any
apparent problems, as long as you close the XMetaL application and
launch it again before the next run of the macro. However, if XMetaL
is launched and the macro is run twice or more before shutting down
the application, it is no longer possible to launch it again until the
user logs off the machine and logs back in. When the user logs off, a
dialog box appears reporting a memory access violation ("xmetal.exe -
Application Error. The instruction at referenced
memory at . The memory could not be read."). To be
clear — the macro does what it's supposed to (display a dialog box
with a string) each time it is run. The problem only manifests itself
when trying to launch the application again or when logging off the
machine. Interestingly, when the application is in this "I can't be
launched again and I will report a failure at logoff time" state there
is no evidence that the process is still hanging around. It doesn't
show up in Process Manager, for example. But it seems clear from the
title bar of the error dialog that it's coming from the application.
At this point I don't know for sure whether the problem is caused by
something the XMetaL application is doing wrong (and presumably
differently than what it does when the scripting engine is JScript) or
by something that pyscript.py is doing. I am aware that I could easily
get caught between two projects, each of which claims that the failure
must be caused by the other. I have reported the problem to the XMetaL
support team and I am waiting to hear what they have to say. I'm
hoping that the data point of "one macro invocation succeeds, two
macro invocations fail" provides a useful clue.
What would be helpful from the pywin32 team's side would be any
suggestions you might have for how I might go about tracking what's
going on, presumably by hacking pyscript.py and/or framework.py in
win32comext/axscript/client to add logging to the file system.
Perhaps, for example, I'll be able to find something which is supposed
to get released at application shutdown time but which isn't. I'm
still trying to absorb the axscript/client code and wrap my head
around what the processing flow is and what the different classes are
doing.
Any clues for where to start? Anyone else run into similar memory
access violations using Python as a Windows Scripting Host?
Many thanks!
--
Bob Kline
https://www.rksystems.com
mailto:bkl...@rksystems.com
___
python-win32 mailing list
python-win32@python.org
https://mail.python.org/mailman/listinfo/python-win32
