Re: [python-win32] win32api.FindFiles hangs (was COM registration hangs up: 32 bit Python 3.7 on 64 bit Win 7)
On 12/19/18, Boylan, Ross wrote: > Eryk, thank you for that very detailed explanation. The directory is on a > network share; is NtQueryDirectoryFile[Ex] still the call that would be made > for it? It's the only system call to scan a directory. It's implemented in the I/O manager by sending an IRP_MJ_DIRECTORY_CONTROL: IRP_MN_QUERY_DIRECTORY request to the device stack. In this case the requested directory information is FileBothDirectoryInformation. Locally, the filesystem for a network share is the MUP device (multiple UNC provider), which proxies access to a redirector such as SMB (mrxsmb). Since the same call works from other systems, it's probably safe to assume the problem is in the local device stack and not the server on the other end of the wire. ___ python-win32 mailing list python-win32@python.org https://mail.python.org/mailman/listinfo/python-win32
Re: [python-win32] win32api.FindFiles hangs (was COM registration hangs up: 32 bit Python 3.7 on 64 bit Win 7)
Eryk, thank you for that very detailed explanation. The directory is on a network share; is NtQueryDirectoryFile[Ex] still the call that would be made for it? Ross From: eryk sun Sent: Wednesday, December 19, 2018 3:33:09 PM To: python-win32@python.org Cc: Boylan, Ross Subject: Re: [python-win32] win32api.FindFiles hangs (was COM registration hangs up: 32 bit Python 3.7 on 64 bit Win 7) On 12/18/18, Boylan, Ross wrote: > BTW, I installed Visual Studio and wrote a little C++ program that exercised > FindNextFile. It failed in exactly the same way (returns same filename > forever if on a network drive and search spec has the exact file name), but > only when run from the original machine. The find-file handle returned by FindFirstFile[Ex] and used by FindNextFile is a pointer to a structure that includes the directory handle, a pointer to a 4 KiB file-record buffer (unless the caller requests FIND_FIRST_EX_LARGE_FETCH, which uses a 64 KiB buffer), and a pointer to the current entry's offset in the buffer. The buffer gets filled via the native system call NtQueryDirectoryFile[Ex]. The NextEntryOffset field of each variable-sized entry is used to traverse the buffer. The offset value is 0 in the last record. As the FindNextFile calls traverse the buffer, the current-entry pointer is updated for the next call. At the final entry, the pointer gets reset back to the base address. In this case, the next FindNextFile call will scan in a new batch of file records via NtQueryDirectoryFile[Ex]. This continues until the system call returns STATUS_NO_MORE_FILES (0x8006), in which case FindNextFile fails with the last error set to ERROR_NO_MORE_FILES (18). This is the normal way a directory scan is completed. Here's my guess about what's going wrong in your case. NtQueryDirectoryFile[Ex] can fail with STATUS_BUFFER_OVERFLOW (0x8005) if the buffer isn't large enough for a complete entry. In practice this shouldn't occur because the buffers used by FindFirstFile[Ex] and FindNextFile are large enough for at least one complete entry. We might have a filesystem that, on subsequent calls, stores a partial last entry in the buffer and fails with STATUS_BUFFER_OVERFLOW. FindNextFile handles this case by forgetting the partial entry, which it does by setting the NextEntryOffset field to 0 in the previous entry, if any. Then, even if there's only a single entry, it continues past the overflow error to return the current entry. When NtQueryDirectoryFile[Ex] is called again, it should resume at the entry that was partially returned in the previous call. But if it was only a single entry, and the buffer-overflow was due to a bug in the device stack, then we'll scan the same entry again with the same buffer-overflow error, and we're stuck in an infinite loop. Probably this bug is due to a filter driver in the device stack, rather than the filesystem driver. ___ python-win32 mailing list python-win32@python.org https://mail.python.org/mailman/listinfo/python-win32
Re: [python-win32] win32api.FindFiles hangs (was COM registration hangs up: 32 bit Python 3.7 on 64 bit Win 7)
On 12/18/18, Boylan, Ross wrote: > BTW, I installed Visual Studio and wrote a little C++ program that exercised > FindNextFile. It failed in exactly the same way (returns same filename > forever if on a network drive and search spec has the exact file name), but > only when run from the original machine. The find-file handle returned by FindFirstFile[Ex] and used by FindNextFile is a pointer to a structure that includes the directory handle, a pointer to a 4 KiB file-record buffer (unless the caller requests FIND_FIRST_EX_LARGE_FETCH, which uses a 64 KiB buffer), and a pointer to the current entry's offset in the buffer. The buffer gets filled via the native system call NtQueryDirectoryFile[Ex]. The NextEntryOffset field of each variable-sized entry is used to traverse the buffer. The offset value is 0 in the last record. As the FindNextFile calls traverse the buffer, the current-entry pointer is updated for the next call. At the final entry, the pointer gets reset back to the base address. In this case, the next FindNextFile call will scan in a new batch of file records via NtQueryDirectoryFile[Ex]. This continues until the system call returns STATUS_NO_MORE_FILES (0x8006), in which case FindNextFile fails with the last error set to ERROR_NO_MORE_FILES (18). This is the normal way a directory scan is completed. Here's my guess about what's going wrong in your case. NtQueryDirectoryFile[Ex] can fail with STATUS_BUFFER_OVERFLOW (0x8005) if the buffer isn't large enough for a complete entry. In practice this shouldn't occur because the buffers used by FindFirstFile[Ex] and FindNextFile are large enough for at least one complete entry. We might have a filesystem that, on subsequent calls, stores a partial last entry in the buffer and fails with STATUS_BUFFER_OVERFLOW. FindNextFile handles this case by forgetting the partial entry, which it does by setting the NextEntryOffset field to 0 in the previous entry, if any. Then, even if there's only a single entry, it continues past the overflow error to return the current entry. When NtQueryDirectoryFile[Ex] is called again, it should resume at the entry that was partially returned in the previous call. But if it was only a single entry, and the buffer-overflow was due to a bug in the device stack, then we'll scan the same entry again with the same buffer-overflow error, and we're stuck in an infinite loop. Probably this bug is due to a filter driver in the device stack, rather than the filesystem driver. ___ python-win32 mailing list python-win32@python.org https://mail.python.org/mailman/listinfo/python-win32
Re: [python-win32] win32api.FindFiles hangs (was COM registration hangs up: 32 bit Python 3.7 on 64 bit Win 7)
The same problem occurs from another Win 7 machine so it is not-machine
specific, though it might be Win 7 specific. The C++ test program works fine
from Win Server 2012 R2, run against the same share.
From: Boylan, Ross
Sent: Tuesday, December 18, 2018 11:50:13 AM
To: Tim Roberts; python-win32@python.org
Subject: Re: [python-win32] win32api.FindFiles hangs (was COM registration
hangs up: 32 bit Python 3.7 on 64 bit Win 7)
BTW, I installed Visual Studio and wrote a little C++ program that exercised
FindNextFile. It failed in exactly the same way (returns same filename forever
if on a network drive and search spec has the exact file name), but only when
run from the original machine.
This is only one of many signs that the machine (not PyWin) is messed up; we're
getting a replacement.
Thanks for your help.
Ross
P.S. Does anyone know why the python com code is doing a Dir() when it already
has the exact file name?
From: Boylan, Ross
Sent: Wednesday, December 5, 2018 5:40:56 PM
To: Tim Roberts; python-win32@python.org
Subject: Re: [python-win32] win32api.FindFiles hangs (was COM registration
hangs up: 32 bit Python 3.7 on 64 bit Win 7)
Yes, the directory is a net share. On a local hard drive I do not experience
this problem.
If I ask for *.py I get a list back that includes the file. If I ask for
"BSTI*" I get back a list of 1 element, the file. But if I ask for
"BSTImport.py" it hangs.
My experiments with Dir$() in Visual Basic , reported earlier, strongly suggest
the problem is that Windows FindNextFile function simply keeps returning the
file name forever.
I've reported this to the people running the server, though it certainly could
be a problem on my local machine.
I code in C, but don't have a development environment handy. Hence the test in
VBA.
Ross
From: python-win32 on
behalf of Tim Roberts
Sent: Wednesday, December 5, 2018 2:21:33 PM
To: python-win32@python.org
Subject: Re: [python-win32] win32api.FindFiles hangs (was COM registration
hangs up: 32 bit Python 3.7 on 64 bit Win 7)
Boylan, Ross wrote:
>
> Is this some kind of string conversion issue? My installation is borked?
> Manual debugging statements show sys.argv[0] is 'BSTImport.py'.
>
> win32api.FindFiles('BSTImport.py')
> in a python 3.7 shell (32 bit) hangs.
That's quite bizarre. The code for win32api.FindFiles is
straightforward. Is this a special directory, like a net share or
something? If you ask for *.py, is BSTImport.py in the list you get back?
Do you code in C? You might code up a trivial C app to run a
FindFirstFile/FindNextFile loop with the same parameters to see if the
same thing happens. Of course, I'm not sure what the next step would
be, whether that worked or failed.
--
Tim Roberts, t...@probo.com
Providenza & Boekelheide, Inc.
___
python-win32 mailing list
python-win32@python.org
https://mail.python.org/mailman/listinfo/python-win32
___
python-win32 mailing list
python-win32@python.org
https://mail.python.org/mailman/listinfo/python-win32
Re: [python-win32] win32api.FindFiles hangs (was COM registration hangs up: 32 bit Python 3.7 on 64 bit Win 7)
BTW, I installed Visual Studio and wrote a little C++ program that exercised
FindNextFile. It failed in exactly the same way (returns same filename forever
if on a network drive and search spec has the exact file name), but only when
run from the original machine.
This is only one of many signs that the machine (not PyWin) is messed up; we're
getting a replacement.
Thanks for your help.
Ross
P.S. Does anyone know why the python com code is doing a Dir() when it already
has the exact file name?
From: Boylan, Ross
Sent: Wednesday, December 5, 2018 5:40:56 PM
To: Tim Roberts; python-win32@python.org
Subject: Re: [python-win32] win32api.FindFiles hangs (was COM registration
hangs up: 32 bit Python 3.7 on 64 bit Win 7)
Yes, the directory is a net share. On a local hard drive I do not experience
this problem.
If I ask for *.py I get a list back that includes the file. If I ask for
"BSTI*" I get back a list of 1 element, the file. But if I ask for
"BSTImport.py" it hangs.
My experiments with Dir$() in Visual Basic , reported earlier, strongly suggest
the problem is that Windows FindNextFile function simply keeps returning the
file name forever.
I've reported this to the people running the server, though it certainly could
be a problem on my local machine.
I code in C, but don't have a development environment handy. Hence the test in
VBA.
Ross
From: python-win32 on
behalf of Tim Roberts
Sent: Wednesday, December 5, 2018 2:21:33 PM
To: python-win32@python.org
Subject: Re: [python-win32] win32api.FindFiles hangs (was COM registration
hangs up: 32 bit Python 3.7 on 64 bit Win 7)
Boylan, Ross wrote:
>
> Is this some kind of string conversion issue? My installation is borked?
> Manual debugging statements show sys.argv[0] is 'BSTImport.py'.
>
> win32api.FindFiles('BSTImport.py')
> in a python 3.7 shell (32 bit) hangs.
That's quite bizarre. The code for win32api.FindFiles is
straightforward. Is this a special directory, like a net share or
something? If you ask for *.py, is BSTImport.py in the list you get back?
Do you code in C? You might code up a trivial C app to run a
FindFirstFile/FindNextFile loop with the same parameters to see if the
same thing happens. Of course, I'm not sure what the next step would
be, whether that worked or failed.
--
Tim Roberts, t...@probo.com
Providenza & Boekelheide, Inc.
___
python-win32 mailing list
python-win32@python.org
https://mail.python.org/mailman/listinfo/python-win32
___
python-win32 mailing list
python-win32@python.org
https://mail.python.org/mailman/listinfo/python-win32
Re: [python-win32] win32api.FindFiles hangs (was COM registration hangs up: 32 bit Python 3.7 on 64 bit Win 7)
Yes, the directory is a net share. On a local hard drive I do not experience
this problem.
If I ask for *.py I get a list back that includes the file. If I ask for
"BSTI*" I get back a list of 1 element, the file. But if I ask for
"BSTImport.py" it hangs.
My experiments with Dir$() in Visual Basic , reported earlier, strongly suggest
the problem is that Windows FindNextFile function simply keeps returning the
file name forever.
I've reported this to the people running the server, though it certainly could
be a problem on my local machine.
I code in C, but don't have a development environment handy. Hence the test in
VBA.
Ross
From: python-win32 on
behalf of Tim Roberts
Sent: Wednesday, December 5, 2018 2:21:33 PM
To: python-win32@python.org
Subject: Re: [python-win32] win32api.FindFiles hangs (was COM registration
hangs up: 32 bit Python 3.7 on 64 bit Win 7)
Boylan, Ross wrote:
>
> Is this some kind of string conversion issue? My installation is borked?
> Manual debugging statements show sys.argv[0] is 'BSTImport.py'.
>
> win32api.FindFiles('BSTImport.py')
> in a python 3.7 shell (32 bit) hangs.
That's quite bizarre. The code for win32api.FindFiles is
straightforward. Is this a special directory, like a net share or
something? If you ask for *.py, is BSTImport.py in the list you get back?
Do you code in C? You might code up a trivial C app to run a
FindFirstFile/FindNextFile loop with the same parameters to see if the
same thing happens. Of course, I'm not sure what the next step would
be, whether that worked or failed.
--
Tim Roberts, t...@probo.com
Providenza & Boekelheide, Inc.
___
python-win32 mailing list
python-win32@python.org
https://mail.python.org/mailman/listinfo/python-win32
___
python-win32 mailing list
python-win32@python.org
https://mail.python.org/mailman/listinfo/python-win32
Re: [python-win32] win32api.FindFiles hangs (was COM registration hangs up: 32 bit Python 3.7 on 64 bit Win 7)
Boylan, Ross wrote:
Is this some kind of string conversion issue? My installation is borked?
Manual debugging statements show sys.argv[0] is 'BSTImport.py'.
win32api.FindFiles('BSTImport.py')
in a python 3.7 shell (32 bit) hangs.
That's quite bizarre. The code for win32api.FindFiles is
straightforward. Is this a special directory, like a net share or
something? If you ask for *.py, is BSTImport.py in the list you get back?
Do you code in C? You might code up a trivial C app to run a
FindFirstFile/FindNextFile loop with the same parameters to see if the
same thing happens. Of course, I'm not sure what the next step would
be, whether that worked or failed.
--
Tim Roberts, t...@probo.com
Providenza & Boekelheide, Inc.
___
python-win32 mailing list
python-win32@python.org
https://mail.python.org/mailman/listinfo/python-win32
[python-win32] win32api.FindFiles hangs (was COM registration hangs up: 32 bit Python 3.7 on 64 bit Win 7)
I tried using my 64 bit Python 3.6 and ran into the same problem.
Tracing through with pdb, the following line seems to be where it hung up:
> c:\program
> files\python36\lib\site-packages\win32com\server\register.py(419)RegisterClasses()
-> moduleName = os.path.splitext(win32api.FindFiles(sys.argv[0])[0][8])[0]
(Pdb) n
Is this some kind of string conversion issue? My installation is borked?
Manual debugging statements show sys.argv[0] is 'BSTImport.py'.
win32api.FindFiles('BSTImport.py')
in a python 3.7 shell (32 bit) hangs.
I think I installed pywin32 via pip.
Ross
From: Boylan, Ross
Sent: Tuesday, December 4, 2018 1:48 PM
To: python-win32@python.org
Subject: Re: COM registration hangs up: 32 bit Python 3.7 on 64 bit Win 7
Sorry: previous message went out before complete.
Terminal:
Z:\FahyGroup\Ross Boylan\Wk devel>py BSTImport.py --debug
aatmpt_bst
In main. Attempting to register.
Args = ['BSTImport.py', '--debug']
And then it just sits there. ^C at the terminal does not interrupt it, though
I can kill the python process. Searching in the registry does not indicate any
of the keys associated with my class.
How do I solve this, or at least debug it?
The main application is in 32 bit Access 2010, although the system is 64 bit
Win 7. When I discovered I couldn't open the relevant libraries (esp DAO) from
64 bit python 3.6 I installed 32 bit python 3.7. I can use it as a client
application, but wanted to make it a server.
BSTImport.py includes
import pdb, traceback
import atexit, csv, sys
import pythoncom, win32com.client
from win32com.server.exception import COMException
from win32com.client import constants
from win32com.server.register import UseCommandLine
TmpTable = "aatmpt_bst" # table to import csv into
print(TmpTable)
class BSTImport:
# COM Support
_public_methods_ = ['Fred']
_reg_clsid_ = "{050321C2-9B99--B5E9-B636DFD94C4D}"
_reg_desc_ = "BST Test Com Server in Python 32 bit"
_reg_progid_ = "Fahy.BST"
def Fred(self, name, obj):
return "Hey {}; I'm Fred. I mess with {}.".format(name, TmpTable)
# other stuff only relevant as a client
if __name__ == "__main__":
print("In main. Attempting to register.\nArgs = {}".format(sys.argv))
UseCommandLine(BSTImport)
print("I think I'm done.")
___
python-win32 mailing list
python-win32@python.org
https://mail.python.org/mailman/listinfo/python-win32
