Re: [Pytables-users] New find_library('hdf5dll.dll') call fails in frozen app
Hi Stuart,
Il 27/08/2012 17:43, Stuart Mentzer ha scritto:
Hello,
I upgraded to PyTables 2.4.0 and I was freezing an application on Windows
with PyInstaller. The frozen app fails at this new find_library call in
__init__.py:
if not ctypes.util.find_library('hdf5dll.dll'):
raise ImportError('Could not load hdf5dll.dll, please ensure' +
' that it can be found in the system path')
PyInstaller correctly places this DLL in the same directory as the
application .exe where standard Windows DLL search logic will find it.
Apparently the find_library doesn't do that in a frozen application. That is
a big problem. I had to comment this code out to get a working frozen app.
That code was added in revision e9f6919.
It is mainly a sanity check added under request of one of our users:
https://github.com/PyTables/PyTables/pull/146
This is on Windows 7 64-bit with a 32-bit Python toolchain. Trying both
PyInstaller 1.5.1 and 2.0.
Should I file a bug report? Any easy work-around?
Thanks,
Stuart
Yes please file a pull request with your patch.
It would be nice to preserve the sanity check in standard case so,
maybe, a good solution could be adding some check on sys.frozen or
something like that.
Thank you
--
Antonio Valentino
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
Re: [Pytables-users] New find_library('hdf5dll.dll') call fails in frozen app
Hi Stuart,
Il 27/08/2012 17:43, Stuart Mentzer ha scritto:
Hello,
I upgraded to PyTables 2.4.0 and I was freezing an application on Windows
with PyInstaller. The frozen app fails at this new find_library call in
__init__.py:
if not ctypes.util.find_library('hdf5dll.dll'):
raise ImportError('Could not load hdf5dll.dll, please ensure' +
' that it can be found in the system path')
PyInstaller correctly places this DLL in the same directory as the
application .exe where standard Windows DLL search logic will find it.
Apparently the find_library doesn't do that in a frozen application. That is
a big problem. I had to comment this code out to get a working frozen app.
That code was added in revision e9f6919.
It is mainly a sanity check added under request of one of our users:
https://github.com/PyTables/PyTables/pull/146
This is on Windows 7 64-bit with a 32-bit Python toolchain. Trying both
PyInstaller 1.5.1 and 2.0.
Should I file a bug report? Any easy work-around?
Thanks,
Stuart
Yes please file a pull request with your patch.
It would be nice to preserve the sanity check in standard case so,
maybe, a good solution could be adding some check on sys.frozen or
something like that.
Thank you
Antonio,
Thanks for the quick response.
To save time I did the report as a simple Issue
(https://github.com/PyTables/PyTables/issues/177) showing the patch code that
preserves and with a reference to #146. I hope that suffices.
Stuart
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
Re: [Pytables-users] New find_library('hdf5dll.dll') call fails in frozen app
Hi Christoph,
thank you very much for your hints.
Il 27/08/2012 19:22, Christoph Gohlke ha scritto:
On 8/27/2012 9:42 AM, Antonio Valentino wrote:
Hi Stuart,
Il 27/08/2012 17:43, Stuart Mentzer ha scritto:
Hello,
I upgraded to PyTables 2.4.0 and I was freezing an application on Windows
with PyInstaller. The frozen app fails at this new find_library call in
__init__.py:
if not ctypes.util.find_library('hdf5dll.dll'):
raise ImportError('Could not load hdf5dll.dll, please ensure'
+
' that it can be found in the system path')
PyInstaller correctly places this DLL in the same directory as the
application .exe where standard Windows DLL search logic will find it.
Apparently the find_library doesn't do that in a frozen application. That
is a big problem. I had to comment this code out to get a working frozen
app.
That code was added in revision e9f6919.
It is mainly a sanity check added under request of one of our users:
https://github.com/PyTables/PyTables/pull/146
This is on Windows 7 64-bit with a 32-bit Python toolchain. Trying both
PyInstaller 1.5.1 and 2.0.
Should I file a bug report? Any easy work-around?
Thanks,
Stuart
Yes please file a pull request with your patch.
It would be nice to preserve the sanity check in standard case so,
maybe, a good solution could be adding some check on sys.frozen or
something like that.
Thank you
Hello,
As a workaround for frozen distributions, try to add the sys.executable
directory to os.environ['PATH'] before importing tables.
Christoph, I suppose it can also be done at this point too:
https://github.com/PyTables/PyTables/blob/develop/tables/__init__.py#L24
Isn't it?
Please Stuart, can you try this fix as well.
Ctypes only tries to find a library in the os.environ['PATH']
directories, not the current directory or the sys.executable directory
as one could expect.
http://hg.python.org/cpython/file/64640a02b0ca/Lib/ctypes/util.py#l48
As a workaround, for distributions that place the HDF5 and other DLLs in
the tables package directory, tables.__init__.py adds the tables package
directory to os.environ['PATH']. This also makes sure that the DLLs are
found when loading the hdf5Extension.pyd and other C extension modules
(another common problem). The use of __file__ to get the tables
directory should better be wrapped in a try..except statement.
https://github.com/PyTables/PyTables/blob/develop/tables/__init__.py#L24
Christoph
Thanks, I'll fix it ASAP.
P.S.: please Christoph, do you have some hint for gh-175 [1]?
There is something we can do in PyTables?
[1] https://github.com/PyTables/PyTables/issues/175
--
Antonio Valentino
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
Re: [Pytables-users] New find_library('hdf5dll.dll') call fails in frozen app
On 8/27/2012 1:22 PM, Christoph Gohlke wrote:
On 8/27/2012 9:42 AM, Antonio Valentino wrote:
Hi Stuart,
Il 27/08/2012 17:43, Stuart Mentzer ha scritto:
Hello,
I upgraded to PyTables 2.4.0 and I was freezing an application on Windows
with PyInstaller. The frozen app fails at this new find_library call in
__init__.py:
if not ctypes.util.find_library('hdf5dll.dll'):
raise ImportError('Could not load hdf5dll.dll, please ensure'
+
' that it can be found in the system path')
PyInstaller correctly places this DLL in the same directory as the
application .exe where standard Windows DLL search logic will find it.
Apparently the find_library doesn't do that in a frozen application. That
is a big problem. I had to comment this code out to get a working frozen
app.
That code was added in revision e9f6919.
It is mainly a sanity check added under request of one of our users:
https://github.com/PyTables/PyTables/pull/146
This is on Windows 7 64-bit with a 32-bit Python toolchain. Trying both
PyInstaller 1.5.1 and 2.0.
Should I file a bug report? Any easy work-around?
Thanks,
Stuart
Yes please file a pull request with your patch.
It would be nice to preserve the sanity check in standard case so,
maybe, a good solution could be adding some check on sys.frozen or
something like that.
Thank you
Hello,
As a workaround for frozen distributions, try to add the sys.executable
directory to os.environ['PATH'] before importing tables.
Ctypes only tries to find a library in the os.environ['PATH']
directories, not the current directory or the sys.executable directory
as one could expect.
http://hg.python.org/cpython/file/64640a02b0ca/Lib/ctypes/util.py#l48
As a workaround, for distributions that place the HDF5 and other DLLs in
the tables package directory, tables.__init__.py adds the tables package
directory to os.environ['PATH']. This also makes sure that the DLLs are
found when loading the hdf5Extension.pyd and other C extension modules
(another common problem). The use of __file__ to get the tables
directory should better be wrapped in a try..except statement.
https://github.com/PyTables/PyTables/blob/develop/tables/__init__.py#L24
Christoph
Hi Christoph,
Thanks for the info/suggestions. It might be nice to add your comments to the
Issue #177 I created.
I was aware that altering the PATH is a work-around. Patching PyTables is
cleaner
and seems like the proper fix, and I think we agree that there is a problem here
that should be addressed. Maybe the PyTables test suite should even include a
frozen application test.
Thanks,
Stuart
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
Re: [Pytables-users] New find_library('hdf5dll.dll') call fails in frozen app
On 8/27/2012 1:52 PM, Antonio Valentino wrote:
Hi Christoph,
thank you very much for your hints.
Il 27/08/2012 19:22, Christoph Gohlke ha scritto:
On 8/27/2012 9:42 AM, Antonio Valentino wrote:
Hi Stuart,
Il 27/08/2012 17:43, Stuart Mentzer ha scritto:
Hello,
I upgraded to PyTables 2.4.0 and I was freezing an application on
Windows with PyInstaller. The frozen app fails at this new find_library
call in __init__.py:
if not ctypes.util.find_library('hdf5dll.dll'):
raise ImportError('Could not load hdf5dll.dll, please
ensure' +
' that it can be found in the system path')
PyInstaller correctly places this DLL in the same directory as the
application .exe where standard Windows DLL search logic will find it.
Apparently the find_library doesn't do that in a frozen application. That
is a big problem. I had to comment this code out to get a working frozen
app.
That code was added in revision e9f6919.
It is mainly a sanity check added under request of one of our users:
https://github.com/PyTables/PyTables/pull/146
This is on Windows 7 64-bit with a 32-bit Python toolchain. Trying both
PyInstaller 1.5.1 and 2.0.
Should I file a bug report? Any easy work-around?
Thanks,
Stuart
Yes please file a pull request with your patch.
It would be nice to preserve the sanity check in standard case so,
maybe, a good solution could be adding some check on sys.frozen or
something like that.
Thank you
Hello,
As a workaround for frozen distributions, try to add the sys.executable
directory to os.environ['PATH'] before importing tables.
Christoph, I suppose it can also be done at this point too:
https://github.com/PyTables/PyTables/blob/develop/tables/__init__.py#L24
Isn't it?
Please Stuart, can you try this fix as well.
Can't try it right now but I can't see why adding sys.excutable to the PATH at
that point
wouldn't work as an alternative to skipping the sanity check when frozen. My
broader
concern is whether fiddling with the PATH is the best or really the correct
fix. If you
need to exactly replicate Windows DLL search rules (which, just to make life
fun, vary
depending on the Windows version and the state of the SafeDllSearchMode*) then
you should
do that rather than hacking the PATH. For example, what if there is another
hdf5dll.dll
already on the PATH? In that case you'd better put sys.executable in the front
of the PATH,
but that might break something else. Maybe 99.9% of the time the PATH hack
works but
it can also fail.
I think the only proper fix is to make find_library follow the Windows rules
exactly.
Short of that, it is not safe to use it when the library might be in different
places,
such as frozen vs non-frozen applications.
[*]
http://msdn.microsoft.com/en-us/library/windows/desktop/ms682586%28v=vs.85%29.aspx
Ctypes only tries to find a library in the os.environ['PATH']
directories, not the current directory or the sys.executable directory
as one could expect.
http://hg.python.org/cpython/file/64640a02b0ca/Lib/ctypes/util.py#l48
As a workaround, for distributions that place the HDF5 and other DLLs in
the tables package directory, tables.__init__.py adds the tables package
directory to os.environ['PATH']. This also makes sure that the DLLs are
found when loading the hdf5Extension.pyd and other C extension modules
(another common problem). The use of __file__ to get the tables
directory should better be wrapped in a try..except statement.
https://github.com/PyTables/PyTables/blob/develop/tables/__init__.py#L24
Christoph
Thanks, I'll fix it ASAP.
P.S.: please Christoph, do you have some hint for gh-175 [1]?
There is something we can do in PyTables?
[1] https://github.com/PyTables/PyTables/issues/175
Stuart
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
Re: [Pytables-users] New find_library('hdf5dll.dll') call fails in frozen app
On 8/27/2012 10:58 AM, Stuart Mentzer wrote:
On 8/27/2012 1:22 PM, Christoph Gohlke wrote:
On 8/27/2012 9:42 AM, Antonio Valentino wrote:
Hi Stuart,
Il 27/08/2012 17:43, Stuart Mentzer ha scritto:
Hello,
I upgraded to PyTables 2.4.0 and I was freezing an application on
Windows with PyInstaller. The frozen app fails at this new find_library
call in __init__.py:
if not ctypes.util.find_library('hdf5dll.dll'):
raise ImportError('Could not load hdf5dll.dll, please
ensure' +
' that it can be found in the system path')
PyInstaller correctly places this DLL in the same directory as the
application .exe where standard Windows DLL search logic will find it.
Apparently the find_library doesn't do that in a frozen application. That
is a big problem. I had to comment this code out to get a working frozen
app.
That code was added in revision e9f6919.
It is mainly a sanity check added under request of one of our users:
https://github.com/PyTables/PyTables/pull/146
This is on Windows 7 64-bit with a 32-bit Python toolchain. Trying both
PyInstaller 1.5.1 and 2.0.
Should I file a bug report? Any easy work-around?
Thanks,
Stuart
Yes please file a pull request with your patch.
It would be nice to preserve the sanity check in standard case so,
maybe, a good solution could be adding some check on sys.frozen or
something like that.
Thank you
Hello,
As a workaround for frozen distributions, try to add the sys.executable
directory to os.environ['PATH'] before importing tables.
Ctypes only tries to find a library in the os.environ['PATH']
directories, not the current directory or the sys.executable directory
as one could expect.
http://hg.python.org/cpython/file/64640a02b0ca/Lib/ctypes/util.py#l48
As a workaround, for distributions that place the HDF5 and other DLLs in
the tables package directory, tables.__init__.py adds the tables package
directory to os.environ['PATH']. This also makes sure that the DLLs are
found when loading the hdf5Extension.pyd and other C extension modules
(another common problem). The use of __file__ to get the tables
directory should better be wrapped in a try..except statement.
https://github.com/PyTables/PyTables/blob/develop/tables/__init__.py#L24
Christoph
Hi Christoph,
Thanks for the info/suggestions. It might be nice to add your comments to the
Issue #177 I created.
I was aware that altering the PATH is a work-around. Patching PyTables is
cleaner
and seems like the proper fix, and I think we agree that there is a problem
here
that should be addressed. Maybe the PyTables test suite should even include a
frozen application test.
Thanks,
Stuart
Hi Stuart,
I'll try to work on a patch tonight. It's probably better to use Ctypes
LoadLibrary instead of find_library because that makes sure all HDF5
dependencies are found, it (supposedly) searches the Windows DLL search
path not just os.environ['PATH'], and (supposedly) takes into account
libraries already loaded into the process.
Christoph
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
Re: [Pytables-users] New find_library('hdf5dll.dll') call fails in frozen app
On 8/27/2012 2:45 PM, Christoph Gohlke wrote:
On 8/27/2012 10:58 AM, Stuart Mentzer wrote:
On 8/27/2012 1:22 PM, Christoph Gohlke wrote:
On 8/27/2012 9:42 AM, Antonio Valentino wrote:
Hi Stuart,
Il 27/08/2012 17:43, Stuart Mentzer ha scritto:
Hello,
I upgraded to PyTables 2.4.0 and I was freezing an application on
Windows with PyInstaller. The frozen app fails at this new find_library
call in __init__.py:
if not ctypes.util.find_library('hdf5dll.dll'):
raise ImportError('Could not load hdf5dll.dll, please
ensure' +
' that it can be found in the system path')
PyInstaller correctly places this DLL in the same directory as the
application .exe where standard Windows DLL search logic will find it.
Apparently the find_library doesn't do that in a frozen application. That
is a big problem. I had to comment this code out to get a working frozen
app.
That code was added in revision e9f6919.
It is mainly a sanity check added under request of one of our users:
https://github.com/PyTables/PyTables/pull/146
This is on Windows 7 64-bit with a 32-bit Python toolchain. Trying both
PyInstaller 1.5.1 and 2.0.
Should I file a bug report? Any easy work-around?
Thanks,
Stuart
Yes please file a pull request with your patch.
It would be nice to preserve the sanity check in standard case so,
maybe, a good solution could be adding some check on sys.frozen or
something like that.
Thank you
Hello,
As a workaround for frozen distributions, try to add the sys.executable
directory to os.environ['PATH'] before importing tables.
Ctypes only tries to find a library in the os.environ['PATH']
directories, not the current directory or the sys.executable directory
as one could expect.
http://hg.python.org/cpython/file/64640a02b0ca/Lib/ctypes/util.py#l48
As a workaround, for distributions that place the HDF5 and other DLLs in
the tables package directory, tables.__init__.py adds the tables package
directory to os.environ['PATH']. This also makes sure that the DLLs are
found when loading the hdf5Extension.pyd and other C extension modules
(another common problem). The use of __file__ to get the tables
directory should better be wrapped in a try..except statement.
https://github.com/PyTables/PyTables/blob/develop/tables/__init__.py#L24
Christoph
Hi Christoph,
Thanks for the info/suggestions. It might be nice to add your comments to the
Issue #177 I created.
I was aware that altering the PATH is a work-around. Patching PyTables is
cleaner
and seems like the proper fix, and I think we agree that there is a problem
here
that should be addressed. Maybe the PyTables test suite should even include a
frozen application test.
Thanks,
Stuart
Hi Stuart,
I'll try to work on a patch tonight. It's probably better to use Ctypes
LoadLibrary instead of find_library because that makes sure all HDF5
dependencies are found, it (supposedly) searches the Windows DLL search
path not just os.environ['PATH'], and (supposedly) takes into account
libraries already loaded into the process.
Christoph
Great. I'll be happy to test a patch in the frozen context.
Stuart
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
