Hi Dave,
Thanks for the references. We identified the cause of the slowdowns so I
just wanted to report back the findings. We were emitting a debug statement
from inside a loop (that processed about 35K items). Obviously, the debug
calls are not the issue per se, since they are also emitted when run as
source. However, instead of placing logging.getLogger('xyz') in each module
we have a central debug facility that uses inspect.getmodule() to determine
("guess") the name of the module that emitted the debug message. This
getmodule() calls os.path.isfile() or os.path.exists() which in turn calls
stat() using the OS's API, however it calls stat() on a file that only
exists in the pyinstaller build environment
(build/pyinstaller/out00-PYZ.pyz/volatility.debug). Thus, as a pyinstaller
exe, it calls stat() 35K times for a file that doesn't exist. We were able
to significantly speed up the binary's runtime by just creating (via touch)
an empty file named build/pyinstaller/out00-PYZ.pyz/volatility.debug since
stat() appears to be significantly faster on files that exist versus files
that don't. Of course, that was just for testing. It turns out
inspect.getfile() returns similar information as inspect.getmodule() but it
doesn't look for a non-existing file in the pyinstaller build
directory....so we will probably use that instead....and/or disable the
debug call inside the processing loop.
Cheers,
MHL
On Wednesday, June 3, 2015 at 9:24:43 PM UTC-5, davecortesi wrote:
>
>
>
> On Wednesday, June 3, 2015 at 7:23:56 PM UTC-7, davecortesi wrote:
>>
>> Regarding imports, in the discussion on an issue at github[1], it was
>> pointed out how the bootloader sets up sys.path,
>>
>> # Append lib directory at the end of sys.path and not at the beginning.
>>
>> # Python will first try necessary libraries from the system and fallback
>>
>> # to the lib directory.
>>
>>
>> This would mean that your program imports whatever is locally installed,
>> if any, in preference to what was embedded in the executable. In a dev
>> environment, they should be the same. Whatever PyInstaller found while
>> bundling, should still be there. However, is there any possibility that
>> when you are testing the bundled app, it is picking up a different version
>> of some lib?
>>
>
>
> Left out the link [1]
> https://github.com/pyinstaller/pyinstaller/pull/1231#issuecomment-108670045
>
>
--
You received this message because you are subscribed to the Google Groups
"PyInstaller" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/pyinstaller.
For more options, visit https://groups.google.com/d/optout.
