Hi, I have a problem I have been trying to solve throughout the last month to no avail, I am hoping you will be able to help.
I am the creator of a free, open source peer–to–peer and encrypted forum
application called Aether. ( www.getaether.net ) I have recently released v1.1
version, but I am having significant problems packaging the app with
PyInstaller.
The first version of the app also used PyInstaller, which can be obtained at
the website. The new version is however much more performant mainly because I
have split the networking backend from the front–end GUI into two separate
processes, which is the reason I am having the problem with packaging.
Ampoule is a twisted plugin that in essence creates another Python process in
which to run code. It’s fairly straightforward, and it uses Twisted’s own
spawnProcess API, which seems to be supported by PyInstaller as far as my
Google–fu tells me.
When I package my app using PyInstaller, the main process works flawlessly.
When it tries to instantiate the second process, the instantiated process
breaks and the main process starts to create an infinite amount of subprocesses.
1) It is specified Ampoule behaviour to restart dead processes, that’s where
the repeated new processes come from. The second process dies very early in its
lifetime.
2) When I try to instantiate an empty process, which is a process with no code
provided to it, it still crashes. Ditto when I use the default Ampoule stub. So
whatever is throwing an exception or crashing is not in my code.
3) I believe I am providing the imports and it imports the modules required.
Here is the source code where Ampoule instantiates a worker process:
def spawnProcess(processProtocol, bootstrap, args=(), env={},
path=None, uid=None, gid=None, usePTY=0,
packages=()):
env = env.copy()
pythonpath = []
for pkg in packages:
p = os.path.split(imp.find_module(pkg)[1])[0]
# imp.find_module: look to the modules to find it,
# get the address [1] and get the former path. [0]
if p.startswith(os.path.join(sys.prefix, 'lib')):
# If it starts with usr/local or any other
# system dir, its already importable, no worries. ignore.
continue
pythonpath.append(p)
# if not, appendt to pythonpath array.
pythonpath = list(sets.Set(pythonpath))
# ensure everything is unique and convert to list.
pythonpath.extend(env.get('PYTHONPATH', '').split(os.pathsep))
env['PYTHONPATH'] = os.pathsep.join(pythonpath)
args = (sys.executable, '-c', bootstrap) + args
# childFDs variable is needed because sometimes child processes
# misbehave and use stdout to output stuff that should really go
# to stderr. Of course child process might even use the wrong FDs
# that I'm using here, 3 and 4, so we are going to fix all these
# issues when I add support for the configuration object that can
# fix this stuff in a more configurable way.
if IS_WINDOWS:
return reactor.spawnProcess(processProtocol, sys.executable, args,
env, path, uid, gid, usePTY)
else:
return reactor.spawnProcess(processProtocol, sys.executable, args,
env, path, uid, gid, usePTY,
childFDs={0:"w", 1:"r", 2:"r", 3:"w",
4:"r"})
As you can see, it’s mostly a wrapper to Twisted’s spawnProcess, with a few
additions to resolve the imports.
I have almost no visibility into this process as the child crashes before it
can log back anything or even return an exception. The process starter at the
main process does not return any exceptions.
What approach would you recommend to fix this issue? I have been stuck at this
phase for about a month, I am willing to give a shot at almost anything. It’s
also a project very dear to me—I’ve been working on this for one and a half
year now—so, whatever it takes…
Thank you,
Burak / @nehbit
signature.asc
Description: Message signed with OpenPGP using GPGMail
