Thanks for taking a look Giovanni;
As a follow up to this, I found that I could get around this problem
by restructuring the startup of processes so that there is only one at
a time being started. For example, my code to start the processes
used to be structured like this:
for i in xrange(ncpus):
p = mp.Process(target=worker.ff, args=(TaskQueue,DoneQueue))
p.start()
processes.append(p)
But if I guarantee that process "p" is fully started up and has gone
into a wait state by the time I try to start another in this loop (I
can post how I did it, if it matters; all I did was to create a 'pipe'
between the master and worker process, and receive back a 'ready to
go' notification through the pipe on the master, before trying to
start the next process), then I have none of these collision problems.
Of course, that's a crude workaround and it makes my startup times
really long, but it does work. If it matters, I'm using the one-dir
method in pyinstaller.
In the one-dir method, in the decompression of the pyz, isn't it
possible that one process is just overwriting another's files? I
can't quite tell from the docs how the decompression happens in a one-
dir installation. Whatever temp files are being formed there,
couldn't we just append a process ID to the name to avoid a collision
(if that is indeed what is happening..it seems like it).
On Nov 23, 2:30 pm, Giovanni Bajo <[email protected]> wrote:
> On Sat, 2010-11-20 at 08:12 -0800, dhyams wrote:
> > When I start up the code, though (at which time I also spawn the
> > processes), I get the attached stack trace (at bottom of email). I
> > think what is going on is that each process is trying to unpack to the
> > same spot, but I'm not sure how to adjust where things are being
> > unpacked to. The _MEIPASS2 variable seems to be just one to inform,
> > not adjust...I presume that the deciding on where to unpack is
> > determined in the bootloader.
>
> > Further evidence that it's a 'unpackaging collision' is that it's not
> > the same module that fails every time....it's just a random one in my
> > list of imports.
>
> > Is there a workaround for this?
>
> Not really. What you say should not happen because the bootloader
> removes the _MEIPASS2 variable after it's used and before the
> application starts (see launch.c, and search for MEIPASS2).
>
> It's still visible at the Python level though (and used by rthooks and
> people wanting to know where DATA files have been unpacked), for reasons
> that elude me right now. So there is surely something weird going on
> here.
> --
> Giovanni Bajo :: [email protected]
> Develer S.r.l. :: http://www.develer.com
>
> My Blog:http://giovanni.bajo.it
> Last post: Compile-time Function Execution in D
--
You received this message because you are subscribed to the Google Groups
"PyInstaller" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/pyinstaller?hl=en.
