On 13 September 2012 18:36, Michael McCracken <michael.mccrac...@gmail.com>wrote:
> FWIW, here's how I do something similar now, to avoid having many > copies of the Qt libraries. > There is one master app, and several sub-apps. > > * call setup() for each of the apps, generating full separate apps > with copies of the Qt libraries and other stuff > -- each call has a unique directory sent in the setup options > "bdist_base" and "dist_dir". This avoids sharing build state, as > mentioned earlier in the thread. This seems to work fine, calling > setup() multiple times in the same script. > > * inside main app Contents/Resources, create empty sub.app/ directory > * for each sub app: > ** copy sub.app/Contents/MacOS into > main.app/Contents/Resources/sub.app/Contents/MacOS > ** copy sub.app/Contents/Info.plist into > main.app/Contents/Resources/sub.app/Contents/Info.plist > ** copy everything in sub.app/Contents/Resources/ that isn't include > or lib into main.app/Contents/Resources/sub.app/Contents/Resources/ > ** create symlinks for sub.app/Contents/Resources/include -> > main.app/Contents/Resources/include (do the same with Resources/lib) > 3. merge the contents of > sub.app/Contents/Resources/lib/python*/site-packages.zip into the main > app's copy using zipfile > 4. merge the contents of the > sub.app/Contents/Resources/lib/python*/lib-dynload directory into the > main app's directory > > NOTE: this will ignore any packages in > sub.app/Contents/Resources/lib/python*/ that aren't in the > site-packages.zip. > That's fine for me, since I have the same "packages" option for all of the > apps. > > -mike > > That's pretty similar to what I do, I'm interested to know though how you do this tough 'each call has a unique directory sent in the setup options "bdist_base" and "dist_dir"', that might solve my problem. > > On Thu, Sep 13, 2012 at 1:53 AM, Ronald Oussoren <ronaldousso...@mac.com> > wrote: > > > > On 13 Sep, 2012, at 10:47, Paul Wiseman <poal...@gmail.com> wrote: > > > > On 13 September 2012 07:18, Ronald Oussoren <ronaldousso...@mac.com> > wrote: > >> > >> > >> On 10 Sep, 2012, at 16:37, Paul Wiseman <poal...@gmail.com> wrote: > >> > >> Ah, > >> > >> I've found out how to recreate the error > >> > >> If I create a main.py with nothing but 'import sqlalchemy' > >> > >> then use the following setup.py: > >> > >> from setuptools import setup > >> > >> setup( > >> version="1", > >> name="TestApp1", > >> app=["main.py"], > >> setup_requires=["py2app"] > >> ) > >> > >> setup( > >> version="1", > >> name="TestApp2", > >> app=["main.py"], > >> setup_requires=["py2app"] > >> ) > >> > >> If it doesn't produce the error it's probably because of this: "The > >> "cprocessors" module in SQLAlchemy is written in C and compiles > >> conditionally, based on if the current platform supports compiling it. > If > >> not present, SQLAlchemy continues to work perfectly well as the hooks > which > >> attempt to import it will fall back to pure-Python functions instead." > So > >> you may have a cprocessors.py which I dont think you'd get the problem, > only > >> if it compiled the .so when sqlalchemy installed. > >> > >> > >> I had the cprocessors extension in my build (that is, py2app mentioned > in > >> copied the extension) > >> > >> > >> I get the error, but only when it builds the second app. In my main > build > >> script I make a few apps in the same script (I make 3 apps which get > moved > >> into the main app, any additional code in their site-packages.zip is > moved > >> into the main apps zip, I remove the "sub-apps" Contents/Resources/lib > >> folder and symlink it at run time to the main apps lib folder.) > >> > >> Is this a bug or are you never supposed to run multiple setups in the > same > >> build? If not how can I achieve the above? > >> > >> > >> Calling distutils.setup multiple times is at best untested with py2app, > >> and I wouldn't be surprised if it causes problems due to state leaking > from > >> one build into the next. A workaround would be to use the subprocess > module > >> to run the setup jobs in separate processes. > >> > > > > Isn't the problem that they share dist folders, not a process? if not > where > > does the state exist? Would I need to subprocess them from different > > directories? > > > > > > The py2app command itself has state and I haven't reviewed the code to > know > > for sure that all state is cleaned up when the command is used twice in > the > > same process. BTW. I also don't know if distutils.setup creates a new > py2app > > command object every time it is called, if the second call to > > distutils.setup creates a new py2app command object there is no > information > > leakage. > > > > > >> > >> BTW. I don't quite understand what you are trying to do with these 3 > apps. > >> Are you building 3 apps that share a lot of code/resources and where you > >> want two of the apps to link to the 3th one to reduce the amount of > >> disk-space used? > >> > > > > Yea exactly, I have some smaller apps which are used for specific > separate > > jobs (one has a simple gui and generates and gathers log files from the > main > > app and zips them up should the main app ever fail to open for instance), > > the jobs are all to do with the main app and all use a sub set of code to > > the main app, so I put the apps in the Resources folder and symlink the > lib > > folder so I can include them with only using a little extra disk space, > but > > more importantly keeping the installer size down. > > > > > > That sounds like something that would be useful to support directly. I'll > > add it to the list of nice-to-have featuers, but don't know when I'll get > > around to looking into this. > > > > Ronald > > > > > >> > >> Ronald > >> > >> > >> On 10 September 2012 13:18, Ronald Oussoren <ronaldousso...@mac.com> > >> wrote: > >>> > >>> > >>> On 9 Sep, 2012, at 20:34, Paul Wiseman <poal...@gmail.com> wrote: > >>> > >>> Hey, > >>> > >>> When building an app that is using sqlalchemy I get this error: > >>> > >>> creating python loader for extension 'sqlalchemy.cprocessors' > >>> error: > >>> > /Users/paul/Source/Python/build/bdist.macosx-10.6-intel/python2.7-standalone/app/temp/sqlalchemy/cprocessors.py: > >>> No such file or directory > >>> > >>> I took a look in site packages and there is no cprocessors.py, but a > >>> cprocessors.so - so maybe it is just looking for the wrong extension > >>> > >>> I tried adding "sqlalchemy.cprocessors" to the includes list in py2app > >>> but that hasn't helped. > >>> > >>> I was wondering if I can fool it by dropping an empty cprocessors.py so > >>> it will build, then swap it out afterwards for the so, but I'm sure > there's > >>> a better way and I'm not convinced that could even work. > >>> > >>> Surely py2app doesn't assume every extension is .py, or if it does can > it > >>> be changed? > >>> > >>> > >>> Py2app does not assume that every extension is a python file. Given the > >>> messasge I'd say that the error occurs in the code path that creates a > >>> helper python file that actually loads the exention. > >>> > >>> A little background information: when py2app creates the application > >>> bundle all modules are stored in a zipfile and loaded using python's > >>> zipimporter. Extensions cannot be stored in the zipfiles because the > >>> zipimporter doesn't support that. Py2app therefore creates a > placeholder > >>> python module in the zipfile that loads the extensions from a > directory in > >>> the application bundle. > >>> > >>> BTW. could you please create a sample project that demonstrates the > >>> problem? I've tried to reproduce your problem on my machine and failed > to do > >>> so. I did run into another problem, py2app generated an incomplete > bundle > >>> due to confusion between a _collections submodule in SQLAlchemy and the > >>> _collections extension in the stdlib; that's something I'm currently > trying > >>> to fix. > >>> > >>> Ronald > >>> > >>> _______________________________________________ > >>> Pythonmac-SIG maillist - Pythonmac-SIG@python.org > >>> http://mail.python.org/mailman/listinfo/pythonmac-sig > >>> unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG > >>> > >>> > >> > >> > > > > > > > > _______________________________________________ > > Pythonmac-SIG maillist - Pythonmac-SIG@python.org > > http://mail.python.org/mailman/listinfo/pythonmac-sig > > unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG > > >
_______________________________________________ Pythonmac-SIG maillist - Pythonmac-SIG@python.org http://mail.python.org/mailman/listinfo/pythonmac-sig unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
