Re: [Zope] Re: MemoryError exception when importing a zexp file
(Wed, May 30, 2007 at 10:31:31AM +0100) Ben Lobo wrote/schrieb/egrapse: Thanks for the tip. I tried increasing the swap file to 10GB but still got the same memory error. Some operating systems limit the amount of RAM a single process can consume. Sometimes that amount can be adjusted. Also 32bit systems can only use a limited amount of RAM per process due to the limit of what you can address with 32bits -- over that amount you can give the system 100GB or 1000GB of swap, it won't change a thing. Even if those do not apply, I guess it's very much possible for a process to eat up all the RAM, no matter how much that is, all it would need is a little bit of recursion. I'm now considering exporting the Plone site (cos that's what it is) one piece at a time and reimporting it into a clean Plone site. Any thoughts on whether or not this is a good idea? Not mentioning Plone in the first place is problematic. In fact *plone* is problematic. Did you verify with plohn people (e.g. on a Plone mailing list) that current plone versions can be imported/exported? (Because the last time I made the mistake to do a project with plone you couldn't import/export a plone site -- or rename it, move it to another instance or anything similar for that matter -- without major pain. It's been a while though, so things could have changed.) Summary: Go and ask the plone people, if you can't import anyway, the whole exercise is in vain. Good luck! Regards, Sascha ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: MemoryError exception when importing a zexp file
On Thu, May 31, 2007 at 09:55:26AM +0200, Sascha Welter wrote: ... Did you verify with plohn people (e.g. on a Plone mailing list) that current plone versions can be imported/exported? (Because the last time I made the mistake to do a project with plone you couldn't import/export a plone site -- or rename it, move it to another instance or anything similar for that matter -- without major pain. It's been a while though, so things could have changed.) I spent several hours recently to import a .zexp of a Plone 2.0 site. Got nowhere. One suggestion I got (which I wasn't able to follow) was the following. If you have access to the original server, try this: 1) Make a full backup of the entire ZODB, eg. using repozo.py. 2) Set up a new Zope instance with all the same products installed, using your new backup copy of the ZODB. 3) Go into the ZMI for this new instance, and delete everythign EXCEPT this plone site. 4) Pack the database to get rid of all the other stuff. Or make a new clean backup. There. You now have a Data.fs that contains only your Plone site. Instead of importing the .zexp, just mount the database at the appropriate path. Disclaimer: I haven't tried this yet. Summary: Go and ask the plone people, if you can't import anyway, the whole exercise is in vain. +1 -- Paul Winkler http://www.slinkp.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: MemoryError exception when importing a zexp file
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 30 May 2007, at 11:31, Ben Lobo wrote: Thanks for the tip. I tried increasing the swap file to 10GB but still got the same memory error. I'm now considering exporting the Plone site (cos that's what it is) one piece at a time and reimporting it into a clean Plone site. Any thoughts on whether or not this is a good idea? I guess the first question should be why you are doing that export/ import at all? Is there any good reason for it? jens -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (Darwin) iD8DBQFGXUuhRAx5nvEhZLIRAg8LAJ9EvvNAMjCw8M6KhmnXv6gO23A2iwCfQmvi 00br+QjNySuhxGDF4QQ9dXg= =qYau -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Re: MemoryError exception when importing a zexp file
Hi, Failing with 10 GB seems weird... Did you monitor real memory usage to see where it failed? I'm thinking maybe the python interpreter / zope / something is limiting its memory usage (IIRC, Java VMs are limited to 128M by default) Could someone confirm/infirm this? I don't have any experience w/ Plone, so I don't know whether it would work, but I imagine you might have to re-import the pieces in the right order, so that there wouldn't be any broken dependencies (just speculating here) I don't know what get's exported in the .zexp file, but maybe packing the database to remove old versions of the Zope objects will reduce the size (not sure whether the history is included in the export... but maybe) Cheers Jonathan -Message d'origine- De : [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] De la part de Ben Lobo Envoyé : mercredi 30 mai 2007 11:32 À : zope@zope.org Objet : [Zope] Re: MemoryError exception when importing a zexp file Thanks for the tip. I tried increasing the swap file to 10GB but still got the same memory error. I'm now considering exporting the Plone site (cos that's what it is) one piece at a time and reimporting it into a clean Plone site. Any thoughts on whether or not this is a good idea? Cheers Ben Winterflood, Jonathan wrote: Hi, Maybe you could try giving the machine more swap space (partition in Linux, file in Windows) You'd probably have performance issues, but if the import is a one-timer, it might be acceptable. Jonathan -Message d'origine- De : [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] De la part de Jens Vagelpohl Envoyé : mardi 29 mai 2007 09:09 À : zope list user Objet : Re: [Zope] MemoryError exception when importing a zexp file -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 29 May 2007, at 08:36, Ben Lobo wrote: Hi, I'm trying to import a .zexp file that's 1.6GB but it continually fails due to a MemoryError exception. I guess that adding more RAM would probably fix this problem but does anyone know if there's any way of getting around this sort of problem without increasing the RAM? Short of increasing memory on that specific host you could also create a sandbox on a different machine with more RAM and then copy the database when it's done. jens This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: MemoryError exception when importing a zexp file
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 30 May 2007, at 12:12, Winterflood, Jonathan wrote: Hi, Failing with 10 GB seems weird... Did you monitor real memory usage to see where it failed? I'm thinking maybe the python interpreter / zope / something is limiting its memory usage (IIRC, Java VMs are limited to 128M by default) Could someone confirm/infirm this? There is no such limiting mechanism anywhere in the stack. However, there may always be OS-level limitations imposed by the systems administrator. jens -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (Darwin) iD8DBQFGXVxkRAx5nvEhZLIRAuZbAJ90Y6NLwZVo/AErmNbGGhIfUzdLmACfZbd4 GAgyEd3UI+0h5vj6820oIA0= =Igdw -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Re: MemoryError exception when importing a zexp file
Winterflood, Jonathan wrote at 2007-5-30 12:12 +0200: ... Failing with 10 GB seems weird... Did you monitor real memory usage to see where it failed? A 32 bit Linux will limit the amount of virtual memory per process to somewhat between 2.5 and 3 GB -- for reasons of address bit restrictions: With 32 bit addresses you can only address about 4 GB and parts of the address space is reserved for other purposes (than heap). You need a 64 bit architecture to have a significantly higher virtual address space. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
