Re: [ZODB-Dev] 32-bit vs 64-bit - RelStorage on MySQL
On 18/11/2010 18:48, Hanno Schlichting wrote: > For Zope based applications I've generally seen 50% to 100% memory > growth when moving to 64bit. That's why we stick to 32bit for a number > of memory hungry applications (*wink* Plone). Interestingly, we actually moved some plone based apps to amd64 machines to allow them to grow beyond 3GB (as PAE only allows up to 3GB per process), as they simply couldn't operate properly with only 3GB. But we have bucketloads of ram on those machines, so the increase in memory usage is probably way less important than having access to more of it :-) ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 32-bit vs 64-bit - RelStorage on MySQL
On 11/18/2010 09:54 AM, Chris Withers wrote: > On 18/11/2010 16:48, Hanno Schlichting wrote: >> On Thu, Nov 18, 2010 at 5:19 PM, Leonardo Santagada >> wrote: >>> On Thu, Nov 18, 2010 at 1:47 PM, Chris Withers >>> wrote: On 18/11/2010 15:39, Marius Gedminas wrote: > About the only noticeable difference -- other than the obvious memory > growth What obvious memory growth? >>> >>> The one from pointers and anything related to memory going from 32bits >>> to 64bits in size. Py_objects get fatter because of that. >> >> For Zope based applications I've generally seen 50% to 100% memory >> growth when moving to 64bit. That's why we stick to 32bit for a number >> of memory hungry applications (*wink* Plone). > > Right, but, even if the MySQL server is 64-bit, if the clients are > 32-bit, they won't have this problem, will they? Ok, you have a 32 bit Python client connecting to a 64 bit MySQL server, right? I don't expect any problems. All the pickling and unpickling will be done on 32-bit clients. MySQL sees the pickles as opaque binary streams. The only remaining question is whether the MySQL wire protocol can handle your setup. I suspect it can. If it can't, then you'll be filing MySQL bugs, since RelStorage, Python, and even the MySQL-Python DB-API adapter are all ignorant of the MySQL wire protocol. Shane ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 32-bit vs 64-bit - RelStorage on MySQL
> Right, but, even if the MySQL server is 64-bit, if the clients are > 32-bit, they won't have this problem, will they? if your running a 64bit python - you will see substantial memory growth. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 32-bit vs 64-bit - RelStorage on MySQL
On 18/11/2010 16:48, Hanno Schlichting wrote: > On Thu, Nov 18, 2010 at 5:19 PM, Leonardo Santagada > wrote: >> On Thu, Nov 18, 2010 at 1:47 PM, Chris Withers >> wrote: >>> On 18/11/2010 15:39, Marius Gedminas wrote: About the only noticeable difference -- other than the obvious memory growth >>> >>> What obvious memory growth? >> >> The one from pointers and anything related to memory going from 32bits >> to 64bits in size. Py_objects get fatter because of that. > > For Zope based applications I've generally seen 50% to 100% memory > growth when moving to 64bit. That's why we stick to 32bit for a number > of memory hungry applications (*wink* Plone). Right, but, even if the MySQL server is 64-bit, if the clients are 32-bit, they won't have this problem, will they? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 32-bit vs 64-bit - RelStorage on MySQL
On Thu, Nov 18, 2010 at 5:19 PM, Leonardo Santagada wrote: > On Thu, Nov 18, 2010 at 1:47 PM, Chris Withers wrote: >> On 18/11/2010 15:39, Marius Gedminas wrote: >>> About the only noticeable difference -- other than the obvious memory >>> growth >> >> What obvious memory growth? > > The one from pointers and anything related to memory going from 32bits > to 64bits in size. Py_objects get fatter because of that. For Zope based applications I've generally seen 50% to 100% memory growth when moving to 64bit. That's why we stick to 32bit for a number of memory hungry applications (*wink* Plone). Hanno ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 32-bit vs 64-bit - RelStorage on MySQL
On Thu, Nov 18, 2010 at 1:47 PM, Chris Withers wrote: > On 18/11/2010 15:39, Marius Gedminas wrote: >> About the only noticeable difference -- other than the obvious memory >> growth > > What obvious memory growth? The one from pointers and anything related to memory going from 32bits to 64bits in size. Py_objects get fatter because of that. -- Leonardo Santagada ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 32-bit vs 64-bit - RelStorage on MySQL
On 18/11/2010 15:39, Marius Gedminas wrote: > About the only noticeable difference -- other than the obvious memory > growth What obvious memory growth? I guess I'm more concerned with a mix of 32 and 64 bit in a RelStorage setup... Does that mean my question is essentially "do 32-bit MySQL clients play nicely with a 64-bit MySQL server" or is it more complicated than that? Do pickles go nicely across 32-bit/64-bit boundaries? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 32-bit vs 64-bit
On Thu, Nov 18, 2010 at 03:25:17PM +, Chris Withers wrote: > Hi All, > > Are there any intricacies with migrating a ZODB-backed Zope 2 app from > 32-bit to 64-bit machines? I haven't noticed any problems, when I moved my personal website from a 32-bit machine to a 64-bit one. About the only noticeable difference -- other than the obvious memory growth -- is that string hashes are computed differently, which changes the ordering of dict.keys(). Thankfully, pickles do not rely on that implementation detail. Marius Gedminas -- It's possible to describe anything in mathematical notation. I recall seeing some paper once in which someone had created a mathematical description of C. (I forget whether or not this included the preprocessor.) As an achievement, this is somewhat like building a full-size model of the Eiffel Tower out of tongue depressors. It's clearly not the act of a talentless man, but you have to wonder what he said when he applied for his grant. -- Mencius Moldbug signature.asc Description: Digital signature ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] 32-bit vs 64-bit
Hi All, Are there any intricacies with migrating a ZODB-backed Zope 2 app from 32-bit to 64-bit machines? The specific scenario I have in mind is moving a 32-bit set of ZODBs into MySQL-backed RelStorages where the MySQL server runs 64-bit but the clients are still 32-bit, although in one instance the clients will also be 64-bit. Any pointers/pitfalls/etc gratefully recieved! cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] RelStorage support to Microsoft SQLServer
> my suggestion: use mxodbc. involve shane so he can fold it into the main > relstorage line in the future. maybe even budget some money to pay him > so he has interest in the outcome of the mssql/odbc adapter for relstorage. We have made very good experience with mxODBC to connect to SQL Server. The ODBC module coming with Python Win is also an option on Windows, although it leads to (very) occasional crashes and has a reduced feature set. However, I don't use RelStorage nor ZODB; I've rolled my own pickle store on SQL Server, storing a subset of the attributes in relational tables for fast searches - better suits the peculiar needs of Quantax. Best regards, Martin COMIT AG - A Swisscom Company Martin Gfeller, Senior Manager http://www.comit.ch/, http://my.comit.ch/ http://www.quantax.com/ - Market Risk & Return ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
