[Zope3-Users] Re: mysql connection
Hi Roman, it depends on what you are trying to do: When you want a zope database adapter for mysql: look here http://svn.zope.org/mysqldbda/ If you want ORM than try zalchemy. http://pypi.python.org/pypi/z3c.zalchemy/0.2.1 If you need a kind of SQL-Expression than take a look at z3c.sqlalchemy. http://pypi.python.org/pypi/z3c.sqlalchemy/1.0.11 HTH Christian I am trying to connect to mysql. What is the preferred package to do that ? I am using zopeproject. I tried to add dependency to mysqldbda, but it fails. The 2nd edition of the book mentions mysql utility, but it is not installed by default it seems. Thanks, Roman [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
[Zope3-Users] security policy - help
Hello zopers, I'm following the Zope Book example to create custom components. I created a shelf component and a book component and I let you the guess about the containment relationship ;-) In the web admin interface I see my components, I create them and browse them, I see the books put in (on ? ) the shelf. BUT when I try to access via ftp, I get a Access denied error message, but look : ONLY if there are some books in the shelf; if it's empty, it gives me the empty listing without raising errors. Any hint ? Why the same components are accessible by an user via the web and NOT via the ftp ? Please don't send me to RTFM, it's a silly thing I can't stand I have to read the security part of the book for this ! And, further, in the log/access.log file I can see the http get requests but not the ftp accesses ? How do I get the ftp activities logged too ? Also, in the etc/zope.conf file, at the bottom I see this: # devmode # # Switches the Developer Mode on and off. # # Default: # devmode off # #devmode on what is developer mode ? What should I expect if I set it on ? Thanks anyone for any hint Bye Catonano ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Re: mysql connection
On Nov 14, 2007, at 12:01 AM, Christian Klinger wrote: Hi Roman, it depends on what you are trying to do: When you want a zope database adapter for mysql: look here http://svn.zope.org/mysqldbda/ Yes. Maybe it's a question about zopeproject. I added mysqldbda dependency to setup.py and ran buildout, but it fails to find it. What's the naming scheme for the libraries ? I have a copy downloaded from zope.org, but I am unsure where to install it manually in the zopeproject framework. The installation instructions in mysqldbda are obsolete (referring to cvs). So somehow this package doesn't fit neatly into the new easy way of doing things. I wondered if I should use it at all. If you want ORM than try zalchemy. http://pypi.python.org/pypi/ z3c.zalchemy/0.2.1 If you need a kind of SQL-Expression than take a look at z3c.sqlalchemy. http://pypi.python.org/pypi/z3c.sqlalchemy/1.0.11 HTH Christian I am trying to connect to mysql. What is the preferred package to do that ? I am using zopeproject. I tried to add dependency to mysqldbda, but it fails. The 2nd edition of the book mentions mysql utility, but it is not installed by default it seems. Thanks, Roman [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] - --- ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
[Zope3-Users] (solved?) Large mappings or sequences in ZODB eat all the memory
Christophe Combelles a écrit : Hello, What should I do to have a data structure which is memory scalable? Consider the following large btree: $ ./debugzope from BTrees.OOBTree import OOBTree root['btree']=OOBTree() for i in xrange(70): ... root['btree'][i] = tuple(range(i,i+30)) ... import transaction transaction.commit() Quit and restart ./debugzope Now I just want to know if some value is in the btree: 'value' in root['btree'].values() Ok, the story could be called: ZODB is great, but take care of what you do with persistency. There are 3 solutions to this problem. One ugly, one workaround, and the correct one. I found the ugly one; thanks to Dennis and Chris for pointing to the workaround and the correct one. The whole btree is raised to the memory, even when I do a simple loop such as: for i in root['btree']: ... pass (it's the same with items(), iteritems(), values(), itervalues().) 1) First the *ugly* one: I abort the transaction every N loops: import transaction a=0 for i in root['b']: ... a+=1 ... if not a % 5000: ... transaction.abort() ... That works, but that's definitely not the right thing to do, I suspect that by aborting the transaction in the middle of the read, someone else might be able to modify the btree before I've finished my read. (zodb experts, please confirm) 2) Now a good *workaround* (that I will eventually use, because it's too late for me to change the data structure of my app, and it happens to be the fastest solution). It's almost the same, except that instead of aborting the transaction, we periodically minimize the cache of the connection to the ZODB: a=0 for i in root['btree']: ... a+=1 ... if not a % 5000: ... root['btree']._p_jar.cacheMinimize() ... This way, the maximum memory used corresponds to 5000 tuples. 3) the *correct* solution is to store real persistent objects in the btree. (ie objects that derive from persistent.Persistent). That works , and eats zero memory. But it's slower than tuples. Non-persistent tuples are persisted because they are part of a persistent object, but they are considered an integral part of the btree, and not individual separate persistent objects. That's my understanding, however that does not really explain why looping over non-persistent objects in a btree should absolutely raise everything in the memory. And what about IIBTrees? (integers are not persistent by themselves) Christophe or compute the length len(root['btree']) (I'm already using some separate lazy bookkeeping for the length, but even if len() is time consuming for a btree, it should be possible from a memory point of view) This loads the whole btree in memory (~500MB), and that memory never gets released! If the btree grows, how will I be able to use it? (2GB) I've tried to scan the btree by using slices, using root['btree'].itervalues(min,max), and by trying to do some transaction.abort()/commit()/savepoint()/anything() between the slices. But every slice I parse allocates yet another amount of memory, and when the whole btree has been scanned using slices, it's like the whole btree was in memory. I've also tried with lists, the result is the same, except the memory gets eaten even quicker. What I understand is that the ZODB wakes up everything, and the memory allocator of python (2.4) never release the memory. Is there a solution or something I missed in the API of the ZODB or BTrees or python itself? thanks, Christophe ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] security policy - help
At 13.21 14/11/2007, catonano wrote: Hello zopers, I'm following the Zope Book example to create custom components. I created a shelf component and a book component and I let you the guess about the containment relationship ;-) In the web admin interface I see my components, I create them and browse them, I see the books put in (on ? ) the shelf. BUT when I try to access via ftp, I get a Access denied error message, but look : ONLY if there are some books in the shelf; if it's empty, it gives me the empty listing without raising errors. Any hint ? Why the same components are accessible by an user via the web and NOT via the ftp ? I forgot to explain that I'm trying to give my components a representation as files, as the example in the book ! That's why I try to see them in ftp ! Thabks anyhow ! Catonano ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] (solved?) Large mappings or sequences in ZODB eat all the memory
On Nov 14, 2007 1:40 PM, Christophe Combelles [EMAIL PROTECTED] wrote: That's my understanding, however that does not really explain why looping over non-persistent objects in a btree should absolutely raise everything in the memory. When using non-persistent objects, the records holding the leaf buckets of the tree actually contain the serialized values, so just loading the buckets (necessary for iteration to load the keys) loads the stored values. And what about IIBTrees? (integers are not persistent by themselves) The I in IIBTree and IOBTree means that the buckets actually store the intergers directly; they aren't Python integer objects in the storage. This makes them highly efficient at storing the integers, both in and out of memory. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
[Zope3-Users] Re: mysql connection
Roman Budzianowski schrieb: On Nov 14, 2007, at 12:01 AM, Christian Klinger wrote: Hi Roman, it depends on what you are trying to do: When you want a zope database adapter for mysql: look here http://svn.zope.org/mysqldbda/ Yes. Maybe it's a question about zopeproject. I added mysqldbda dependency to setup.py and ran buildout, but it fails to find it. What's the naming scheme for the libraries ? I have a copy downloaded from zope.org, but I am unsure where to install it manually in the zopeproject framework. The installation instructions in mysqldbda are obsolete (referring to cvs). So somehow this package doesn't fit neatly into the new easy way of doing things. I wondered if I should use it at all. I don´t know if there is an egg of mysqldbda. If there is not you can try to use the addtional buildout receipe infrae.subversion [1] which allows you to include a package from an svn tree. I don´t know zopeproject details, but i think this should be doable. HTH Christian [1] http://pypi.python.org/pypi?%3Aaction=searchterm=infrae.subversionsubmit=search If you want ORM than try zalchemy. http://pypi.python.org/pypi/z3c.zalchemy/0.2.1 If you need a kind of SQL-Expression than take a look at z3c.sqlalchemy. http://pypi.python.org/pypi/z3c.sqlalchemy/1.0.11 HTH Christian I am trying to connect to mysql. What is the preferred package to do that ? I am using zopeproject. I tried to add dependency to mysqldbda, but it fails. The 2nd edition of the book mentions mysql utility, but it is not installed by default it seems. Thanks, Roman [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
