Re: [Zope-DB] Working with ZPsycopgDA (or any DA) - commiting
Am 11.08.2006, 16:56 Uhr, schrieb <[EMAIL PROTECTED]>: Hi I'm working with a Postgresql database which I have to insert a lot of data to... I'm using the ZPsycopgDA database adapter, and ZSQL methods. I have noticed that though I am calling the zsql method no data gets really commited to the database. It all gets entered to database at the very end. This behavior seems reasonable for keeping database consistent, but in my situation it is really not so good - I have couple houndreds megabytes of data and as I believe it all gets stored in memory before being sent to the database causing serious system lags. I could be wrong though - this is only my observation... What could be another reason for slooow inserts? Either way, it would be far better for me if every "INSERT" would be commited immediatelly or better yet if I could force a "commit" within my code. Is there some way I can do it? Maciej, if you want transactional integrity then be prepared to pay the price: do not adjust Zope's transaction settings. As Andreas says you can make some adjustments on the server to speed up the process but please note that Zope is infamously slow for SQL statements although this usually does not play an important part. If you are doing a one off import then you are better working directly with Python. If you do need to work within Zope then note that using bound parameters should give you an additional performance boost over anything you can set on the server. Charlie ___ Zope-DB mailing list Zope-DB@zope.org http://mail.zope.org/mailman/listinfo/zope-db
Re: [Zope-DB] Working with ZPsycopgDA (or any DA) - commiting
--On 11. August 2006 16:56:59 + [EMAIL PROTECTED] wrote: Hi :-) I'm working with a Postgresql database which I have to insert a lot of data to... I'm using the ZPsycopgDA database adapter, and ZSQL methods. I have noticed that though I am calling the zsql method no data gets really commited to the database. It all gets entered to database at the very end. The DA is usually tied to the transaction system of Zope. So this behaviour is totally perfect and usually highly desirable. This behavior seems reasonable for keeping database consistent, but in my situation it is really not so good - I have couple houndreds megabytes of data and as I believe it all gets stored in memory ZSQL methods are executed at the time they are called and immediately executed on the server. THat does not mean that the every ZSQL method causes an automatic commit after execution (see above). before being sent to the database causing serious system lags. You might google for "postgres autocommit mode". I could be wrong though - this is only my observation... What could be another reason for slooow inserts? What means slow inserts? Depending on the isolation level, changes are visible to other apps only after a commit..that's how databases usually work. Either way, it would be far better for me if every "INSERT" would be commited immediatelly or better yet if I could force a "commit" within my code. Is there some way I can do it? See above. You can configure autocommit mode either on the server side or on the client side with psycopg (check the autocommit() method of the connection object as returned through psycopg.connect()). You might extend the ZPsycopgDA sources in order to create a DB connection either with or without autocommit mode. -aj pgpNynVV7WGY5.pgp Description: PGP signature ___ Zope-DB mailing list Zope-DB@zope.org http://mail.zope.org/mailman/listinfo/zope-db
