[Zope-dev] Re: straighting out the SQLAlchemy integration mess
Martijn Faassen wrote: Hi there, [I originally picked this up on a thread on zope3-users, but this deserves its own thread here] There are at least three approaches to SQLAlchemy integration with Zope: * z3c.zalchemy (Christian Theune) * z3c.sqlalchemy (Andreas Jung) * collective.lead (Laurence Rowe) All of these are in various states of brokenness. z3c.zalchemy doesn't work with SQLAlchemy trunk. collective.lead works with it, but only if you check out a particular branch, and not with sqlite. Quite possibly z3c.sqlalchemy has a release that actually works. One out of three is not bad... :) Then there's also mentions about WSGI-based integration, and I think in Plone, Alchemist probably also does its own integration... There must be a reason for this proliferation of approaches. What is it? We all get along, don't we? I know that the various packages are taking code and approaches from each other too. Can't we work together more and at least come up with *one* package that works? Perhaps factor out some low-level commonality than then all share? Criticize one of the other packages until you're satisfied, and then retire your own package perhaps? I know the various packages add on their own approaches to configuration and might offer higher level container approaches. Those could be in different packages, sharing a foundation. In the end, I hope we will end up with just *one* integration layer, that is released, that works with Zope 2 and Zope 3 and a recent release of SQLAlchemy, that is documented, and that people know about. We can then offer packages on top of this that offer extra features. I'm all for one integration layer, even though I spurred one of the above (collective.lead). I would ask that we be very conservative, though. collective.lead wants to provide: - automatic transaction integration with ZODB transactions - an easy way to register your database connections - an easy way to look those up as utilities and get an ORM session Everything else (e.g. Zope ORM integration, configuration abstractions, etc) should be in some other package. collective.lead has a reasonable degree of traction in the Plone universe, at least, but should work just fine with plain Zope 3. We have other things that work on top it (mercury, rope, probably others). The branch that works with trunk also has some carefully worked out TPC support, which is difficult to get right, though I'm disappointed that we seem to have broken sqlite integration. Hopefully it's an easy fix, though. I'm CC'ing Laurence to ask: - can we get a release soon? - can we fix the sqlite integration? Cheers, Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] straighting out the SQLAlchemy integration mess
On Tue, Apr 8, 2008 at 11:54 PM, Martijn Faassen [EMAIL PROTECTED] wrote: All of these are in various states of brokenness. z3c.zalchemy doesn't work with SQLAlchemy trunk. collective.lead works with it, but only if you check out a particular branch, and not with sqlite. Quite possibly z3c.sqlalchemy has a release that actually works. One out of three is not bad... :) We are going into production with collective.lead and are committed to releasing the 1.0 branch as 1.0. sqlite works just great for us, we use it to run unit tests and for developers that just need to adjust the styling and such. The production environment will run against Oracle. There must be a reason for this proliferation of approaches. What is it? We all get along, don't we? I know that the various packages are taking code and approaches from each other too. Different use-cases and philosophies of development. In the end, I hope we will end up with just *one* integration layer, that is released, that works with Zope 2 and Zope 3 and a recent release of SQLAlchemy, that is documented, and that people know about. We can then offer packages on top of this that offer extra features. Sounds like a good idea. -- Martijn Pieters ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Non-ASCII characters in URLs
Dieter Maurer wrote: Wichert Akkerman wrote at 2008-4-7 20:45 +0200: ... Almost surely, Alexander wants to ask why Zope does not allow non-ASCII characters in ids. And, in fact, there are only two reasons: * lazyness of the Zope developpers: without the restriction to ASCII characters careful quoting (and unquoting) is necessary in order to adhere to RFC 2396 (the modern uri syntax specification) This is becoming increasingly painful I will soon have a patch against Zope 2.11b1 which gets rid of this restriction. If there is consense, I can add it to the Zope repository. +1 from my side. Saves me the work to cleanup my own dirty patch :-)) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] [Proposal] Allow (almost) arbitrary ids in Zope2
= Introduction = Zope2 allows only a very restricted set of characters in `ObjectManager` ids -- a subset of the characters which can be used unescaped in urls. Neither non-ASCII characters are allowed nor some quite important ASCII characters. When Zope is used outside the english language domain and accessed via WebDAV, then the restriction to ASCII characters only is heavily felt by WebDAV users, as they are accustomed to use readable names (with non-ASCII characters) in their file systems. They cannot understand why such objects cannot be transfered into Zope (especially given the most unhelpful error messages of some WebDAV clients). The limited set of allowed ASCII characters leads to other problems: e.g. email addresses cannot be used as ids as [EMAIL PROTECTED] is disallowed. = Feature = Allow almost arbitrary characters in ids. A few restrictions remain: * ids cannot start with a '_' because they would not be traversable * ids cannot contain '/' because Zope code uses `urllib.[un]quote` on complete urls (a bug, [un]quoting must only happen on the individual pieces, never the complete object) and `quote` by default interpretes `/` as safe (not to be quoted). Allowing '/' would risk too many breakages. Allowing non-ASCII characters in ids poses a risk, as the uri standard (RFC 2396) does not allow to specify the encoding used in urls. There is only HTML 4 which recommends that url components are first encoded as utf-8 and then url escaped as necessary. But this recommendation seems not to be widely used. However, when urls are constructed in a client (rather than directly fetched from an HTML source or given by a user) e.g. in Java or !JavaScript, then it is not unlikely that the recommendation is followed (as these systems work internally with unicode and need to use some encoding for url construction). I also have seen a client (the MS WebDAV client) to interpret (binary) urls in some encoding and then recode them with utf-8. I see two options: * Zope follows the HTML recommendation (and uses utf-8 in its generated urls). This implies that either ids need to be unicode or their encoding known (i.e. specifiable). * Zope does not follow the HTML recommendation and uses binary ids (with some unknown encoding) but uses some heuristics to handle HTML 4 compatibly generated urls. As I am mainly concerned with WebDAV access and the WebDAV clients I have seen so far use the binary id interpretation, I favor the second option. -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope Tests: 5 OK
Summary of messages to the zope-tests list. Period Tue Apr 8 11:00:00 2008 UTC to Wed Apr 9 11:00:00 2008 UTC. There were 5 messages: 5 from Zope Tests. Tests passed OK --- Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Tests Date: Tue Apr 8 21:05:25 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-April/009375.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Tests Date: Tue Apr 8 21:06:56 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-April/009376.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Tests Date: Tue Apr 8 21:08:26 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-April/009377.html Subject: OK : Zope-2.11 Python-2.4.4 : Linux From: Zope Tests Date: Tue Apr 8 21:09:56 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-April/009378.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Tests Date: Tue Apr 8 21:11:26 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-April/009379.html ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: AW: [Zope-dev] Re: [Zope3-Users] How do I automatically login a user
Below is a first draft of a proposal to cover the features discussed in
this thread. I have bundled a number of loosely related items into the
proposal. I can split them out to separate proposals if that is considered
a better approach.
There are three items which are not resolved in the high level of the
proposal. However, I believe that there is enough detail in the proposal
at the moment to elicit feedback on the Proposal scope and approach.
If this high-level is considered acceptable to the list, I will do an
implementation and submit a detailed proposal to this list, the Wiki and
Launchpad.
Thanks,
Kevin
Proposal
Session Credentials API Enhancements
o Support remote IP restricted logins
o Support domain specific logins
o Encrypt Password
o Application overriding credentials from request
o Tracking of 'logged in' users
Motivation(s)
Require an API for logging in a user automatically. The mailing list
identified a number of other related requirements. Some items were added
or expanded due to usefulness in Zope2.
Problem(s)
Secure authentication scenarios require access to be limited to a single
IP Address or set of IP Addresses. The Session Credential plugin should
take responsibility for extracting and returning the authentication IP
Address.
Some scenarios use a 'domain' and provide the login within that specific
domain (e.g. a company and users belonging to that company).
For security reasons the password should be encrypted while in the session
store.
The application needs an API so that it can set the credentials for a
session programmatically, i.e. as part of the sign up process.
For monitoring purposes, it is useful to know who is currently 'logged
in'. For strict security is important to logout idle users.
Proposal
1. IP Extraction
Extract the IP Address from the credentials and store it. Return the IP
Address in the dictionary from extractCredentials().
The value from request._environ['HTTP_X_FORWARDED_FOR'] will be used if
present. otherwise request._environ['REMOTE_ADDR'].
This requires a change to ISessionCredentials to provide getIP().
2. Domain
The Session Credentials can optionally provide a 'domain' value. Where the
domain is used the domain should be stored in the client using a cookie,
so that it can default.
The domain functionality should be enabled via the UI.
TODO: Need to rename the domain functionality as the term domain is used
for IP specific security in Zope2.
3. Encrypt password
The Session Credentials should use of an encrypter class to encrypt the
password if configured. The class is used to encrypt the password when
saved and decrypts it when returning the password to the application (via
extraCredentials).
The encrypter class shall support the following Interface:
class IEncryptPassword(Interface):
def encrypt(login, password):
Return an encrypted version of the password
def decrypt(login, encrypted):
Return a decrypted version of the password
TODO: Determine the mechanism for locating/configuring this class.
TODO: Identify a mechanism for recovering if you use this mechanism with
no users with encrypted passwords.
4. Application Overriding Login Credentials
The ICredentialsPlugin.extractCredentials method will have two new
parameter, overrides and mode.
overrides will provide a dictionary containing ('login', 'password' and
'domain') which will override the same values in the request object if
they are present in the request object.
mode will define what to do with the parameters if session credentials are
already stored:
STORED_REQUEST_MODE:If stored values, used them otherwise use the
request (current functionality)
REQUEST_STORED_MODE:Try for parameters in the request. If they are
not
present use stored values if they are available.
REQUEST_ONLY_MODE: Ignore values if they are stored. Use the
request only.
The class will use a full set of credentials from either the request or
storage. It will not mix them.
An adapter will be provided to make access to the login simpler. This
should be part of zope.app.security. The adapter shall adapt a request
object.
class ILogin(Interface):
Provide support for logging in directly
def login(overrides, mode=REQUEST_ONLY):
Login using credentials provided in the overrides
parameter. {'login':'', 'password': '', 'domain': ''}
If values are not provided in the dictionary, the
request
is searched
def setCredentials(overrides, mode=REQUEST_ONLY):
Store the credentials for the current session. Do not
[Zope-dev] Re: straighting out the SQLAlchemy integration mess
The transaction manager here: http://svn.plone.org/svn/collective/collective.lead/branches/elro-tpc/collective/lead/tx.py Has support for TPC and savepoints. It is tested and works. The only issue with sqlite is that the default in the branch is to use two-phase commit and sqlite does not support this. Apart from that the module is completely self contained. The public interfaces to it is the two functions at the bottom, join_transaction and dirty_session. The idea of the dirty_session stuff is to prevent unnecessary commits when data is only read. Session writes trigger the 'dirtying' automatically with a SessionExtension: http://svn.plone.org/svn/collective/collective.lead/branches/elro-tpc/collective/lead/database.py The downside of this approach is that the when you interact with the connection directly to modify data you must call dirty_session yourself. If this is a problem for you then you may set up the transaction to be in the dirty state initially. Thread based session management is used (scoped_session). There is fairly extensive testing of this and the transaction support here: http://svn.plone.org/svn/collective/collective.lead/branches/elro-tpc/collective/lead/tests.py I think it would be worth factoring out the threaded session support as well as the transactions. @Christian: I'm not quite sure what you mean by: - Database triggers as events I don't see how you could reasonably get a database trigger on the db server to instantiate a zope event on the db client. Perhaps the database could be persuaded to make a request to a URL on a trigger, but this seems a bit slow and maybe not a great solution. - Making SA 0.4 work with security proxies I don't know anything about security proxies in zope 3, but surely this is just a matter of what the base class is / the container implementation? I don't see why an SQL Alchemy mapped object would be any different to any other object. @Martin: Should one phase commit be set as the default to make it easier to work with sqlite (and mssql)? Probably yes. Should the default be for sessions to start out `active` or `dirty`? If they start out `dirty` then existing 1.0 code should work as before. It'll probably be next week before I manage to get this tidied up and merged back to trunk. @everyone: If we can all agree to use the same basic session and transaction management then we should probably push for it to be included as a sqlalchemy extension module. Laurence Martin Aspeli wrote: Martijn Faassen wrote: Hi there, [I originally picked this up on a thread on zope3-users, but this deserves its own thread here] There are at least three approaches to SQLAlchemy integration with Zope: * z3c.zalchemy (Christian Theune) * z3c.sqlalchemy (Andreas Jung) * collective.lead (Laurence Rowe) All of these are in various states of brokenness. z3c.zalchemy doesn't work with SQLAlchemy trunk. collective.lead works with it, but only if you check out a particular branch, and not with sqlite. Quite possibly z3c.sqlalchemy has a release that actually works. One out of three is not bad... :) Then there's also mentions about WSGI-based integration, and I think in Plone, Alchemist probably also does its own integration... There must be a reason for this proliferation of approaches. What is it? We all get along, don't we? I know that the various packages are taking code and approaches from each other too. Can't we work together more and at least come up with *one* package that works? Perhaps factor out some low-level commonality than then all share? Criticize one of the other packages until you're satisfied, and then retire your own package perhaps? I know the various packages add on their own approaches to configuration and might offer higher level container approaches. Those could be in different packages, sharing a foundation. In the end, I hope we will end up with just *one* integration layer, that is released, that works with Zope 2 and Zope 3 and a recent release of SQLAlchemy, that is documented, and that people know about. We can then offer packages on top of this that offer extra features. I'm all for one integration layer, even though I spurred one of the above (collective.lead). I would ask that we be very conservative, though. collective.lead wants to provide: - automatic transaction integration with ZODB transactions - an easy way to register your database connections - an easy way to look those up as utilities and get an ORM session Everything else (e.g. Zope ORM integration, configuration abstractions, etc) should be in some other package. collective.lead has a reasonable degree of traction in the Plone universe, at least, but should work just fine with plain Zope 3. We have other things that work on top it (mercury, rope, probably others). The branch that works with trunk also has some carefully worked out TPC support, which is difficult to get right,
Re: AW: [Zope-dev] Re: [Zope3-Users] How do I automatically login a user
On Apr 9, 2008, at 5:07 AM, kevin gill wrote: 1. IP Extraction Extract the IP Address from the credentials and store it. Return the IP Address in the dictionary from extractCredentials(). The value from request._environ['HTTP_X_FORWARDED_FOR'] will be used if present. otherwise request._environ['REMOTE_ADDR']. On a basis of privacy of attributes starting with underscore, such as _environ, I would suggest using request.headers (for X-Forwarded- For) and request.environment instead. These are defined in the public interface API. -- Zvezdan Petkovic [EMAIL PROTECTED] ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: straighting out the SQLAlchemy integration mess
Martijn Pieters wrote: On Tue, Apr 8, 2008 at 11:54 PM, Martijn Faassen [EMAIL PROTECTED] wrote: All of these are in various states of brokenness. z3c.zalchemy doesn't work with SQLAlchemy trunk. collective.lead works with it, but only if you check out a particular branch, and not with sqlite. Quite possibly z3c.sqlalchemy has a release that actually works. One out of three is not bad... :) We are going into production with collective.lead and are committed to releasing the 1.0 branch as 1.0. sqlite works just great for us, we use it to run unit tests and for developers that just need to adjust the styling and such. The production environment will run against Oracle. The elro-tpc branch (which, I was told, is the future) doesn't appear to work with sqlite out of the box. I'm curious how to do functional tests with collective.lead - I'd like to have some real easy way to get the database set up and tore down down in my tests. Are you doing this? Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: straighting out the SQLAlchemy integration mess
--On 9. April 2008 14:15:38 +0100 Laurence Rowe [EMAIL PROTECTED] wrote: @everyone: If we can all agree to use the same basic session and transaction management then we should probably push for it to be included as a sqlalchemy extension module. I would be happy with such a solution. As said, I am not into the datamanager-building business and more interested in application-integration issues. The basic requirements I have for our applications: - having the same session within one ZODB transaction (per thread) - need to get hold of the underlying database connection for a given session Andreas pgpFEZVYm6t1t.pgp Description: PGP signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] straighting out the SQLAlchemy integration mess
On Tue, Apr 8, 2008 at 5:54 PM, Martijn Faassen [EMAIL PROTECTED] wrote: Hi there, [I originally picked this up on a thread on zope3-users, but this deserves its own thread here] There are at least three approaches to SQLAlchemy integration with Zope: * z3c.zalchemy (Christian Theune) * z3c.sqlalchemy (Andreas Jung) * collective.lead (Laurence Rowe) All of these are in various states of brokenness. z3c.zalchemy doesn't work with SQLAlchemy trunk. collective.lead works with it, but only if you check out a particular branch, and not with sqlite. Quite possibly z3c.sqlalchemy has a release that actually works. One out of three is not bad... :) Then there's also mentions about WSGI-based integration, and I think in Plone, Alchemist probably also does its own integration... ore.alchemist works with sqlalchemy 0.4 series, and i've been doing z3 rdb only apps with it over the last year. i'm biased, but i think it has the easiest api integration of any of the others.. from ore.alchemist import Session thats it.. all you need for transaction integration with sa, no magic utilities to setup mappers, or tables. the rest is as documented by sqlalchemy. it goes on to provide additional zcml syntax for defining connections and binding them to sqlalchemy metadata, but usage is optional. as for the wsgi based integration, to get zope3 without the zodb, i use ore.wsgiapp which just defines a utility which is the traversal root. i'd like to refactor it to use the new zope.publisher paste support directly, as it currently it includes a bunch of boiler plate from a generated There must be a reason for this proliferation of approaches. What is it? We all get along, don't we? I know that the various packages are taking code and approaches from each other too. we all had different needs, some wanted to play in z2 land where broken egg requires are the name of the game, some in z3 land... some were just the first to arrive. i wanted and implemented higher level semantics to generate interfaces from tables, and vice versa, and automatically generate forms etc. Can't we work together more and at least come up with *one* package that works? Perhaps factor out some low-level commonality than then all share? Criticize one of the other packages until you're satisfied, and then retire your own package perhaps? I know the various packages add on their own approaches to configuration and might offer higher level container approaches. Those could be in different packages, sharing a foundation. In the end, I hope we will end up with just *one* integration layer, that is released, that works with Zope 2 and Zope 3 and a recent release of SQLAlchemy, that is documented, and that people know about. We can then offer packages on top of this that offer extra features. that sound good, i'd like to see a common base layer, providing transaction support, simple containers. cheers, kapil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: straighting out the SQLAlchemy integration mess
On Wed, Apr 9, 2008 at 9:15 AM, Laurence Rowe [EMAIL PROTECTED] wrote: The transaction manager here: http://svn.plone.org/svn/collective/collective.lead/branches/elro-tpc/collective/lead/tx.py Has support for TPC and savepoints. It is tested and works. The only issue with sqlite is that the default in the branch is to use two-phase commit and sqlite does not support this. Apart from that the module is completely self contained. The public interfaces to it is the two functions at the bottom, join_transaction and dirty_session. The idea of the dirty_session stuff is to prevent unnecessary commits when data is only read. Session writes trigger the 'dirtying' automatically with a SessionExtension: http://svn.plone.org/svn/collective/collective.lead/branches/elro-tpc/collective/lead/database.py The downside of this approach is that the when you interact with the connection directly to modify data you must call dirty_session yourself. If this is a problem for you then you may set up the transaction to be in the dirty state initially. Thread based session management is used (scoped_session). There is fairly extensive testing of this and the transaction support here: http://svn.plone.org/svn/collective/collective.lead/branches/elro-tpc/collective/lead/tests.py I think it would be worth factoring out the threaded session support as well as the transactions. +1, the collective.lead transaction support looks nice. @Christian: I'm not quite sure what you mean by: - Database triggers as events I don't see how you could reasonably get a database trigger on the db server to instantiate a zope event on the db client. Perhaps the database could be persuaded to make a request to a URL on a trigger, but this seems a bit slow and maybe not a great solution. - Making SA 0.4 work with security proxies I don't know anything about security proxies in zope 3, but surely this is just a matter of what the base class is / the container implementation? I don't see why an SQL Alchemy mapped object would be any different to any other object. its more to do with the fact that proxies are restrictive by default, and the sa api likes to monkey attributes on instances. christian kicked off some nice work with the sa developers at pycon to get this behavior configurable in sa ( instrumentation branch) which obviate the need to remove the proxies before calling into sa apis on an object. @Martin: Should one phase commit be set as the default to make it easier to work with sqlite (and mssql)? Probably yes. Should the default be for sessions to start out `active` or `dirty`? If they start out `dirty` then existing 1.0 code should work as before. It'll probably be next week before I manage to get this tidied up and merged back to trunk. @everyone: If we can all agree to use the same basic session and transaction management then we should probably push for it to be included as a sqlalchemy extension module. sounds good, or failing that (due to additional dependencies), a common base package. cheers, kapil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] straighting out the SQLAlchemy integration mess
Martijn Faassen wrote at 2008-4-8 23:54 +0200: ... There are at least three approaches to SQLAlchemy integration with Zope: Quite possibly z3c.sqlalchemy has a release that actually works. The current release is not very robust against some kinds of temporary problems. When e.g. the database is restarted, z3c.sqlalchemy may enter a state where all database interactions raise an exception until Zope is restarted. Andreas told me that he has a development version which (hopefully) fixes these kinds of problems. -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] straighting out the SQLAlchemy integration mess
--On 9. April 2008 20:28:18 +0200 Dieter Maurer [EMAIL PROTECTED] wrote: Martijn Faassen wrote at 2008-4-8 23:54 +0200: ... There are at least three approaches to SQLAlchemy integration with Zope: Quite possibly z3c.sqlalchemy has a release that actually works. The current release is not very robust against some kinds of temporary problems. When e.g. the database is restarted, z3c.sqlalchemy may enter a state where all database interactions raise an exception until Zope is restarted. This is very likely an issue of the underlying SA implementation. None of the existing wrappers deal with this kind of problems. In fact I've never seen such issues within applications under my control :-) -aj pgp3gi0Wmd3B5.pgp Description: PGP signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] straighting out the SQLAlchemy integration mess
Andreas Jung wrote at 2008-4-9 20:34 +0200: ... Quite possibly z3c.sqlalchemy has a release that actually works. The current release is not very robust against some kinds of temporary problems. When e.g. the database is restarted, z3c.sqlalchemy may enter a state where all database interactions raise an exception until Zope is restarted. This is very likely an issue of the underlying SA implementation. As I have explained to you, z3c.sqlalchemy has a big weakness when there are any exceptions from commit or abort: in this case, the SQLAlchemy resources session and connection are not cleaned up and remain associated with the thread. This thread will always get the unclean resources -- until a restart. That's something, z3c.sqlalchemy can improve -- independently from what SQLAlchemy does. -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: straighting out the SQLAlchemy integration mess
Laurence Rowe wrote: Should one phase commit be set as the default to make it easier to work with sqlite (and mssql)? Probably yes. Ideally we'd guess based on the URL scheme but allow it to be set explicitly, IMHO. Single phase would be the fallback, I guess. Should the default be for sessions to start out `active` or `dirty`? If they start out `dirty` then existing 1.0 code should work as before. What are the implications of having them start out dirty? It'll probably be next week before I manage to get this tidied up and merged back to trunk. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: straighting out the SQLAlchemy integration mess
Martijn Faassen wrote: Martijn Pieters wrote: On Tue, Apr 8, 2008 at 11:54 PM, Martijn Faassen [EMAIL PROTECTED] wrote: All of these are in various states of brokenness. z3c.zalchemy doesn't work with SQLAlchemy trunk. collective.lead works with it, but only if you check out a particular branch, and not with sqlite. Quite possibly z3c.sqlalchemy has a release that actually works. One out of three is not bad... :) We are going into production with collective.lead and are committed to releasing the 1.0 branch as 1.0. sqlite works just great for us, we use it to run unit tests and for developers that just need to adjust the styling and such. The production environment will run against Oracle. The elro-tpc branch (which, I was told, is the future) doesn't appear to work with sqlite out of the box. I think this is a temporary bug, as Laurence pointed out, caused by SQLite not supporting two-phase commit. It should be fixed before release, by adding a one-phase fallback. I'm curious how to do functional tests with collective.lead - I'd like to have some real easy way to get the database set up and tore down down in my tests. Are you doing this? I'm doing a lot of it in Java. :) The typical pattern is to use a test setup that does database clean-down to return it to a steady state. Test data is kept in a declarative file (e.g. with an XML syntax that maps to DB structure) and used to initialise the database before each run. Doing nested transactions and rollback would be nicer, but probably doesn't work in all cases since app code may well do explicit commits. It's never particularly pleasant. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: straighting out the SQLAlchemy integration mess
Andreas Jung wrote: --On 9. April 2008 14:15:38 +0100 Laurence Rowe [EMAIL PROTECTED] wrote: @everyone: If we can all agree to use the same basic session and transaction management then we should probably push for it to be included as a sqlalchemy extension module. I would be happy with such a solution. As said, I am not into the datamanager-building business and more interested in application-integration issues. The basic requirements I have for our applications: - having the same session within one ZODB transaction (per thread) - need to get hold of the underlying database connection for a given session I think that's very much the business collective.lead wants to be in. :) Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: straighting out the SQLAlchemy integration mess
--On 10. April 2008 01:57:12 +0100 Martin Aspeli [EMAIL PROTECTED] wrote: Andreas Jung wrote: --On 9. April 2008 14:15:38 +0100 Laurence Rowe [EMAIL PROTECTED] wrote: @everyone: If we can all agree to use the same basic session and transaction management then we should probably push for it to be included as a sqlalchemy extension module. I would be happy with such a solution. As said, I am not into the datamanager-building business and more interested in application-integration issues. The basic requirements I have for our applications: - having the same session within one ZODB transaction (per thread) - need to get hold of the underlying database connection for a given session I think that's very much the business collective.lead wants to be in. :) You know what was first :-) Even with the collective.lead and z3c.zalchemy datamanager...still two instead of one :-) -aj pgpBydBlV3U6n.pgp Description: PGP signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope] i18n/L10n support in zope2
Dear list members, I am newbie/begineer in area of zope2 + i18n/L10n so please excuse me if this topic has already been discussed on the list, in that case can some one please point me to the past discussions/threads. After reading some basics about i18n+zope2, I have tried following - (zope version - Zope 2.9.7-final, python 2.4.4, win32 and Placeless Translation Service - 1.3.7) Created simple page template - html body span i18n:domain=PlacelessTranslationService i18n:translate=Reload this catalog Reload this catalog /span /body /html Modified language preferences for browsers - (Mozilla Firefox 2.0.0.4 and MS-IE 7.0.5730.11) to set German [de] as primary langage. When I viewed above page in firfox got the corresponing german translation (Diesen Katalog neu einlesen) as specified in related .po file. But with MS-IE, no luck, gives same string - Reload this catalog So is it browser specific/dependable ? Another thing I tried is adding attribute i18n:target=cs, but it seems to have no effect, firefox gives german translation as per language settings as MS-IE gives same string - Reload this catalog Does it means that attribute target is not supported ? Is there any list/documentation about supported i18n attributes ? Also I was not able to find references/examples for DTML, does DTML supports i18n/L10n ? Any help/suggestions/pointers would be of great help. Thanking you, Harshad. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ExternalMethod - add new parameter
If you would elaborate a little, like what you really want to do, you might get responses directing you toward a quicker solution. That aside, if you want to execute a command line application as root from your External Method look at Paramiko as mentioned in my response to the previous External Method question. Paramiko is a python library that allows you to use ssh or sftp. Thomas On Monday 07 April 2008 08:16, rishi pathak wrote: There is a requirement for running some external methods as super user. For this I thought of adding a new parameter.If set the code would be executed with effective uid of root. Can some one point to the code section where zope loads the ExternalMethod codes for execution. -- Thomas McMillan Grant Bennett Appalachian State University Operations Systems AnalystP O Box 32026 University Library Boone, North Carolina 28608 (828) 262 6587 ... using OpenOffice.org, and save them back to disk automatically, in MS Word format. They surf the Web, check e-mail, do instant messsaging, view YouTube videos, visit their Facebook pages, learn touch-typing skills and lots more. Our public library has been offering these Linux public stations for the past three years. - Phil Shapiro Linux Journal January 2008 Library Systems Help Desk: https://www.library.appstate.edu/help/ ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Re: i18n/L10n support in zope2
harshad behere, on 2008-04-09: Dear list members, I am newbie/begineer in area of zope2 + i18n/L10n so please excuse me if this topic has already been discussed on the list, in that case can some one please point me to the past discussions/threads. After reading some basics about i18n+zope2, I have tried following - (zope version - Zope 2.9.7-final, python 2.4.4, win32 and Placeless Translation Service - 1.3.7) Created simple page template - html body span i18n:domain=PlacelessTranslationService i18n:translate=Reload this catalog Reload this catalog /span /body /html I would expect either: i18n:translate= so that the string behind it ends up in the .pot file (and you don't repeat yourself needlessly; also think what would happen when you change one string but forget to change the other). Or: i18n:translate=reload_this_catalog or some other unique message id. But the above code should work. Well, using PlacelessTranslationService as i18n:domain is strange. You should use the name of your own product/package. Modified language preferences for browsers - (Mozilla Firefox 2.0.0.4 and MS-IE 7.0.5730.11) to set German [de] as primary langage. When I viewed above page in firfox got the corresponing german translation (Diesen Katalog neu einlesen) as specified in related .po file. But with MS-IE, no luck, gives same string - Reload this catalog So is it browser specific/dependable ? It should not. Do you have a way to check which browser headers are sent by IE and firefox and if they differ in the language preference? Another thing I tried is adding attribute i18n:target=cs, but it seems to have no effect, firefox gives german translation as per language settings as MS-IE gives same string - Reload this catalog Does it means that attribute target is not supported ? Is there any list/documentation about supported i18n attributes ? I have never seen that attribute. From the top of my head, these i18n attributes are used in zope: translate, domain, name and attributes. Also I was not able to find references/examples for DTML, does DTML supports i18n/L10n ? No. At least, I do not think so, and this grep in a buildout did not return any relevant results: grep i18n $(find . -name '*dtml') -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ExternalMethod - add new parameter
rishi pathak wrote at 2008-4-8 18:03 +0530: I dont have a need to run all the external method as root, only some of them. I did understand this *BUT* you do not have a choice. While a single ExternalMethod runs as root, the complete Zope process runs as root -- and this applies to all requests which are run in parallel with your ExternalMethod. Please reread my former message. If you have touble to understand the terms thread and/or process, consult Wikipedia to get some insight. ... On Tue, Apr 8, 2008 at 12:50 AM, Dieter Maurer [EMAIL PROTECTED] wrote: rishi pathak wrote at 2008-4-7 17:46 +0530: There is a requirement for running some external methods as super user. For this I thought of adding a new parameter.If set the code would be executed with effective uid of root. This is extremely dangerous. To run code as super user, you need to change the effective user id. Changing the effective user id affects the whole process -- not just the thread executing your external method. These things are very difficult to handle in a multi threaded environment, in general. Moreover, running internet driven code uncontrolled as super user is likely to be a big security risk. Let your application write some command to a queue and process the queue asynchronously. The processing can be performed as super user. If this is not possible, let your application communicate with another process which runs as super user -- and pass on synchronous commands from your application to this process. In both cases, it is ensured that only the restricted command set can be used to run something as super user -- and not some arbitrary code -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Help writing test where bobobase_modification_time changes
Peter Bengtsson wrote at 2008-4-8 19:08 +0100: ... The reason it fails on the last line is that the time difference between the first time and the second time is nil. Adding a time.sleep(1) won't change anything because the bobobase_modification_time() doesn't change. How can I solve this? Not easy. bobobase_modification_time() is in fact a DateTime representation of _p_serial. And _p_serial is the transaction id (also a timestamp) that has committed the object state belonging to this object incarnation. Unfortunately, you must not commit a transaction in a test suite (as this may cause interference with other tests). There is a small chance that a savepoint changed _p_serial -- but the chance is only small... You might be able to write _p_serial yourself and thereby trick bobobase_modification_time(). -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Conditional dtml-boundary in dtml-sendmail
Dear list
I'm trying to send a multi-part email after having submitted a
form. In that form, the user will give some basic information
from him, then he will be able to send at most three files.
This is my dtml-method to send email after having submitted the
data:
dtml-let email=REQUEST.get('aba_email_txt').encode('utf-8')
name=REQUEST.get('aau_name_txt').encode('utf-8')
lastName=REQUEST.get('aat_vorname_txt').encode('utf-8')
company=REQUEST.get('aas_firma_txt').encode('utf-8')
file1=REQUEST.get('aap_dateien_anhang_dat')
file2=REQUEST.get('aaq_dateien_anhang_dat')
file3=REQUEST.get('aar_dateien_anhang_dat')
dtml-sendmail smtphost=my_host.my_domain.com
To: [EMAIL PROTECTED]
From: dtml-email;
Subject: Anfrage dtml-name; dtml-lastName; - dtml-company;
Content-Type: text/html; charset=utf-8
dtml-mime type=text/html; charset=utf-8
html
body
Some html code comes here
/body
/html
dtml-boundary type=application/octet-stream disposition=attachment
filename_expr=file1.filename
dtml-var file1.read()dtml-boundary type=application/octet-stream
disposition=attachment
filename_expr=file2.filename
dtml-var file2.read()dtml-boundary type=application/octet-stream
disposition=attachment
filename_expr=file3.filename
dtml-var file3.read()/dtml-mime
/dtml-sendmail
/dtml-let
Let's say that the user doesn't submit file3, on that case, zope will
generate something like this: some_ramdom_id.dat and it will attach
an empty file. I tried something like this:
dtml-if file1.filename not in ['', None]dtml-boundary
type=application/octet-stream disposition=attachment
filename_expr=file1.filename
dtml-var file1.read()/dtml-if
However when I try to save, zope fails and says:
Error Type: Document Template Parse Error
Error Value: Unexpected tag, for tag dtml-boundary
type=application/octet-stream disposition=attachment
filename_expr=file1.filename , on line 215 of sendAnfrage
The only way I found to cope with this was doing a script to validate if a
file has whether a name or not, then I will return a dictionary like this:
{'name': real_name_or_dummy_name, 'data': file.read()_or_dummy_text}
So, I feed the boundary with a dummy file when no file was uploaded. I don't
really like to send dummy files since it may confuse the person receiving
the emails. Is there any better way of achieving this? Please note that
since I'm working in the zmi and I'm not the administrator of the machine
where my page is hosted, I can only use dtml to send mail :-(, thus using
the python mail library isn't an option on this case.
Best regards
Josef
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Conditional dtml-boundary in dtml-sendmail
--On 9. April 2008 22:40:01 +0200 Josef Meile [EMAIL PROTECTED] wrote: Dear list I'm trying to send a multi-part email after having submitted a form. In that form, the user will give some basic information from him, then he will be able to send at most three files. I started shivering while reading your code. We would have written such code five or six years ago - because we had no other choice. Please use the standard 'email' module of Python...this just works and you don't need to write such ugly and error-prone DTML code. -aj pgp4WZlzN2BGk.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
