[Zope3-dev] PersistentInterfaceClass Fix - Issue 747
I've submitted some new tests and a fix for zope.app.interface.PersistentInterfaceClass in: http://www.zope.org/Collectors/Zope3-dev/747 I have commit priveleges to the svn repo, but I'd love to get some review before commiting. baijum from #zope3-dev suggested I post here. Below is a copy of the text from issue 747. > zope.app.interface.PersistentInterfaceClass uses a PersistentDict for > the dependents attribute instead of the WeakKeyDictionary used in > zope.interface.interface.InterfaceClass. There are a number of bugs > associated with this approach not exposed in any tests. Attached is a > diff to the tests of zope.app.interface and zope.app.component to > expose these bugs. > Firstly, if IFoo is a zope.app.interface.PersistentInterfaceClass > instance and zope.interface.directlyProvides(foo, IFoo), the > declaraion can't be verified from a different ZODB connection with > IFoo.providedBy(foo) due to the following. When the ZODB is trying to > unpickle the dependents attribute of IFoo, it procedes down the > serialization of IFoo to the ProvidesClass instance representing the > declaration. It begins reconstituting the ProvidesClass instance, > which calls IFoo.subscribe(provides) which accesses the dependents > attribute of IFoo. Since the ProvidesClass instance isn't persistent, > it has no oid the ZODB circular reference check doesn't catch this > circle. As a result the fix is to make sure that the declarations > instances in the dependents attribute are themselves persistent. > Also attached is a diff to zope.app.interface that replaces > PersistentInterfaceClass.dependents with a custom dict that converts > non-persistent declarations being added to the dependents attribute > into persistent versions of the same. > This fixes the problem but is not optimal because there are then two > instances for the same declaration, one in the ProvidesClass instance > stored in the object's __provides__ attribute and the other in the > dependents attribute of the PersistentInterfaceClass. > It seems like the more appropriate solution would be to check for > PersistentInterfaceClass instances in > zope.interface.declarations.directlyProvides and use the persistent > declaration classes for those declaraions. Since > PersistentInterfaceClass is in zope.app.interface and zope.interface, > I wasn't sure which was the lesser of the two evils, so I restrained > my changes to zope.app. What might be a better solution? Give me > some feedback and I'll change the implementation. > There's another problem with the > zope.app.interface.PersistentInterfaceClass.dependents attribute. > zope.interface.InterfaceClass.dependents is a WeakKeyDictionary so > that declarations don't keep objects from being freed if the object is > removed. By using a PersistentDict for dependents, the declarations > can keep an object from being freed from memory and/or the ZODB when > the object is removed. > My first patch also includes tests for this bug. These tests seem to > have exposed another unrelated bug. An instance of a class that > subclasses persistent.Persistent is declared that it > zope.interface.directlyProvides a zope.interface.InterfaceClass > instance and then the persistent object is added to the ZODB and > committed. Then if the persistent instance is deleted from the ZODB, > the transaction is committed, and the ZODB is packed, and gc.collect > is run, the ProvidesClass instance in the InterfaceClass instance > still remains. It does not, however, remain if the persistent > interface was never added to the ZODB. > I'm not sure if this represents a potential memory leak or not. What > confuses me is that it all behaves properly unless the persistent > instance is added to the ZODB. I noted the comment about weak > referrences being added to the ZODB optomistically in ZODB.serialize. > Could that be it? Unfortunately, my second patch doesn't include a > fix for this. I'd be happy to investigate this further if given a > little direction. Ross ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] z3c.zalchemy.demos
I've been working with z3c.zalchemy for a short while, and was finding it very hard to get started until Stephan Richter gave me a simple example to work with. In the course of my subsequent investigations, I found and fixed a few bugs in the ZAlchemy code, and produced a few simple demos. With Juergen Kartnaller's blessing, these demos have been checked into the z3c.zalchemy trunk in a 'demo' subpackage. So if you get a fresh svn checkout, you should get them along with all recent fixes: http://svn.zope.org/z3c.zalchemy/trunk/src/z3c/zalchemy/demo/ These demos aren't perfect (they are pretty crude in spots), but they are a decent start to using SQLAlchemy in Zope3 with some integration into the ZMI: Demo 1: Subclasses the included SQLAlchemyContainer to create a container that is preconfigured to hold only one class (HelloWordMessage) Demo 2: Creates a container from scratch that is specialized to only hold one class (HelloWorldMessage2). Demo 3: Demonstrates a rudimentary adapter for storing Dublin Core metadata in the RDBMS for HelloWorldMessage3 objects. HelloWorldMessage3 objects have editable title and description fields incorporated into the add and edit forms. The metadata is stored in a second table. Demo 4: In addition to the Dublin Core integration, demonstrates a HelloWorldMessage4 class that is in itself a container for HelloWorldFragment classes. Feedback (and patches) welcome! - Michael R. Bernstein michaelbernstein.com signature.asc Description: This is a digitally signed message part ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Idea: Failure to lookup adapters
Zope 3 has that as well. However, this only gives you the information of the stack. It doesn't give you information created during the run of e.g. loops, which I imagine would be interesting to create a trail of "what did you do?" information. Well, you could set a list into __traceback_info__ and append to it during each loop, instead of assigning. Anyway, that would only be seen in the case of an exception. Unless we could expose a view that shows the stack without having an exception. -- Sidnei da Silva Enfold Systemshttp://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: zope.tal.xmlparser.XMLParser() dislikes unicode
Tres Seaver wrote: [snip] The "just store the XML" scenario is in surprisingly nice. It only needs attention to encoding and decoding in the always complicated ZPublisher direct output scenario, and in the edit form scenario. As you speculated, this is actually my preference, except that I don't see the need to in scenario D to recode the data and strip the prolog encoding attribute. Why wouldn't we just use the XML template's own declared encoding to encode any data subsituted into the template? I mean, if the user has marked up the document to indicate a "preferred" encoding, why should we bother storing such an encoding in another location? Yes, I was thinking along those lines too. Then the only time we would need to munge the document would be at inclusion time, which is the only time we actually *need* to have unicode in hand. We might even elide the decode-recode stage if the target document uses the same encoding! That such an optimization might not be worth the complexity, however. Yes, one complexity is that trying to do this would break the assumption that ZPT templates always return unicode or pure-ascii strings, not anything else (such as encoded data). Only at the last phase of the publisher will it be encoded into something else. I really appreciate keeping this assumption in place. :) Note that in the inclusion case (scenario E), we almost certainly *should* be stripping the *entire* prolog, which is only valid at the start of the merged document. If you are including it as a document, yes. If you are included it quoted, as for instance the contents of a text area allowing you to edit the XML text directly, then no. This suggests we actually have two scenarios here. I guess there is a subscenario, which is that the "included" document is actually the 'main_template' supplying the prolog: METAL might should leave the prolog alone, while 'tal:replace' and 'tal:content' (with 'structure') would strip it? Yay, another scenario. :) Regards, Martijn ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: zope.tal.xmlparser.XMLParser() dislikes unicode
Dieter Maurer wrote: Martijn Faassen wrote at 2007-1-15 15:44 +0100: Hey, On 1/15/07, Andreas Jung <[EMAIL PROTECTED]> wrote: [snip] ok, got it. But this problem can be solved easily by changing the encoding within the preamble. I would say refusing to guess and bailing out with an error message is better in this case. I disagree with you. Logically, parsing an encoded XML document consists of two passes: decode the encoded string into unicode and reconstruct the XML info elements from the serialization. Traditionally, these two passes are not performed one after the other but folded together in a single pass. But that tradition should not prevent to separate out the (Unicode) decoding phase. And after this phase is done, there is not ambiguity left with the "XML declaration". Its encoding attribute is simply irrelevant for the second phase (apart from generating the PI info element). That's nice as far as it goes. What if after the second phase you need to parse the XML again? What do you do with your encoding header then? If it's irrelevant, you better strip it out before you put it into the parser. Regards, Martijn ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Some small thoughts about the term 'Zope 3 core' in the future
Hi, I was updating one of the proposals in this minutes and had a somewhat hard time to define a goal. I stumbled over the upcoming problem of defining the Zope 3 core and came up with this footnote for myself: Maybe this can be used as a spark to spur some discussion about the term 'Zope 3 core'. Maybe we need new terms in the future, maybe we need to re-define 'Zope 3 core'. Here is the version that appeared to me today: .. [1] Zope 3 core With the upcoming fuzzyness of the Zope 3 core, I found a somewhat interesting definition for me that defines the core as the set of packages that gets labeled with the same release tag together (there can be other ways of labeling that using a SVN tag of course) and that we apply the full rigid development practices for Zope 3 on. IMHO therefore being in 'the core' for a Zope 3 package guarantees a certain quality level. Christian -- gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] server error in apidoc (encoding pb)
Christophe Combelles wrote: Did someone have a look on this error ? Is it a trivial thing, or a local configuration problem, or is it reproduceable? (I write in utf-8, and I put # -*- coding: utf-8 -*- in my interfaces.py) You must use unicode docstrings. The encoding declaration at the top of the module doesn't do any magic by itself but just tells Python from which encoding unicode strings can be decoded. So your module should looks like this: # -*- coding: utf-8 -*- u"""Some docstring text in utf-8.""" -- Dmitry Vasiliev (dima at hlabs.spb.ru) http://hlabs.spb.ru ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Idea: Failure to lookup adapters
Hi, Am Dienstag, den 16.01.2007, 20:28 +0100 schrieb Dieter Maurer: > Sidnei da Silva wrote at 2007-1-15 17:25 -0200: > > ... > >The kind of info I'm looking for is something along the lines: > > > > 'We've tried to look up an adapter for (ISomething, ITheOther) but > >none was found' > > 'Found an adapter for IFoo, which is a base class for the IBar > >interface requested. No adapter has been found for the most-specific > >interface IBar' > > > >Comments? > > In Zope 2, I would use "__traceback_info__" or (more likely) > "__traceback_supplement__" and the traceback formatting > facilities in "zExceptions.ExceptionFormatter". > This is a very efficient way to analyse all problems that > result in an exception -- far better than log entries. Zope 3 has that as well. However, this only gives you the information of the stack. It doesn't give you information created during the run of e.g. loops, which I imagine would be interesting to create a trail of "what did you do?" information. -- gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: zope.tal.xmlparser.XMLParser() dislikes unicode
Tres Seaver wrote at 2007-1-16 10:39 -0500: > ... >As you speculated, this is actually my preference, except that I don't >see the need to in scenario D to recode the data and strip the prolog >encoding attribute. Why wouldn't we just use the XML template's own >declared encoding to encode any data subsituted into the template? Maybe, because an XML template "T1" using encoding "e1" uses a macro from template "T2" encoded with "e2"? Or maybe, because in such a case some values passed into the macro (e.g. the splots) cannot be encoded in "e2"? -- Dieter ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: zope.tal.xmlparser.XMLParser() dislikes unicode
Tres Seaver wrote at 2007-1-15 16:57 -0500: > ... >Frankly, I don't get the desire to *store* a complete XML document (as >opposed to the extracted contents of attributes or nodes) as unicode My desire comes from the easy principle: all text should be unicode. Decoding/encoding happens only at the system boundaries and no longer internally. -- Dieter ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Idea: Failure to lookup adapters
Sidnei da Silva wrote at 2007-1-15 17:25 -0200: > ... >The kind of info I'm looking for is something along the lines: > > 'We've tried to look up an adapter for (ISomething, ITheOther) but >none was found' > 'Found an adapter for IFoo, which is a base class for the IBar >interface requested. No adapter has been found for the most-specific >interface IBar' > >Comments? In Zope 2, I would use "__traceback_info__" or (more likely) "__traceback_supplement__" and the traceback formatting facilities in "zExceptions.ExceptionFormatter". This is a very efficient way to analyse all problems that result in an exception -- far better than log entries. -- Dieter ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: zope.tal.xmlparser.XMLParser() dislikes unicode
Martijn Faassen wrote at 2007-1-15 15:44 +0100: > >Hey, > >On 1/15/07, Andreas Jung <[EMAIL PROTECTED]> wrote: >[snip] >> ok, got it. But this problem can be solved easily by changing the encoding >> within the preamble. > >I would say refusing to guess and bailing out with an error message is >better in this case. I disagree with you. Logically, parsing an encoded XML document consists of two passes: decode the encoded string into unicode and reconstruct the XML info elements from the serialization. Traditionally, these two passes are not performed one after the other but folded together in a single pass. But that tradition should not prevent to separate out the (Unicode) decoding phase. And after this phase is done, there is not ambiguity left with the "XML declaration". Its encoding attribute is simply irrelevant for the second phase (apart from generating the PI info element). Thus, there is no guessing; someone else has just performed the first phase of the complete process -- maybe using the "encoding" attribute or some overriding information. -- Dieter ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: zope.tal.xmlparser.XMLParser() dislikes unicode
Chris Withers wrote at 2007-1-14 18:14 +: > ... >The problem comes when someone sends you something like: > >u'' > >What should be done then? We parse the declaration and generate an info element for it but otherwise ignore it as it has lost its meaning after the XML has been converted to Unicode. -- Dieter ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] server error in apidoc (encoding pb)
Christophe Combelles a écrit :
When I put some accentuated character in the docstring of an interface
and I want to see the interface doc in
apidoc->interfaces->search->IMyInterface,
I then get a server error:
Here is the full backtrace:
--
2007-01-16T16:58:11 ERROR SiteError
http://localhost:9673/++apidoc++/Interface/zblog.category.interfaces.ICategory/index.html
Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/zope/publisher/publish.py", line 133,
in publish
result = publication.callObject(request, obj)
File
"/usr/lib/python2.4/site-packages/zope/app/publication/zopepublication.py", line
161, in callObject
return mapply(ob, request.getPositionalArguments(), request)
File "/usr/lib/python2.4/site-packages/zope/publisher/publish.py", line 108,
in mapply
return debug_call(obj, args)
- __traceback_info__: zope.app.pagetemplate.simpleviewclass.SimpleViewClass from
/usr/lib/zope3/lib/python/zope/app/apidoc/ifacemodule/index.pt instance at
0xb57112cc>
File "/usr/lib/python2.4/site-packages/zope/publisher/publish.py", line 114,
in debug_call
return obj(*args)
File
"/usr/lib/python2.4/site-packages/zope/app/pagetemplate/simpleviewclass.py",
line 44, in __call__
return self.index(*args, **kw)
File
"/usr/lib/python2.4/site-packages/zope/app/pagetemplate/viewpagetemplatefile.py",
line 83, in __call__
return self.im_func(im_self, *args, **kw)
File
"/usr/lib/python2.4/site-packages/zope/app/pagetemplate/viewpagetemplatefile.py",
line 51, in __call__
sourceAnnotations=getattr(debug_flags, 'sourceAnnotations', 0),
File "/usr/lib/python2.4/site-packages/zope/pagetemplate/pagetemplate.py",
line 117, in pt_render
strictinsert=0, sourceAnnotations=sourceAnnotations)()
File "/usr/lib/python2.4/site-packages/zope/tal/talinterpreter.py", line 271,
in __call__
self.interpret(self.program)
File "/usr/lib/python2.4/site-packages/zope/tal/talinterpreter.py", line 346,
in interpret
handlers[opcode](self, args)
File "/usr/lib/python2.4/site-packages/zope/tal/talinterpreter.py", line 891,
in do_useMacro
self.interpret(macro)
File "/usr/lib/python2.4/site-packages/zope/tal/talinterpreter.py", line 346,
in interpret
handlers[opcode](self, args)
File "/usr/lib/python2.4/site-packages/zope/tal/talinterpreter.py", line 536,
in do_optTag_tal
self.do_optTag(stuff)
File "/usr/lib/python2.4/site-packages/zope/tal/talinterpreter.py", line 521,
in do_optTag
return self.no_tag(start, program)
File "/usr/lib/python2.4/site-packages/zope/tal/talinterpreter.py", line 516,
in no_tag
self.interpret(program)
File "/usr/lib/python2.4/site-packages/zope/tal/talinterpreter.py", line 346,
in interpret
handlers[opcode](self, args)
File "/usr/lib/python2.4/site-packages/zope/tal/talinterpreter.py", line 949,
in do_defineSlot
self.interpret(slot)
File "/usr/lib/python2.4/site-packages/zope/tal/talinterpreter.py", line 346,
in interpret
handlers[opcode](self, args)
File "/usr/lib/python2.4/site-packages/zope/tal/talinterpreter.py", line 745,
in do_insertStructure_tal
structure = self.engine.evaluateStructure(expr)
File "/usr/lib/python2.4/site-packages/zope/tales/tales.py", line 696, in
evaluate
return expression(self)
- /usr/lib/zope3/lib/python/zope/app/apidoc/ifacemodule/index.pt
- Line 23, Column 4
- Expression:
- Names:
{'args': (),
'context': ,
'default': ,
'loop': {},
'nothing': None,
'options': {},
'repeat': {},
'request': URL=http://localhost:9673/++apidoc++/Interface/zblog.category.interfaces.ICategory/index.html>,
'template':
0xb6a2c9ac>,
'usage': 0xb5711ccc>,
'view': /usr/lib/zope3/lib/python/zope/app/apidoc/ifacemodule/index.pt object at
0xb57112cc>,
'views': at 0xb5711d4c>}
File "/usr/lib/python2.4/site-packages/zope/tales/expressions.py", line 217,
in __call__
return self._eval(econtext)
File "/usr/lib/python2.4/site-packages/zope/tales/expressions.py", line 211,
in _eval
return ob()
File
"/usr/lib/python2.4/site-packages/zope/app/apidoc/ifacemodule/browser.py", line
85, in getDoc
return renderText(iface.__doc__, inspect.getmodule(iface))
File "/usr/lib/python2.4/site-packages/zope/app/apidoc/utilities.py", line
327, in renderText
source = createObject(format, text)
File "/usr/lib/python2.4/site-packages/zope/component/_api.py", line 224, in
createObject
return getUtility(IFactory, __factory_name, context)(*args, **kwargs)
File "/usr/lib/python2.4/site-packages/zope/app/renderer/__init__.py", line
41, in __call__
source = Source(ustr)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 38: ordinal
not in range(128)
127.0.0.1 - - [16/Jan/2007:16:58:11 +0200] "GET
/++apidoc++/Interface/zblog.category.interfaces.ICategory/index.html HTTP/1.1"
500 94 "http://localhost:9673/++apid
[Zope3-dev] Re: server error in apidoc (encoding pb)
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Christophe Combelles wrote:
> Did someone have a look on this error ? Is it a trivial thing, or a local
> configuration problem, or is it reproduceable?
> (I write in utf-8, and I put # -*- coding: utf-8 -*- in my interfaces.py)
>
> Christophe
>
>
> Christophe Combelles a écrit :
>> Hello,
>>
>> When I put some accentuated character in the docstring of an interface
>> and I want to see the interface doc in
>> apidoc->interfaces->search->IMyInterface,
>>
>> I then get a server error:
>>
>>
>> File "/usr/lib/python2.4/site-packages/zope/tales/tales.py", line 696,
>> in evaluate
>> return expression(self)
>>- /usr/lib/zope3/lib/python/zope/app/apidoc/ifacemodule/index.pt
>>- Line 23, Column 4
>>- Expression:
>>- Names:
>> {'args': (),
>>'context': > zblog.category.interfaces.ICategoryContainer>,
>>'default': ,
>>'loop': {},
>>'nothing': None,
>>'options': {},
>>'repeat': {},
>>'request': > URL=http://localhost:9673/++apidoc++/Interface/zblog.category.interfaces.ICategoryContainer/index.html>,
>>
>>
>>'template':
>> > at 0xb6a319ac>,
>>'usage': > 0xb5c3ea4c>,
>>'view': > from /usr/lib/zope3/lib/python/zope/app/apidoc/ifacemodule/index.pt
>> object at 0xb5c3ef4c>,
>>'views': > object at 0xb5bf9d6c>}
Please give us the *full* traceback: we can't even see what error is
occuring here, except that something is going wrong in 'view/getDoc'.
Tres.
- --
===
Tres Seaver +1 540-429-0999 [EMAIL PROTECTED]
Palladion Software "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFrPIA+gerLs4ltQ4RAj2tAJ9o5wySWu5nD94p2iF2Ht2K+mlc6gCgz4RY
ZwgDhICeKxd2uX3ZF1fEZOg=
=2ejP
-END PGP SIGNATURE-
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: zope.tal.xmlparser.XMLParser() dislikes unicode
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martijn Faassen wrote: > Andreas Jung wrote: >> >> --On 15. Januar 2007 22:15:46 +0100 Martijn Faassen > [snip] I still don't see what should ambiguous with this approach. >>> Ambiguous in that the string seems to say it's in two encodings at once. >>> You're then "guessing": you're letting the Python string type trump the >>> declaration. Then, since we've shown that leads to bugs, you propose >>> actually change the encoding declaration of the XML document. I wonder >>> what people then expect to happen upon serialization. In effect, your >>> proposal would, I think, serialize to UTF-8 only, right? (in which case >>> the encoding declaration can be dropped as it's the default. >> When you download a ZPT through FTP/WebDAV then the unicode representation >> of the XML will be converted using the 'output_encoding' property of the >> corresponding ZPT which is set when uploading a new XML document (and taken >> from the premable). So when you upload an latin1 XML file you should get >> it back as valid latin1 through FTP/WebDAV. > > Okay, understood, this makes sense in the case of the FTP/WebDAV > support, though recoding to UTF-8 and ripping off the encoding > declaration would also be pretty safe in case of XML. > >> When you download text/xml content through the ZPublisher then the >> ZPublisher will convert unicode textual content to some encoding which is >> either taken from an already set 'content-type: text/...; charset=X' >> HTTP Header or as fallback from the zpublisher-default-encoding property >> as defined in the zope.conf file. > > And the same behavior actually applies to HTML content, right? > >> So the application can specify in both case the encoding of the serialized >> XML content. Where is the problem? > > What I'm trying to express here is that this stuff should not be treated > as "where is the problem?" but should be thought through carefully as > this is extremely easy to do wrong. I'll think it through carefully > here. Let's list some cases: > > A) FTP download: stored ML gets downloaded through FTP/WebDAV support. > > B) FTP upload: external XML gets uploaded through FTP/WebDAV > > C) parse: stored XML is parsed inside of Zope by the page template engine. > > D) publisher download: stored XML is downloaded as text/xml directly > through the publisher > > E) ZPT inclusion: stored XML is included in another page template, for > instance to present it in a text area. > > F) form submit: Text area is then saved and needs to be stored again. > > Andreas Jung proposal (speculation) > === > > As far as I understand it you're proposing: > > * store XML as unicode text > > * separately store the encoding on the page template object > > * also keep the encoding="" bit in the XML preamble when storing. > > Let's go through the cases > > A) FTP download: encode this to whatever encoding is stored on the ZPT > object using Python unicode support. No encoding mangling necessary. > > B) FTP upload: read encoding="" bit and store this on ZPT. Then decode > to unicode using that encoding. Could not be implemented by a > parse/serialization step without extra encoding="" manipulation > afterwards (after decoding to unicode). > > C) parse: Rip out the 'encoding=""' bit before you send it in the > parser. encode to UTF-8 just before entering the parser. > > D) publisher download: Rip out the 'encoding=""' bit. Then encode > according to response header (or zope.conf). Then add back encoding="" > bit stating if output is non-UTF-8 (not Python names like 'latin1' but > encoding identifiers XML is aware of). > > E) ZPT inclusion: Send the unicode text to the page template. > encoding="" bit will be presented in the editor. > > F) form submit: decode to unicode according to encoding of page that > displayed edit form and store it. Read 'encoding=' bit and store it in > ZPT object. Don't manipulate 'encoding=""' bit in XML. > > encoding="" removal: C, D > encoding="" adding: D > encoding="" reading: B, F > encode from unicode: A, C, D > decode to unicode: B, F > > no encoding="" manipulation required: A, E > no recoding required: E > straightforward: E > > The forms editor scenario (E and F) is potentially confusing as the user > may be tempted by the ability to use encoding="" to paste latin-1 XML > text. Editor could say it only wants it in whatever encoding the page is > in, though. > > Martijn Faassen proposal > > > If you rip out the encoding before data is stored in the page template > and then store as unicode, then we have the following cases: > > A) FTP download: Encode to UTF-8, output in UTF-8 only. No encoding > mangling necessary. > > B) FTP upload: read encoding="" bit and decode to unicode accordingly. > Rip out encoding="". Could be done by a parse/serialization step, then > decode result to unicode. > > C) parse: enco
[Zope3-dev] Re: SVN: zope.app.annotation/ Add zope.app.annotation as a top-level package.
Baiju M wrote: Log message for revision 72044: Add zope.app.annotation as a top-level package. Changed: A zope.app.annotation/ A zope.app.annotation/branches/ A zope.app.annotation/tags/ A zope.app.annotation/trunk/ -=- Is it really necessary to create the BBB packages in zope.app (annotation, traversing, etc.) as separate eggs? They're gonna go away in Zope 3.5 anyway. I think they could be part of the zope.app egg in Zope 3.4. Let's not create more work than we already have with this :) -- http://worldcookery.com -- Professional Zope documentation and training 2nd edition of Web Component Development with Zope 3 is now shipping! ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Idea: Failure to lookup adapters
On Tuesday 16 January 2007 08:23, Marius Gedminas wrote: > explainAdapter is probably a bad name for this. Maybe the whole API is > bad. Maybe it would be more useful to list all adapters that could > adapt your object to the desired interface, and list them in the order > of specificness: You can do this already in APIDOC today. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Idea: Failure to lookup adapters
On Tue, Jan 16, 2007 at 08:36:48AM +0100, Christian Theune wrote: > Am Montag, den 15.01.2007, 17:25 -0200 schrieb Sidnei da Silva: > > One of the most common issues that I've faced, and watched several > > people struggle with, during Zope 3 development is to figure why a > > certain adapter is not found, or why a certain other adapter is found > > instead of the one you are expecting. ... > > One of the ideas that occurred me was that in debug mode we could > > create a wrapper around the to-be-returned adapter and stuff some > > trace information in there to help diagnose possible issues. Or maybe > > we could just use the logging framework to output some information. > > > > The kind of info I'm looking for is something along the lines: > > > > 'We've tried to look up an adapter for (ISomething, ITheOther) but > > none was found' > > 'Found an adapter for IFoo, which is a base class for the IBar > > interface requested. No adapter has been found for the most-specific > > interface IBar' > > Yes please. =) > > I'd love some debugging support for that as well. I like the idea of a > log that gives you enough input to reason about what the CA did to > derive its decision. > > In an advanced step, something interactive might be worthwhile to allow > poking into the registrations conveniently. Perhaps a useful question to ask here is "why wasn't my adapter chosen?" $ bin/debugzope >>> from zope.component.introspection import explainAdapter >>> from mypackage import MyAdaper, IMyDesiredInterface >>> my_obj = app.root()['somefolder']['someobj'] >>> explainAdapter(my_obj, IMyDesiredInterface, desired_adapter=MyAdapter) 'MyAdapter is registered for IFoo, but context does not provide IFoo.' explainAdapter is probably a bad name for this. Maybe the whole API is bad. Maybe it would be more useful to list all adapters that could adapt your object to the desired interface, and list them in the order of specificness: >>> for reg in matchingAdapters(my_obj, IMyDesiredInterface): ... print reg.factory, reg.required (, ) (, ) (, ) Marius Gedminas -- ... Another nationwide organization's computer system crashed twice in less than a year. The cause of each crash was a computer virus -- Paul Mungo, Bryan Glough _Approaching_Zero_ (in 1986 computer crashes were something out of the ordinary. Win95 anyone?) signature.asc Description: Digital signature ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: zope.tal.xmlparser.XMLParser() dislikes unicode
Andreas Jung wrote: --On 15. Januar 2007 22:15:46 +0100 Martijn Faassen [snip] I still don't see what should ambiguous with this approach. Ambiguous in that the string seems to say it's in two encodings at once. You're then "guessing": you're letting the Python string type trump the declaration. Then, since we've shown that leads to bugs, you propose actually change the encoding declaration of the XML document. I wonder what people then expect to happen upon serialization. In effect, your proposal would, I think, serialize to UTF-8 only, right? (in which case the encoding declaration can be dropped as it's the default. When you download a ZPT through FTP/WebDAV then the unicode representation of the XML will be converted using the 'output_encoding' property of the corresponding ZPT which is set when uploading a new XML document (and taken from the premable). So when you upload an latin1 XML file you should get it back as valid latin1 through FTP/WebDAV. Okay, understood, this makes sense in the case of the FTP/WebDAV support, though recoding to UTF-8 and ripping off the encoding declaration would also be pretty safe in case of XML. When you download text/xml content through the ZPublisher then the ZPublisher will convert unicode textual content to some encoding which is either taken from an already set 'content-type: text/...; charset=X' HTTP Header or as fallback from the zpublisher-default-encoding property as defined in the zope.conf file. And the same behavior actually applies to HTML content, right? So the application can specify in both case the encoding of the serialized XML content. Where is the problem? What I'm trying to express here is that this stuff should not be treated as "where is the problem?" but should be thought through carefully as this is extremely easy to do wrong. I'll think it through carefully here. Let's list some cases: A) FTP download: stored ML gets downloaded through FTP/WebDAV support. B) FTP upload: external XML gets uploaded through FTP/WebDAV C) parse: stored XML is parsed inside of Zope by the page template engine. D) publisher download: stored XML is downloaded as text/xml directly through the publisher E) ZPT inclusion: stored XML is included in another page template, for instance to present it in a text area. F) form submit: Text area is then saved and needs to be stored again. Andreas Jung proposal (speculation) === As far as I understand it you're proposing: * store XML as unicode text * separately store the encoding on the page template object * also keep the encoding="" bit in the XML preamble when storing. Let's go through the cases A) FTP download: encode this to whatever encoding is stored on the ZPT object using Python unicode support. No encoding mangling necessary. B) FTP upload: read encoding="" bit and store this on ZPT. Then decode to unicode using that encoding. Could not be implemented by a parse/serialization step without extra encoding="" manipulation afterwards (after decoding to unicode). C) parse: Rip out the 'encoding=""' bit before you send it in the parser. encode to UTF-8 just before entering the parser. D) publisher download: Rip out the 'encoding=""' bit. Then encode according to response header (or zope.conf). Then add back encoding="" bit stating if output is non-UTF-8 (not Python names like 'latin1' but encoding identifiers XML is aware of). E) ZPT inclusion: Send the unicode text to the page template. encoding="" bit will be presented in the editor. F) form submit: decode to unicode according to encoding of page that displayed edit form and store it. Read 'encoding=' bit and store it in ZPT object. Don't manipulate 'encoding=""' bit in XML. encoding="" removal: C, D encoding="" adding: D encoding="" reading: B, F encode from unicode: A, C, D decode to unicode: B, F no encoding="" manipulation required: A, E no recoding required: E straightforward: E The forms editor scenario (E and F) is potentially confusing as the user may be tempted by the ability to use encoding="" to paste latin-1 XML text. Editor could say it only wants it in whatever encoding the page is in, though. Martijn Faassen proposal If you rip out the encoding before data is stored in the page template and then store as unicode, then we have the following cases: A) FTP download: Encode to UTF-8, output in UTF-8 only. No encoding mangling necessary. B) FTP upload: read encoding="" bit and decode to unicode accordingly. Rip out encoding="". Could be done by a parse/serialization step, then decode result to unicode. C) parse: encode to UTF-8 just before entering the parser. D) publisher download: Encode according to response header or zope.conf. Add in encoding="" if output is non-UTF-8 using XML names for encoding. E) ZPT inclusion: send unicode text to the page template. No encoding="" bit will be in the XML presented in th
Re: [Zope3-dev] Idea: Failure to lookup adapters
On Monday 15 January 2007 14:25, Sidnei da Silva wrote: > 'We've tried to look up an adapter for (ISomething, ITheOther) but > none was found' > 'Found an adapter for IFoo, which is a base class for the IBar > interface requested. No adapter has been found for the most-specific > interface IBar' > > Comments? My major concern is speed here. Any debug handling will make the registry slower; I am using adapters very heavily in my code and any slow-down of one lookup would be multiplied many times. I wonder whether a different approach could be taken. The APIDOC code does already a registration analysis. Maybe we can use the registrations in a away that they simulate the adapter registry. Thus you could provide APIDOC (via the Web UI or Python) with a component or instance and it tells you how it tries to look up the adapter. Clearly that would be much slower, but okay for inspection. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: zope.tal.xmlparser.XMLParser() dislikes unicode
Tres Seaver wrote: [snip] Unicode XML is not only problematic for streaming. For instance, you *can't* pass a Unicode string to the libxml2 *at all* , unless you want a core dump. The API requires that you pass it strings encoded as UTF8. You can in lxml. :) libxml2 as a C API doesn't even support any unicode string type as far as I am aware. It *requires* UTF-8-encoded strings. See http://xmlsoft.org/xml.html 12. So what is this funky "xmlChar" used all the time? It is a null terminated sequence of utf-8 characters. And only utf-8! You need to convert strings encoded in different ways to utf-8 before passing them to the API. This can be accomplished with the iconv library for instance. Um, Tres, no need to tell me about the libxml2 API.. There is also the libxml2 *python* API, which I believe has a knob to turn on the ability to pass in unicode strings, though I haven't tried that myself. Then there's of course lxml, which is a Python-layer which requires unicode or plain-ascii strings in its DOM-ish (elementtree API), and encoded data for the parser. We should distinguish the behavior of libxml2 as a tree API (utf-8 all the way) and as a parser/serializer (all sorts of encodings). Generally XML libraries make a distinction between the two. Frankly, I don't get the desire to *store* a complete XML document (as opposed to the extracted contents of attributes or nodes) as unicode: it isn't as though it can be easily processed in that form without re-encoding (even if lxml is the one doing the re-encoding). It isn't "discourse", in the Zope3 sense of "text intended for human consumption", and the tools people use with it are all going to expect some kind of validly-encoded string. There are objects that allow you to edit XML; the ZPT page is an example. I do not know whether it stores as unicode right now, but you can argue it's text intended for human consumption, as humans are supposed to be editing it. :) It may indeed make more sense to store this information as UTF-8 however from an efficiency point of view. This would probably still require recoding the data into unicode for the purposes of inspecting it and editing it. Regards, Martijn ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] server error in apidoc (encoding pb)
Did someone have a look on this error ? Is it a trivial thing, or a local
configuration problem, or is it reproduceable?
(I write in utf-8, and I put # -*- coding: utf-8 -*- in my interfaces.py)
Christophe
Christophe Combelles a écrit :
Hello,
When I put some accentuated character in the docstring of an interface
and I want to see the interface doc in
apidoc->interfaces->search->IMyInterface,
I then get a server error:
File "/usr/lib/python2.4/site-packages/zope/tales/tales.py", line 696,
in evaluate
return expression(self)
- /usr/lib/zope3/lib/python/zope/app/apidoc/ifacemodule/index.pt
- Line 23, Column 4
- Expression:
- Names:
{'args': (),
'context': zblog.category.interfaces.ICategoryContainer>,
'default': ,
'loop': {},
'nothing': None,
'options': {},
'repeat': {},
'request': URL=http://localhost:9673/++apidoc++/Interface/zblog.category.interfaces.ICategoryContainer/index.html>,
'template':
at 0xb6a319ac>,
'usage': 0xb5c3ea4c>,
'view': from /usr/lib/zope3/lib/python/zope/app/apidoc/ifacemodule/index.pt
object at 0xb5c3ef4c>,
'views': object at 0xb5bf9d6c>}
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/ccomb%40free.fr
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
