[Zope3-dev] Re: Big Problem with FTP in Zope 3.3.0, seems to be the result of using `v = self.get(name, self); if v is self...` in a land where Proxy objects can kill identity comparison
I have to follow this up. While I thought I was doing a good job
reporting and testing the problem, I didn't do a hard enough
control-group study. So, I started to investigate further:
Even though I saw absolutely nothing about our ReadDirectory subclass
that would cause the strange behavior, I decided to double check the
behavior against a fresh install of Zope 3.3.0, with only my
pdb.set_trace statement in place (this time I placed it in
``zope.app.filerepresentation.ReadDirectory.__getitem__`` so I could
watch traversal).
Trying to chdir to '/customer/bogus' in FTP yielded the desired result::
ftp> cd customer/bogus
550 /customer/bogus: No such file or directory.
After seeing that, I stepped through the debugger. Sure enough, the
``if v is self`` failed in ``ReadDirectory.__getitem__``, causing the
situation I explained before:
Basically, in the FTP situation (or any situation wherein proxies are
being made), the following scenario happens. The _dir is used in place
of 'self' to show that it's what ultimately gets returned to the
publisher::
1. publishTraverse('bogus') from customer
2. _dir = IReadDirectory(customer)
3. _dir['bogus'] is called
a. v = _dir.get('bogus', _dir) [returns _dir]
b. ``if v is _dir: raise KeyError('bogus')`` fails, as v and _dir
are the same proxied objects, but have diferent identities at the
Python level.
c. Instead of raising a KeyError on the bogus (ha!) name, ``_dir``
is returned.
So this phantom ReadDirectory object gets sent out to the publisher.
The same situation is happening on a clean Zope 3.3.0 instance and on
the instance on which we found the problem (which provides a
subclassed ReadDirectory adapter for our container objects). If Zope's
ReadDirectory and Bottlerocket's ContentContainerReadDirectory are
being pushed up to the publisher as phantoms in each case, why does
the default setup return the proper FTP "No such file" error where the
Bottlerocket setup allows one to enter this non-existing directory?
I traced the behavior down to
``zope.app.publication.ftp.FTPPublication`` and the *callObject*
method. `callObject` essentially is doing this (staying with the
'_dir' object mentioned above)::
_dir = ob
view = queryMultiAdapter((ob, ftprequest), name='ls', default=self)
Ignoring the ``default=self`` here, `queryMultiAdapter` **fails on the
default setup, but returns FTPView in our custom setup!** Again - we
do nothing special in our ZCML registration, and our custom setup
subclasses from the default ReadDirectory, what's going on? I decided
to look at the list of provided interfaces at the moment of
``FTPPublication.callObject``
The behavior of a *clean setup*, doing ``chdir ... bogus``::
(Pdb) pp list(providedBy(ob).__sro__)
[,
,
]
Hmmm. So I tried this on our *custom setup*, and got this::
(Pdb) pp list(providedBy(ob).__sro__)
[,
,
,
,
,
,
,
,
,
,
]
Well it turns out that there's one place wherein we diverge from the
base ReadDirectory: we include a ``zope.interface.implements``
declaration::
class ContentContainerReadDirectory(ReadDirectory):
implements(zope.filerepresentation.interfaces.IReadDirectory)
Between the screaming and the sobbing, I commented out the
``implements`` line and tried again::
(Pdb) pp list(providedBy(ob).__sro__)
[,
,
,
]
And lo and behold, our custom setup worked::
ftp> cd /customer/bogus
550 /customer/bogus: No such file or directory.
So!!! This situation happened because our class had the audacity to
declare its instances' provided interface! The same interface that the
ZCML adapter declarations say they provide. It seems to change the
interface / specification order in a way that then elicits different
behavior. The FTPView commands are registered for IReadContainer,
which is the direct parent of IReadDirectory.. So when I had the
``implements`` line in our custom setup, we could change into those
non-existing directories since the IReadDirectory adapter of the
parent directory ('customer') was being returned by its own getitem
lookup; and by declaring implements directly, IReadContainer shows up
in the SRO, allowing Zope to go ahead and bind the 'ls' view to this
funky directory.
This seems MAJORLY broken: the only way to get the right behavior
(aside from patching / overriding getitem) is to NOT declare support
(in Python) for the interface my IReadDirectory adapters provide.
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Big Problem with FTP in Zope 3.3.0, seems to be the result of using `v = self.get(name, self); if v is self...` in a land where Proxy objects can kill identity comparison
Jeff Shell wrote: [lots of stuff I don't know much about, so I won't comment on] Is there any reason why the ``_marker = object() get(..., default=_marker)`` paradigm isn't used? After encountering this, I am rather uneasy about the number of places in which I see this "self as default marker check" paradigm. My biggest unease comes from not knowing proxies and when things are proxied and when they're not and when I have to be careful about them and when I don't. I like the "marker" flavor of the pattern better too, but for a different reason. When I see something in the second argument to the get, I don't know if it's /really/ the default, or if it's just being used as a placeholder so I have to investigate to find out. I prefer the more explicit version. -- Benji York Senior Software Engineer Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Big Problem with FTP in Zope 3.3.0, seems to be the result of using `v = self.get(name, self); if v is self...` in a land where Proxy objects can kill identity comparison
Hello. I'm filing this in the Collector but wanted to post it as an email first before I had to wrangle the formatting to work in the collector (although it might work straight over). We've encountered a problem with FTP in Zope 3.3. The problem is that you can CWD into non existant paths. Tools like Fetch, Dreamweaver, whatever, will test to see if it has to make a directory by trying to change-dir to that directory, and will make a folder if that fails. But if the change-dir command works for non-existant containers, then everything blows up once it starts trying to upload files into said non-existant containers. I have a basic (but real) example of this. We have the following basic folder structure:: [root] - customer - winter - info - services Following a basic FTP session:: ftp> cd / 250 Requested File Action Completed OK ftp> cd customer 250 Requested File Action Completed OK ftp> ls drwxrwx--- 1 nana 0 Oct 27 23:23 winter 226 Transfer Complete. ftp> cd winter 250 Requested File Action Completed OK ftp> ls drwxr-x--- 1 nana 0 Oct 19 18:40 info drwxr-x--- 1 nana 0 Oct 27 17:40 services 226 Transfer Complete. Everything is OK so far. But lets go back up a level (to 'customer') and try to get to a non-existant folder named 'bogus':: ftp> cd .. 250 Requested File Action Completed OK ftp> pwd 257 "/customer" ftp> ls drwxrwx--- 1 nana 0 Oct 27 23:23 winter 226 Transfer Complete. ftp> cd bogus 250 Requested File Action Completed OK ftp> ls drwxr-x--- 1 nana 0 Jan 01 1970 bogus 226 Transfer Complete. Crap So after digging through the FTP Server code from twisted.protocols.ftp to zope.app.twisted.ftp.* to zope.app.publication.ftp to zope.app.ftp.FTPView, I came to this: In file zope/app/ftp/__init__.py, I put a ``import pdb; pdb.set_trace()`` call in line 68 (at start of ``def ls(...)`` method). These are some of the results while at that breakpoint. I found something quite interesting:: (Pdb) self._dir # Getting a real element of out self._dir (Pdb) self._dir['winter'] # Getting a bogus element out of self._dir (Pdb) self._dir['bogus'] # Hmm, those look downright identical (Pdb) self._dir['bogus'] is self._dir False # Oh crap, there's probably proxies. (Pdb) from zope import proxy (Pdb) proxy.sameProxiedObjects(self._dir, self._dir['bogus']) True Where did this come from? The FTPView's publishTraverse method is implemented as:: def publishTraverse(self, request, name): return self._dir[name] The 'CWD' command is implemented in Zope 3 as traversing to the full path and checking that it can call 'ls' on it:: ftp> cd customer/bogus (Pdb) print self.request command: ls filter: None path: /customer/bogus The expected behavior is that /customer/bogus should fail during publishTraverse because 'bogus' is not in 'customer'. Now it should be noted that self._dir in this instance is a ReadDirectory subclass. However, it does not override get or getitem (our subclass overrides the keys method only as a cheap filtering mechanism). So the problem is happening on ``return self._dir[name]``, which leads to zope.app.folder.filerepresentation.ReadDirectory:: def __getitem__(self, key): v = self.get(key, self) if v is self: raise KeyError(key) return v I've been noticing that ``get(key, self)... if v is self`` pattern is being used more and more in Zope. I'm not sure if this is a good practice, with all of the proxying that can occur. Proxying seems to be done at a lower level than I can comprehend. I just know that it kills identity comparison (even though items look identical when looking at their default repr() strings). Notice this similar situation in plain old debugzope, free from security concerns and whatever else pops up:: bin> ./debugzope >>> from zope.filerepresentation.interfaces import IReadDirectory >>> _dir = IReadDirectory(root['customer']) >>> _dir # Successful getitem >>> _dir['winter'] # Failure getitem - the one that succeeds in FTP Land >>> _dir['bogus'] Traceback (most recent call last): File "", line 1, in ? File "/opt/local/zope33/lib/python/zope/app/folder/filerepresentation.py", line 57, in __getitem__ raise KeyError(key) KeyError: 'bogus' This time we see the expected behavior. Because of this, I spent a lot more time tracing how FTP works in Zope that I probably needed to because my first try - "Let's see if IReadDirectory is doing the right thing by firing up debugzope.." - worked as expected. Moral of the story: don't use ``query/get (..., default=self)``, ``if value is self`` paradigm? Be careful of identity comparisons? Is ther
Re: [Zope3-dev] Re: PyCon 2007 web frameworks panel
Martin Aspeli wrote: > Paul Winkler wrote: > >> This sounds like something that could stand being talked about >> in public more often... I've never heard of Zope Grok :) > > Me Grok... me SMASH ZCML! Whenever Grok speaks, he always capitalizes: ME GROK SMASH ZCML! Unfortunately, he's a kind of limited caveman and his speaking is very still very rough. -- gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: OpenPGP digital signature ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: PyCon 2007 web frameworks panel
Paul Winkler wrote: This sounds like something that could stand being talked about in public more often... I've never heard of Zope Grok :) Me Grok... me SMASH ZCML! Martin ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] PyCon 2007 web frameworks panel
On Tue, Oct 31, 2006 at 07:54:41PM +0100, Martijn Faassen wrote: > I'm happy to see you do this. I can't make it to that PyCon, unfortunately. > > As to representing other people's opinion: > > Talk about Grok. Better yet, show Grok code. By that time, we should be > quite a way with it, supporting most Zope 3 features out of the box. I > think Grok does quite well in the simplicity and developer productivity > departments. Simplicity and developer productivity is something that is > heavily advocated for some other Python web frameworks. Zope 3 had a bit > of trouble to keep up with that. With Zope Grok, no more trouble. This sounds like something that could stand being talked about in public more often... I've never heard of Zope Grok :) -- Paul Winkler http://www.slinkp.com ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] PyCon 2007 web frameworks panel
Jim Fulton wrote: Benji York wrote: Titus Brown is putting together a web framework panel for the next PyCon: http://us.pycon.org/TX2007/WebFrameworksPanel. I just stuck my name on this. :) If someone else wants badly to do this, I'm willing to yield. In any case, I'd like to represent some other people's opinions. I'm happy to see you do this. I can't make it to that PyCon, unfortunately. As to representing other people's opinion: Talk about Grok. Better yet, show Grok code. By that time, we should be quite a way with it, supporting most Zope 3 features out of the box. I think Grok does quite well in the simplicity and developer productivity departments. Simplicity and developer productivity is something that is heavily advocated for some other Python web frameworks. Zope 3 had a bit of trouble to keep up with that. With Zope Grok, no more trouble. [snip] I think I'll set up a Wiki page to describe some of my ideas and to solicit others. Okay, let us know. Regards, Martijn ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] PyCon 2007 web frameworks panel
Titus Brown is putting together a web framework panel for the next PyCon: http://us.pycon.org/TX2007/WebFrameworksPanel. -- Benji York Senior Software Engineer Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] PyCon 2007 web frameworks panel
Benji York wrote: Titus Brown is putting together a web framework panel for the next PyCon: http://us.pycon.org/TX2007/WebFrameworksPanel. I just stuck my name on this. :) If someone else wants badly to do this, I'm willing to yield. In any case, I'd like to represent some other people's opinions. If anyone wants to send me their answers, I'd be happy to incorporate them into my presentation, with attribution. The Zope community is diverse and I expect many people to have opinions that differ from mine and I'd like to represent them. Some especially juicy areas include ideas about templating and o/r mapping. I think I'll set up a Wiki page to describe some of my ideas and to solicit others. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] international publicity to zope3
Hi, Markus Leist wrote: > Hi there, > > maybe this is an interesting chance to give international publicity to zope3. > see here: > > http://www.plat-forms.org/ > > I think, the result of such programming contest will bring more transparency > in > pros and cons of Zope3 development, which can be published to the world. > > Hey, there are really good people in development of Zope3 and the > component-architecture of “our python-application-server” should > be compared to other applications, or? (ok, not new ...) > > My own software-developer-skill isn't good enough for this contest. > The results will be published on a high level; > zope3 and the developers can obtain great acceptance, i think. > > Any developer there ( 3 developer a 30 h)? Thanks for pointing this out. I've been talking to the organizers already and Martijn Faassen, Philipp von Weitershausen and me are thinking about setting up a team already. I'm also giving a talk about Zope itself at the conference where plat_forms takes place. I'll post an update once everything is clear. Christian -- gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: OpenPGP digital signature ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] international publicity to zope3
Hi there, maybe this is an interesting chance to give international publicity to zope3. see here: http://www.plat-forms.org/ I think, the result of such programming contest will bring more transparency in pros and cons of Zope3 development, which can be published to the world. Hey, there are really good people in development of Zope3 and the component-architecture of “our python-application-server” should be compared to other applications, or? (ok, not new ...) My own software-developer-skill isn't good enough for this contest. The results will be published on a high level; zope3 and the developers can obtain great acceptance, i think. Any developer there ( 3 developer a 30 h)? Bye, Markus Leist ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
