Re: [Zope-dev] FTP interface being worked on?
--On Saturday, March 17, 2001 08:46:26 PM -0500 Fred Wilson Horch [EMAIL PROTECTED] wrote: I think lossless serialization should be an explicit goal. If a developer doesn't provide specific object serialization methods, then a default method (perhaps XML) should be invoked that is lossless even if hard to work with. I'm not sure what the caveats were that lead to the non-lossless guarantee. Think of the filesystem representation of a ZCatalog. What is lossless vs. non-lossless? If the filesystem representation dumps evrything required to recreate a working copy of the catalog after a (perhaps lengthy) computation but doesn't actually dump the full current contents is that a lossless representation? The whole point for us is to get full control of our objects through CVS. And grep and emacs, etc. At least for us. This is really the big issue. If all you need is CVS, a "morally binary" XML representation can do. Zope already provides one, though it is not ideal for CVS. If you want to be able to use other file system based tools (a.k.a. normal development tools) then you need a representation much closer normal text. It's almost obvious what this should be for folders, DTML, ZSQL, PythonScripts, etc. It's much less obvious what this should be for ZCatalogs, Racks (yeah DC probably doesn't care but I do), ZClasses, etc. It may be hard to come up with something better than XML pickles, which I agree should probably be the default if nothing better is specified. Then there is metadata. That leads into your next question: Can anyone tell me where my effort would best be spent? Would it be best for me to start with FSDump or ZCVSMixin and corrupt them to serve my evil plans, or should I start from scratch based on the object serialization discussion we've been having (but with the explicit goal of lossless serialization, unlike Chris' proposal)? The difference is that ZCVSMixin reads and writes XML pickles because capturing all metadata was a major goal. We can't live with the extreme unfriendlyness of XML pickles to other tools. FSDump tries to capture all metadata explicitly in ".props" files. I suspect that it is much closer to the eventual file system representation of Chris' proposal. FSDump has no read capability. At IPC9, someone from DC told me that Tres was worried that read capability would be a giant security hole. I can't remember if that someone was Tres or not. IMHO, the solution to this probably involves forcing read to be invoked only from outside of Zope (or maybe only from a local machine login?). I'm not sure how this would be done. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] FTP interface being worked on?
"Dan L. Pierson" wrote in part: I think lossless serialization should be an explicit goal. What is lossless vs. non-lossless? If the filesystem representation dumps evrything required to recreate a working copy of the catalog after a (perhaps lengthy) computation but doesn't actually dump the full current contents is that a lossless representation? Yes, in my book (as long as original and recreated copies of the catalog are functionally identical). I'm using lossless in the sense it is used in the compression field. If you can recreate the same objects from the representation (even if it requires several computational steps) then the representation is "lossless". A "lossy" representation would mean that you lose some piece of information that is essential to getting back to the original state of the object database. For images, JPEG is a lossy compression scheme, which means it is one way. If you convert a TIFF to JPEG, then you can't go back to the exact same TIFF. By contrast, PNG is lossless. You can convert from TIFF to PNG and back to TIFF and get the exact same TIFF. My concern is that if the plain text serialized format is lossy, it will be one way only. That is not good for us. To preserve the round trip ability, we need a lossless representation. The whole point for us is to get full control of our objects through CVS. And grep and emacs, etc. Yes. CVS is the principal driver for us, but grep and emacs are important too. If all you need is CVS, a "morally binary" XML representation can do. Zope already provides one, though it is not ideal for CVS. If you want to be able to use other file system based tools (a.k.a. normal development tools) then you need a representation much closer normal text. It's almost obvious what this should be for folders, DTML, ZSQL, PythonScripts, etc. It's much less obvious what this should be for ZCatalogs, Racks (yeah DC probably doesn't care but I do), ZClasses, etc. Good points. I'm eager to hear from anyone else with a perspective on this before I start coding something up. Thanks, Fred -- Fred Wilson Horch mailto:[EMAIL PROTECTED] Executive Director, EcoAccess http://ecoaccess.org/ P.O. Box 2823, Durham, NC 27715-2823phone: 919.419-8354 ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] INSTANCE_HOME vs. SOFTWARE_HOME
Hi guys, some people have asked me to use INSTANCE_HOME instead of SOFTWARE_HOME, which breaks their products on debian distros. Now, I'm not sure that won't break other systems if I change it; anyone care to share? Thanks, Morten ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] getPhysicalPath?
Brian Lloyd wrote:
Is there a simple method like getAcquistionPath() that tells
me how I was
required without having to do some sub optimal REQUEST
variable hacking /
string matching?
I don't know of any (but that doesn't mean there isn't one). But
couldn't you walk up the aq_parent?
Acquisition tip-o-the-day: you can use:
print object.aq_chain
The aq_chain attribute (computed at time of access) provides
a list (in reverse order) of the objects in the acquisition
path.
Is this *cough* documented anywhere?
grep -r on the Zope source comes up with (not counting test_AqAlg.py):
./lib/Components/ExtensionClass/src/Acquisition.c:module_aq_chain(PyObject
*ignored, PyObject *args)
./lib/Components/ExtensionClass/src/Acquisition.c: {"aq_chain",
(PyCFunction)module_aq_chain, METH_VARARGS,
./lib/Components/ExtensionClass/src/Acquisition.c: "aq_chain(ob [,
containment]) -- "
So nobody (including the various Products I have installed) uses this
incredibly useful attribute. I have numerous places in my source where I
loop through aq_parent attributes.
It'd be nice to have even a mention in the acquisition documentation.
Poking around the Acquisition C source tells me that there's another
attribute, aq_inner, that isn't documented either.
Richard
--
Richard Jones
[EMAIL PROTECTED]
Senior Software Developer, Bizar Software (www.bizarsoftware.com.au)
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] PATH_INFO reliable?
I want to strip off the SERVER_URL of an absolute url (which I have as a string) without doing string manipulations. Is there some built-in function to do this? So what I'm looking for is actual the reverse of absolute_url. The request variable PATH_INFO looks very good, but I don't know whether this is _always_ created. Thx in advance, Danny P.S.: Maybe I just got something completely wrong here. Help! ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] FTP interface being worked on?
Chris McDonough wrote: Fred Horch wrote: My major question is I don't understand the design decision to allow lossy representation. [...] I think lossless serialization should be an explicit goal. If a developer doesn't provide specific object serialization methods, then a default method (perhaps XML) should be invoked that is lossless even if hard to work with. I think the proposal says something like this. The proposal states in part: If this API is not implemented by the developer, the result is undefined (XML pickle representation if allowed by the object on a per-object basis?). I guess I'm voting to rewrite this sentence: If this API is not implemented by the developer, the result is a default serialized representation (perhaps XML pickle) on a per-object basis. The whole point for us is to get full control of our objects through CVS. That's one use, which is important to you. Another is to use Emacs or Dreamweaver on a representation of, for example, DTML methods on a filesystem, which is important to other folks. The point of having potentially "lossy" representation of objects is to make it easier to work with them in these kinds of tools. Nobody wants to edit XML, AFAICT. I see. I agree with the goal to have a representation that is easy to work with in emacs. Would it be possible to have a "lossless" representation that is also easy to work with? The current XML export format is "lossless" but hard to work with. "Potentially lossy" also doesn't mean "leaky". It just means that folks who expose their objects to this sort of serialization can choose their own format, and if it represents the object adequately for their own use in both directions, it's good enough. Maybe the issue is semantics. I think "potentially lossy" == "potentially leaky". Even a small leak would cause problems for us. Maybe it wouldn't cause problems for others. But it sure seems like it would be possible to create a solution that works for everyone. Namely, a lossless representation that is easy to work with. I completely agree with the point that we want to be able to work with representations of objects using tools like emacs and dreamweaver. In fact, we'd like to use emacs as our front end to CVS. The ideal situation would be to edit Zope objects in emacs, publish them to a Zope object database, test them (perhaps using a separate web browser like Netscape or Internet Explorer), and then once everything is working, commit the objects to a CVS repository (using emacs or from the command line). If you want a lossless "morally binary" representation, it's probably best to use XML export, which is great for your purposes, because it already exists! ;-) I've heard it said that all progress is due to the unreasonable man. So to do my part for progress, what I want is a lossless "morally plain text" representation. ;-) If the existing XML export really was great for our purposes, I'd be done! The problem is that everything comes out in one big monolithic file. That's not good for project management using CVS. (At least, as far as I can tell.) I think there is the potential for a really good solution that solves our need to manage our project via CVS, and to solve the greater need to enhance the Zope management interface to support tools that work with filesystem objects. I am about to jump into this project next week. I do want to stay in touch with anyone who is working on similar projects, so please keep in touch! I will post reports as I make progress in case anyone is interested. Thanks, Fred -- Fred Wilson Horch mailto:[EMAIL PROTECTED] Executive Director, EcoAccess http://ecoaccess.org/ P.O. Box 2823, Durham, NC 27715-2823phone: 919.419-8354 ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] FTP interface being worked on?
On Sun, 18 Mar 2001, Dan L. Pierson wrote: representation of Chris' proposal. FSDump has no read capability. At IPC9, someone from DC told me that Tres was worried that read capability would be a giant security hole. I can't remember if that someone was Tres or not. IMHO, the solution to this probably involves forcing read to be invoked only from outside of Zope (or maybe only from a local machine login?). I'm not sure how this would be done. Presumably the issue here is the one that results in 'import' only working on files stored in the host file system (ie: you have enough authority to have file system privs in the zope directory to import zexp pickles or XML pickles). A file-system-serialized represenatation has the additional advantage over XML pickles that it can be re-parsed and have the security rules applied on read. This however means that XML as the default for objects that don't explicitly implement the file-system-serialize API is probably not secure. For CVS, XML default would be good. For round trip editing using "standard tools", XML default would not be good. So I think XML should be the default for write, but there should be no default for read. --RDM ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] FTP interface being worked on?
On Sun, 18 Mar 2001, Chris McDonough wrote: "Potentially lossy" also doesn't mean "leaky". It just means that folks who expose their objects to this sort of serialization can choose their own format, and if it represents the object adequately for their own use in both directions, it's good enough. If you want a lossless "morally binary" representation, it's probably best to use XML export, which is great for your purposes, because it already exists! ;-) So you are saying that the reconstructed object must be functionally identical, but may or may not be bytewise identical when reconstructed from the serialization. For round trip editing, this seems reasonble. For CVS, not getting a lossless result could result in spurrious diffs. Somehow, though, I don't think this will be a problem in practice, and we can ensure that it won't be by requiring that a read of the reconstructed object produce a bytewise identical serialization to the one which was used to build the reconstructed object. This is likely to be the case in any reasonable serialization implementation, and so as I said should not be a problem in practice. --RDM ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] FTP interface being worked on?
Fred Wilson Horch wrote: Chris McDonough wrote: Fred Horch wrote: My major question is I don't understand the design decision to allow lossy representation. [...] I think lossless serialization should be an explicit goal. If a developer doesn't provide specific object serialization methods, then a default method (perhaps XML) should be invoked that is lossless even if hard to work with. I think the proposal says something like this. The proposal states in part: If this API is not implemented by the developer, the result is undefined (XML pickle representation if allowed by the object on a per-object basis?). I guess I'm voting to rewrite this sentence: If this API is not implemented by the developer, the result is a default serialized representation (perhaps XML pickle) on a per-object basis The whole point for us is to get full control of our objects through CVS. That's one use, which is important to you. Another is to use Emacs or Dreamweaver on a representation of, for example, DTML methods on a filesystem, which is important to other folks. The point of having potentially "lossy" representation of objects is to make it easier to work with them in these kinds of tools. Nobody wants to edit XML, AFAICT. I see. I agree with the goal to have a representation that is easy to work with in emacs. Would it be possible to have a "lossless" representation that is also easy to work with? The current XML export format is "lossless" but hard to work with. "Potentially lossy" also doesn't mean "leaky". It just means that folks who expose their objects to this sort of serialization can choose their own format, and if it represents the object adequately for their own use in both directions, it's good enough. Maybe the issue is semantics. I think "potentially lossy" == "potentially leaky". Even a small leak would cause problems for us. Maybe it wouldn't cause problems for others. But it sure seems like it would be possible to create a solution that works for everyone. Namely, a lossless representation that is easy to work with. I'd first like to say that I applaud the goal stated in the previous line! I think there are two key problems with achieving it. 1) Because everyone writing extensions for Zope can define their own data structures it make it very difficult to store them anywhere but an object database. I think this problem has nearly the same complexity as figuring out the RDBMS table structures necessary for all the Products and builtin objects in Zope... 2) A lesser problem is when trying to edit the serialized "files". Because objects are methods and state how you modify an object can be guided if not controlled. When we have serialized the objects in a Zope system to files, we have exported only the state of the objects in the ZODB. We then have to live with the ability to foul up invariant across many objects by changing some data in the serialized format. A good example would be ZCatalogs. When some piece of data changes the code can automatically call reindex(), if I'm editing a file I might not know that I need to change other files due to runtime dependencies. (I know that ZCatalog is a poor example because earlier in the thread cataloging was discussed wrt lossy/lossless behavior, but it was the easiest for me to make my point with.) Having said that, I suspect that a few systems could solve the first problem. (I don't know how to solve the second one with serialized data...) a) XML is structured enough that it can reliably hold the data from the ZODB. The current XML dump is not useful for this - it would need to create individual files and folders to represent containment. b) A hybrid XML and custom dump solution. An Image for example could dump out as a binary image file with meta-data in a similiarly name XML file. c) A special file system - like ReisferFS might be that system. To my knowledge ReisferFS is eventually intended to be a melding of file system, relation db, and object db. This is obviously serious paraphrasing, but I think it may have enough descriptive power to replace XML. I know know how this would interact with CVS, or be edited, but I thought I'd mention it. None of these solve problem 2), but then again I don't think anything does.. Thanks for listening, John I completely agree with the point that we want to be able to work with representations of objects using tools like emacs and dreamweaver. In fact, we'd like to use emacs as our front end to CVS. The ideal situation would be to edit Zope objects in emacs, publish them to a Zope object database, test them (perhaps using a separate web browser like Netscape or Internet Explorer), and then once everything is working, commit the objects to a CVS repository (using emacs or from the command line). If you want a lossless "morally binary" representation, it's
Re: [Zope-dev] FTP interface being worked on?
Fred wrote: I guess I'm voting to rewrite this sentence: If this API is not implemented by the developer, the result is a default serialized representation (perhaps XML pickle) on a per-object basis I think this makes sense. Maybe the issue is semantics. I think "potentially lossy" == "potentially leaky". Even a small leak would cause problems for us. Maybe it wouldn't cause problems for others. But it sure seems like it would be possible to create a solution that works for everyone. Namely, a lossless representation that is easy to work with. This is possible probably for many objects. DTML Methods, for instance, are basically just big bags of text with some security settings and other associated metadata. Recreating them losslessly from a filesystem representation is pretty easy. ZCatalogs, on the other hand, have lots of state, which is hard to adequately represent in anything but a morally binary representation. Maybe we won't even try to make it editable, and we'll choose to use XML for these. John wrote: I'd first like to say that I applaud the goal stated in the previous line! I think there are two key problems with achieving it. 1) Because everyone writing extensions for Zope can define their own data structures it make it very difficult to store them anywhere but an object database. I think this problem has nearly the same complexity as figuring out the RDBMS table structures necessary for all the Products and builtin objects in Zope... Yes... luckily, because we have OO and polymorphism, we don't have to do this! ;-) I don't think it's reasonable or wise to impose any "master structure" for filesystem serialization of bodies of objects. Each instance (or perhaps each class) should define how best to serialize itself to disk. Representations between classes are likely to be radically different. A place for standardization is in the "properties" file(s) which accompany each object rep... this is likely to be XML or another structured variant. 2) A lesser problem is when trying to edit the serialized "files". Because objects are methods and state how you modify an object can be guided if not controlled. When we have serialized the objects in a Zope system to files, we have exported only the state of the objects in the ZODB. We then have to live with the ability to foul up invariant across many objects by changing some data in the serialized format. A good example would be ZCatalogs. When some piece of data changes the code can automatically call reindex(), if I'm editing a file I might not know that I need to change other files due to runtime dependencies. Yup... it's probably easiest to make ZCatalogs a black box. a) XML is structured enough that it can reliably hold the data from the ZODB. The current XML dump is not useful for this - it would need to create individual files and folders to represent containment. This is pretty easy right now. Ten lines of recursive code can walk the whole tree if necessary and export only leaf objects. b) A hybrid XML and custom dump solution. An Image for example could dump out as a binary image file with meta-data in a similiarly name XML file. Yes, each object should make its own policy regarding its body. Its metadata format should be standardized, however. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Organizing modules in a python product
Thanks, Dieter. You're right, I definitely wouldn't want to use this solution. I was hoping for something that can be done inside the product :-( Dieter Maurer wrote: Itai Tavor writes: ... abbreviated references to sibling subproducts ... Is there any way to set this up so imports like 'import Utils' and 'import Module2' would work in Module1? As a bonus, can I also get __init__.py files in each module directory to be executed? You can manipulate (extend) "sys.path" in your (top level) "__init__.py" file. Python starts its package/module lookup from the elements in "sys.path". However, I doubt very much, that you should do this! You will lose the namespace isolation provided by packages. Dieter -- -- Itai Tavor -- "Je sautille, donc je suis."-- [EMAIL PROTECTED]-- - Kermit the Frog -- -- "What he needs now is understanding... and a confederate victory" -- -- Dr. Jacobi, Twin Peaks -- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Product Tutorial
Hi. Is http://www.zope.org/Members/hathawsh/PythonProductTutorial/index_html?pp=1 still valid, or have things changed since it was written? /Magnus ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
