[Zope-dev] python scripts

2001-05-14 Thread Tom Deprez

Hi,

This weekend I did some work on python scripts and found some things which
suprised me, can someone enlighten me on this why these 'strange' things are
needed? :

1. When calling a method from a python script you've to do this :

(assume x is the value to pass  and name is a parameter of mymethod)

  context.mymethod(name=x)

   The following doesn't works :

 context.mymethod(x) (this gives an error)

   Why

2. Assume you've an sqlmethod which expects 2 parametes, but one is optional
:

eg
select *
from invoice
dtml-sqlgroup where required
   dtml-sqlgroup
dtml-sqltest customersid column=customers_id op=eq
type=int
dtml-and
 dtml-sqltest productsid column=products_id op=eq type=int
optional
   /dtml-sqlgroup
/dtml-sqlgroup

   So, you could run this sql with or without the productsid

  No, calling from python script doesn't works since it

  expects: context.mymethod(customerid=..., productsid=...)

  while context.mymethod(customerid=...) doesn't work..

Now, this implies that I've to write 2 sqlmethods (one with 2 parameters and
one with 1),
which is pretty stupid, since sqlmethods perfect allow to ignore a parameter
when not passed...


Can somebody tell me more on this? Why this is the case and perhaps a
solution for (2)

Thanks in advance,

Tom Deprez











___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread Mail List

From: Joachim Werner [EMAIL PROTECTED]

  This is true in the ZODB, but can be complicated by acquisition. If an
  object can acquire itself, it can cause issues. Plus it becomes
  difficult to know whether objects are clones or just identical
  instances, although this can be mitigated by exposing their Python
  instance id.

 Acquisition is very cool, but it sometimes really sucks ... AFAIK you can
 easily switch it off in your own Python products. But I am still
fighting
 with only getting private variables (i.e. not acquired ones) in DTML ...

From DTML I have used 2 different methods for this:
1) dtml-with expr=object only
 .
/dtml-with

or
2) dtml-if expr=_.hostattr(object.aq_explicit('attribute')
   ...
/dtml-if

In both cases 'object' is the thing with the 'private variables' and in 2),
'attribute' is the 'variable' name.
Jeff



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread Shane Hathaway

Phillip J. Eby wrote:
 
 At 05:42 PM 5/11/01 -0400, Shane Hathaway wrote:
 Phillip J. Eby wrote:
  I'm not quite clear on how exactly you suggest mapping from RDMBS -
  ZODB.  There's a *significant* (IMHO) impedance mismatch between ZODB's
  arbitrarily identified variably structured single records and SQL's
  content-identified fixed-structure record sets.  This is what application
  frameworks/toolkits (such as your own DBAPI product) are needed for.
 
 If you implement this at the Storage level, yes, there is a major
 mismatch.  But at the Connection level it makes a lot of sense.
 Connection basically exposes a pile of pickles as objects; an OR mapping
 exposes a complex schema as objects.
 
 I think that understanding will change the rest of your response. :-)
 
 
 Nope.  As far as I can see, all that does is leverage ZODB Connections as a
 crude sort of Rack that only provides a mapping-like interface for
 retrieving objects, without helping any of the higher-order needs of an
 application.  I guess if you define O-R mapping as can you store and
 retrieve an object's properties from a database, then I guess you've
 succeeded at that point.  But if that was all my O-R apps needed, there
 would've been no reason to use the RDBMS; I could've just used ZODB and a
 BTree with less bother.

I'm telling you there's a lot more you can do with the code that makes
up ZODB.  The mapping interface is not the key to the way Connection
does its work.  OIDs don't have to be strings.  If we just use
cPersistent, cPickleCache (which is misnamed), and Transaction to
implement an OR mapping, here's what we get for free:

- A transparent, transactional database.

- Potential interoperability with everything written for ZODB.

- Robust, tested code.

- Optimizations in C.

Since you challenged me :-) I decided to put together a prototype.  What
I came up with was a database connection that sets a different _p_jar
for each persistent object.  The _p_jar is an object that manages the
storage for only one persistent object, making it possible to customize
the storage.  There was only one hurdle in this approach but I hacked
around it: the tpc_* messages get sent to each _p_jar.  Then I wrote a
very minimal test that doesn't connect to an RDBMS but stores and
retrieves simple objects in a dictionary.

OIDs are still necessary to support multithreaded invalidation
messages.  But the OIDs in the prototype are a tuple consisting of a
schema ID and a record ID.  That way the record IDs only need to be
unique within an RDBMS table (or combination of tables.)

I didn't think anything like this was possible before I got to know
Jim.  I still didn't understand when he presented the idea months ago. 
But now I see the idea really is workable.  The advantage is that it
lets us completely isolate RDBMS storage details from the application.

The next thing to do is to write a fishbowl proposal.

Shane

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread Shane Hathaway

Joachim Werner wrote:
 
  Probably I'm daft because it is Friday night, but AFAIK ZODB and most
 OODB's
  store an object only once, keyed by its object id. The rest is just
 references
  through that oid, so objects that belong to more than one container can be
  added to all these containers and n:m relations are implemented by having
 a
  list of objects on both sides.
 
 Yes, but these references (let's call them symlinks ...) are not in the
 standard Zope. Of course it should be easy to do these things, but except
 for this from Shane I haven't seen anything so far:
 http://www.zope.org/Members/hathawsh/Symlink/index_html

Actually I was referring to the OODB-equivalent of hard links.  It's
only possible from Python products / external methods / filesystem
scripts / Zope core code, but here's how you do it:

ob = some_folder.some_child
some_other_folder.some_child = ob

In other words, if you just assign an object to have multiple parents
then it will just work.  Changes to the object will appear to occur in
both places, but really they're only occurring in one place.  This
behavior is fully persistent, works over ZEO, and ZODB won't even see
there's anything unusual.

 P.S.: Shane, have you developed Symlinks any further? I think they could be
 extremely useful. I tried out the initial release and liked it, except for
 the fact that the symlinks looked EXACTLY like real ones, so they can be
 very irritating ...

I'm not sure what to do with symlinks.  How should security be applied? 
How are they most useful?

It's neat to see it works, though. :-)

Shane

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread Jeff

From: Joachim Werner [EMAIL PROTECTED]

  This is true in the ZODB, but can be complicated by acquisition. If an
  object can acquire itself, it can cause issues. Plus it becomes
  difficult to know whether objects are clones or just identical
  instances, although this can be mitigated by exposing their Python
  instance id.

 Acquisition is very cool, but it sometimes really sucks ... AFAIK you can
 easily switch it off in your own Python products. But I am still
fighting
 with only getting private variables (i.e. not acquired ones) in DTML ...



From DTML I have used 2 different methods for this:
1) 
dtml-with expr=object only
 .
/dtml-with

or
2) 
dtml-if expr=_.hostattr(object.aq_explicit('attribute')
   ...
/dtml-if

In both cases 'object' is the thing with the 'private variables' and in 2),
'attribute' is the 'variable' name.
Jeff




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread Chris Withers

Shane Hathaway wrote:
 
 I'm telling you there's a lot more you can do with the code that makes

snip

 The next thing to do is to write a fishbowl proposal.

This sounds cool but made my head hurt :-S

Can you try and bring this back down to the level of us mere mortals by
explaining how your OR stuff would let me take a table of data in an RDBMS table
and have it appear as objects in the Management Inteferace?

I know that's horribly oversimplified, but some of us only have small brains ;-)

cheers,

Chris

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread Juan David Ibáñez Palomar

 
  P.S.: Shane, have you developed Symlinks any further? I think they could be
  extremely useful. I tried out the initial release and liked it, except for
  the fact that the symlinks looked EXACTLY like real ones, so they can be
  very irritating ...
 
 I'm not sure what to do with symlinks.  How should security be applied? 
 How are they most useful?
 
 It's neat to see it works, though. :-)
 
 Shane
 
 

Last year I needed symlinks and developed a version of your product
where the symlink had a different id than the real object. But the
general solution of a symlink where an arbitrary set of attributes
are from the symlink instance instead of from the real object was
too much difficult for me, so this year we've changed the dessign
of the application and don't need symlinks anymore.


jdavid

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread Shane Hathaway

Chris Withers wrote:
 
 Shane Hathaway wrote:
 
  I'm telling you there's a lot more you can do with the code that makes
 
 snip
 
  The next thing to do is to write a fishbowl proposal.
 
 This sounds cool but made my head hurt :-S
 
 Can you try and bring this back down to the level of us mere mortals by
 explaining how your OR stuff would let me take a table of data in an RDBMS table
 and have it appear as objects in the Management Inteferace?

Sorry, this is at a pretty low level and I do need to explain it better.

One would define an ObjectMappingSchema whose job it is to store and
retrieve objects of a specific type and in a specific location.  It
would usually grab a database connection object to do its work.  When
loading, it would perform a query then manually put attributes into a
persistent object.  When storing, it would grab specific attributes from
the persistent object and execute a statement to store those attributes.

So let's say you want a ZODB to store and retrieve users in a specific
table while putting everything else in pickles.  You would create an
instance of PickleSchema, which implements the ObjectMappingSchema
interface, and tell it to manage everything *except* the users mapping
in BasicUserFolder objects.  You would tell it to store and retrieve
this object using your UserFolderSchema instead.  Your UserFolderSchema
would store and retrieve the users from the USERS and USER_PREFS
tables.  The user table wouldn't require the use of OIDs but would
require unique user IDs.

So in the management interface nothing would change.  Nor would the
application-level Python code.  You would only define a layer that maps
objects to a relational database.  You would still see the user folder
as you do now.

Now, it may be useful to provide a management interface for defining the
schema mapping.  I haven't approached that yet; AFAICT this is where the
work done on SmartObjects and DBObjects would be very useful.  Initially
I was planning for people to code the mapping purely in Python so we
could gain experience and find common patterns before inventing a UI.

Shane

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread Chris Withers



Shane Hathaway wrote:
 
 One would define an ObjectMappingSchema whose job it is to store and
 retrieve objects of a specific type and in a specific location.  It
 would usually grab a database connection object to do its work.  When
 loading, it would perform a query then manually put attributes into a
 persistent object.  When storing, it would grab specific attributes from
 the persistent object and execute a statement to store those attributes.

How does this differ from ZPatterns? Phil?

 Now, it may be useful to provide a management interface for defining the
 schema mapping.  I haven't approached that yet; AFAICT this is where the
 work done on SmartObjects and DBObjects would be very useful.  Initially
 I was planning for people to code the mapping purely in Python so we
 could gain experience and find common patterns before inventing a UI.

Sounds good to me :-) Thanks for the explanation...

cheers,

Chris

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread John D. Heintz

I think this is a great idea!  I would definetely like to use and contribute 
to this effort.  Having this kind of flexibily would be fantastic.

After demonstratable Python code is working I would request that usability 
issues (UI Schema mapper, data migration/schema evolution tools, ZEO 
integration, multi-Storage uses) be addressed sooner than later.  

John

On Monday 14 May 2001 10:47, Shane Hathaway wrote:
 Chris Withers wrote:
  Shane Hathaway wrote:
   I'm telling you there's a lot more you can do with the code that makes
 
  snip
 
   The next thing to do is to write a fishbowl proposal.
 
  This sounds cool but made my head hurt :-S
 
  Can you try and bring this back down to the level of us mere mortals by
  explaining how your OR stuff would let me take a table of data in an
  RDBMS table and have it appear as objects in the Management Inteferace?

 Sorry, this is at a pretty low level and I do need to explain it better.

 One would define an ObjectMappingSchema whose job it is to store and
 retrieve objects of a specific type and in a specific location.  It
 would usually grab a database connection object to do its work.  When
 loading, it would perform a query then manually put attributes into a
 persistent object.  When storing, it would grab specific attributes from
 the persistent object and execute a statement to store those attributes.

 So let's say you want a ZODB to store and retrieve users in a specific
 table while putting everything else in pickles.  You would create an
 instance of PickleSchema, which implements the ObjectMappingSchema
 interface, and tell it to manage everything *except* the users mapping
 in BasicUserFolder objects.  You would tell it to store and retrieve
 this object using your UserFolderSchema instead.  Your UserFolderSchema
 would store and retrieve the users from the USERS and USER_PREFS
 tables.  The user table wouldn't require the use of OIDs but would
 require unique user IDs.

 So in the management interface nothing would change.  Nor would the
 application-level Python code.  You would only define a layer that maps
 objects to a relational database.  You would still see the user folder
 as you do now.

 Now, it may be useful to provide a management interface for defining the
 schema mapping.  I haven't approached that yet; AFAICT this is where the
 work done on SmartObjects and DBObjects would be very useful.  Initially
 I was planning for people to code the mapping purely in Python so we
 could gain experience and find common patterns before inventing a UI.

 Shane

 ___
 Zope-Dev maillist  -  [EMAIL PROTECTED]
 http://lists.zope.org/mailman/listinfo/zope-dev
 **  No cross posts or HTML encoding!  **
 (Related lists -
  http://lists.zope.org/mailman/listinfo/zope-announce
  http://lists.zope.org/mailman/listinfo/zope )

-- 
. . . . . . . . . . . . . . . . . . . . . . . .

John D. Heintz | Senior Engineer

1016 La Posada Dr. | Suite 240 | Austin TX 78752
T 512.633.1198 | [EMAIL PROTECTED]

w w w . d a t a c h a n n e l . c o m

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread Shane Hathaway

Chris Withers wrote:
 
 Shane Hathaway wrote:
 
  One would define an ObjectMappingSchema whose job it is to store and
  retrieve objects of a specific type and in a specific location.  It
  would usually grab a database connection object to do its work.  When
  loading, it would perform a query then manually put attributes into a
  persistent object.  When storing, it would grab specific attributes from
  the persistent object and execute a statement to store those attributes.
 
 How does this differ from ZPatterns? Phil?

ZPatterns implements storage logic on the application level. 
Applications have to be aware of (in fact they have to be centered
around) ZPatterns.  This alternate approach keeps storage logic
independent of application logic.  It lets you take any code written for
the ZODB and move it to your RDBMS schema without changing the code
(most of the time :-) ).

Shane

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread Chris Withers

Shane Hathaway wrote:
 
 ZPatterns implements storage logic on the application level.
 Applications have to be aware of (in fact they have to be centered
 around) ZPatterns.  This alternate approach keeps storage logic
 independent of application logic.  It lets you take any code written for
 the ZODB and move it to your RDBMS schema without changing the code
 (most of the time :-) ).

Now that sounds really f^%$^ing cool :-) How can I help?

Chris

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread Shane Hathaway

Chris Withers wrote:
 
 Shane Hathaway wrote:
 
  ZPatterns implements storage logic on the application level.
  Applications have to be aware of (in fact they have to be centered
  around) ZPatterns.  This alternate approach keeps storage logic
  independent of application logic.  It lets you take any code written for
  the ZODB and move it to your RDBMS schema without changing the code
  (most of the time :-) ).
 
 Now that sounds really f^%$^ing cool :-) How can I help?

For now, I can make public the code I wrote for experimentation:

http://www.zope.org/Members/hathawsh/orconn.tar.gz

The archive contains two Python files and a patch.  Put them in the ZODB
directory.  Perform the patch to DB.py.  (It's very minimal.)  Then run
testOR.py.  It just changes a mapping, but it does it transparently and
transactionally. :-)

I need to fishbowl this before going any further.

Shane

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] Disabling anonymous webdav access

2001-05-14 Thread Ivo van der Wijk

Hi All,

As someone pointed out on #zope, it is possible to view folder contents
using a webdav client as an anonymous user.

I.e. download cadaver (http://www.webdav.org/cadaver/), open 
yourzopeserver:8080 and do ls. Then decide if you want anyone to be 
able to access this. Eventhough hiding this information may be security
by obscurity, there are some things you just don't want everyone to see.

This allows you to see, for example, the installed products on the server.
A hacker might use this knowledge to exploit some known bug in a zope product
if one exists.

Most people (like me) probably think it's harmless to let old
objects, documents etc linger around as you can't view them in listings
through ftp or http. They don't realize webdav is running by default. Actually,
it can't even be disabled! (z2.py -X -w80 won't do the trick!)

Personally I'd rather see this secured. It's not possible to disable 
'view contents information' for anonymous users in zope, as this will ruin
your entire site (all anonymous access will then be disabled), so the solution
would be to create a new permission for access contents through webdav.

And that's what the following (trivial) patch does. 

After applying you'll get a new permission in your security tab, which 
is set to manager by default. To get the old behaviour back, just set the
permission back to anonymous.

Apply it using patch -p1 ../webdav.patch in your SOFTWARE_HOME (i.e. the 
Zope-2.3.2-src dir).

Or just edit lib/python/webdav/Resource.py by hand :)

I've tested it with Zope 2.3.2, I can't guarantee it will work with other
versions (use at your own risk anyway).

-- cut here --
*** Zope-2.3.2-orig/lib/python/webdav/Resource.py   Tue Mar 27 21:50:37 2001
--- Zope-2.3.2-src/lib/python/webdav/Resource.pyMon May 14 19:16:46 2001
***
*** 109,115 
  
  __ac_permissions__=(
  ('View', ('HEAD',)),
! ('Access contents information',  ('PROPFIND',)),
  ('Manage properties',('PROPPATCH',)),
  ('Delete objects',   ('DELETE',)),
  )
--- 109,115 
  
  __ac_permissions__=(
  ('View', ('HEAD',)),
! ('Access contents information through WebDav',  ('PROPFIND',)),
  ('Manage properties',('PROPPATCH',)),
  ('Delete objects',   ('DELETE',)),
  )
-- cut here --

Cheers,

Ivo

-- 
Drs. I.R. van der Wijk  -=-
Brouwersgracht 132  Amaze Internet Services V.O.F.
1013 HA Amsterdam   -=-
Tel: +31-20-4688336  Linux/Web/Zope/SQL
Fax: +31-20-4688337   Network Solutions
Web: http://www.amaze.nl/Consultancy
Email:   [EMAIL PROTECTED]   -=-


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread Phillip J. Eby

At 12:26 PM 5/14/01 -0400, Shane Hathaway wrote:
Chris Withers wrote:
 
  Shane Hathaway wrote:
  
   One would define an ObjectMappingSchema whose job it is to store and
   retrieve objects of a specific type and in a specific location.  It
   would usually grab a database connection object to do its work.  When
   loading, it would perform a query then manually put attributes into a
   persistent object.  When storing, it would grab specific attributes from
   the persistent object and execute a statement to store those attributes.
 
  How does this differ from ZPatterns? Phil?

ZPatterns implements storage logic on the application level.
Applications have to be aware of (in fact they have to be centered
around) ZPatterns.  This alternate approach keeps storage logic
independent of application logic.  It lets you take any code written for
the ZODB and move it to your RDBMS schema without changing the code
(most of the time :-) ).

I am reminded of a passage from Dirk Gently's Holistic Detective Agency, 
wherein Dirk decides to cut to the heart of the problem, and simply writes 
down the answer.  Now, he declares, he is faced with only a relatively much 
easier problem: What strange language did he write the answer down in?

While I'm not saying that what you're doing isn't possible, I *will* say 
that I believe you are replacing a small amount of API-specificity in the 
code with a similar amount of specificity, coupled with a much larger 
complexity in the mapping layers.  It is, IMHO, much harder to write 
generic mappings at the schema layer where you propose, than to do so at 
the domain logic layer where ZPatterns operates.  Ty and I spent many weeks 
chewing over designs in the space you're talking about, back before 
ZPatterns existed, after consulting at length with Jim Fulton and Michel 
Pelletier.  Our conclusion was that yes, it was *possible* to do the 
mapping you're talking about, but that it was very high impedance for the 
kinds of applications we had in mind.

I'll give a simple example to show what I'm talking about.  Consider a task 
framework where TaskPerformers are assigned Tasks, in a simple one-to-many 
relationship.  TaskPerformers, in this framework, perform up to hundreds of 
tasks per day, thousands per month.  Assuming an RDBMS backend, how do you 
propose to map this in your storage model?

My thought on this, is that you will be forced to explicitly consider the 
nature of this relationship and its storage at the application level.  If 
you write explicitly for ZODB, you might use a BTree, for example.  Or 
perhaps you'd have some kind of global container for the Tasks, with a 
Catalog to index them, and a method on TaskPerformer to query the Catalog.

How would you propose to map this to an RDMS?  Well, substituting catalog 
queries might be relatively straightforward, but perhaps rather time 
consuming to develop in a generic way.  The BTree might give you a bit more 
trouble.  Probably you'd have to end up using some sort of generic 
queryable containers that translated to BTrees or Catalogs or RDBMS tables, 
depending...

And lo!  You now have an application framework.  Oops.

We haven't even addressed performance issues yet, which are the real 
killer.  The only way (again IMHO) to get reasonably high performance that 
is portable across different databases is to be able to write code that 
is optimized based on the back-end.  The only way you can push this code 
under the hood is to create an all-purpose query mechanism.  Otherwise, 
you must expose enough of the inner workings to the application developer 
that they can include platform-specific code for each circumstance.

All this is of course just my opinion, and quite possibly wrong.  But I 
think that an application developer would get more mileage out of your 
DBAPI product (coupled with some utility classes to replace Racks) or from 
ZPatterns than they would from this approach to O-R mapping.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments withORMapping

2001-05-14 Thread Karl Anderson

Casey Duncan [EMAIL PROTECTED] writes:

 I am not arguing necessarily for SQL as a query language for the ZODB.
 Although it is an accepted standard, but not a perfect one by any means
 especially for OODBs. Its appeal lies mainly in the high level of
 community familiarity and the plethora of SQL software to borrow from.

Does anyone have an opinion on the possible usefulness of XPath,
XQuery, and other XML standards for this?  Someone suggested (on the
zope-xml wiki) that it would be nice to be able to drop in a cataloger
that supported a presumably standard and presumably well-known XML
query syntax, and which would work throughout the database because
Zope objects would support DOM.

This is all speculation, and I personally don't know much right now
about XML database interfaces and how finished or well-regarded they
are.

-- 
Karl Anderson  [EMAIL PROTECTED]

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] (LONG) RE: [SmartObjects] Wrap-up of the discussion going on on zope-dev?

2001-05-14 Thread Albert Langer

[Joachim Werner]
As most of you might have recognized, there is a very active thread
(actually two interwoven ones) about RO-mapping etc. going on on zope-dev.
I'd be very happy if someone could wrap up the stuff from there on the
SmartObjects list. ;-)

[Albert]
Thanks for the pointer. As I read them all through at once, after
seeing your pointer, I made notes before joining the discussion,
which may help, though far from a wrap-up.

* SUMMARY *

There seem to be 6 separate topics:

1) Nature of SmartObjects, ZPatterns and Transwarp frameworks and
relation to Zope/ZODB framework.

2) Improvements to ZCatalog search efficiency

3) Query syntax - see Zwiki:
http://dev.zope.org/Wikis/DevSite/Proposals/UnionAndIntersectionOperations

4) Schema transparency (an oxymoron?)

5) Storage of ZODB objects in separate DBMS tables per class - Object
Mapping.

6) Requirements for Object Relational Mapping.


* DETAILED NOTES *

Here's a list of all the items I found in the May archives
with the term ORMapping in the subject line, from thread
Experiments in ORMapping and interwoven thread oodb
philosophics was [above] until I subscribed. Will comment
later on subsequent messages rather than making notes.

If there are any other subject headings missed, or other postings
(including earlier months), please let me know.

Highlights before links attempt to be objective but are limited
by space, by my specific interests in the topic and by my lack of
thorough understanding of Zope internals and consequent previous
non-participation in [zope-dev] with consequent lack of appreciation
of where various posters are coming from.

Quotes following links are verbatim extracts (not necessarily
in same order or in context). Must read means important message
with no or inadequate highlights or extracts given here.

I have omitted numerous affirmations that ZODB meets most current
Zope requirements (perhaps with improvements to ZCatalog)
in view of the fact that nobody appeared to be arguing otherwise.

BTW the date order used in list archives could be better in numeric
order as that might more closely reflect which messages are likely
to have been seen before replying. I have re-arranged.

Starting from 10 May 2001:

Shane Hathaway (DC) - start of discussion - must read
http://lists.zope.org/pipermail/zope-dev/2001-May/011101.html
includes link http://www.zope.org/Members/hathawsh/ormapping.tar.gz
overview in file: sketch

Tino Wildenhain - asks about ZPatterns, ZClasses, doesn't want
purely relational application server.
http://lists.zope.org/pipermail/zope-dev/2001-May/011102.html

SH - answers TW, says not hard to implement by mapping classes
to a table.
http://lists.zope.org/pipermail/zope-dev/2001-May/011103.html

TW - answers SH, prefers improved OODB because doesn't like
mapping classes to schema.
http://lists.zope.org/pipermail/zope-dev/2001-May/011105.html

SH - answers TW, enhanced ZODB storage (RDBMS and BerkelyDBM) is in some
ways the best OODB.
http://lists.zope.org/pipermail/zope-dev/2001-May/011107.html
The other motivations for an RDBMS are (1) people have existing schemas
and want Zope to access the same data as their existing apps, and they
want it to be transparent, and (2) tables with millions of entries are
easily stored in Zope but the perception is that the catalog isn't as
fast as a database index.  No one has done any tests AFAIK. [...]
That's one reason ZODB is so nice.  You can write an application without
writing a formal schema.

Casey Duncan (Kaivo) - suggests Matisse or Objectivity but limited by
not supporting ZODB versions. BerkelyDBM best soon. Mentions slow XML
storage startup.
http://lists.zope.org/pipermail/zope-dev/2001-May/011108.html

JW - already 2 OR mapping projects, SmartObjects and TransWarp,
no duplication because Phillip from Transwarp participating in
SmartObjects list.
http://lists.zope.org/pipermail/zope-dev/2001-May/011109.html

TW - answers CD, XML just another pickle format.
http://lists.zope.org/pipermail/zope-dev/2001-May/00.html

SH - answers JW. SmartObjects, ZPatterns and Transwarp require new
database API instead of maintaining transparency like ZODB. Projects
should look at replacing parts of ZODB instead of adding complexity.
http://lists.zope.org/pipermail/zope-dev/2001-May/01.html
ZODB has pieces that can be split apart and replaced as needed, such
as caching, persistence, transactions, the pickle jar, the
multi-threaded connection factory, and the storage layer.  I'm hoping
we can achieve OR mapping by only replacing the pickle jar, i.e.
Connection.py.

Cees de Groot - answers SH. Confirms advantage of not having to
write formal schema, migrating from PostgreSQL to ZODB for that.
File Storage faster than Oracle and is basic transaction log
so nothing could be more reliable.
http://lists.zope.org/pipermail/zope-dev/2001-May/011124.html
Are people using ZODB for non-Zope data? I'd be very interested to discuss
things like emulating extents, patterns for indexing, 

RE: oodb philosophics ;) was: Re: [Zope-dev] Experiments withORMapping

2001-05-14 Thread Albert Langer

[Karl Anderson]
Casey Duncan [EMAIL PROTECTED] writes:

 I am not arguing necessarily for SQL as a query language for the ZODB.
 Although it is an accepted standard, but not a perfect one by any means
 especially for OODBs. Its appeal lies mainly in the high level of
 community familiarity and the plethora of SQL software to borrow from.

Does anyone have an opinion on the possible usefulness of XPath,
XQuery, and other XML standards for this?  Someone suggested (on the
zope-xml wiki) that it would be nice to be able to drop in a cataloger
that supported a presumably standard and presumably well-known XML
query syntax, and which would work throughout the database because
Zope objects would support DOM.

This is all speculation, and I personally don't know much right now
about XML database interfaces and how finished or well-regarded they
are.

[Albert]
An excellent introduction to this topic is:

Putting XML in context with hierarchical, relational, and
object-oriented models by David Mertz.
ftp://www6.software.ibm.com/software/developer/library/x-matters8.pdf

Author is a python developer with lots of interesting XML stuff.
See also his xml_matters 1 and 2 for xml_object and xml_pickle with
much nicer pythonic syntax instead of using DOM directly.

Article is also *essential* background for the distinction between
Object Mapping and Object Relational Mapping which needs to be
understood by anyone participating in this discussion.

An example of a python ODBMS with some partial support for OQL is 4ODS
from 4 Suite, which uses a very natural pythonic syntax for objects
stored in and queried from PostgreSQL:

Following is from 4Suite-docs-0.11/4Suite-0.11/html/4ODS-userguide.html
available via:

http://4suite.org/download.html#4Suite_Documentation


How to use the system (a very basic walk through)

First create a ODL file that represents what you want to store in test.odl

module simple {
  class person {
attribute string name;
attribute double weight;
relationship Person spouse inverse Person ::spouse_of;
relationship Person spouse_of inverse Person ::spouse;
relationship listPerson children  inverse Person ::child_of;
relationship Person child_of inverse Person ::children;
  };
  class employee (extends person) {
attribute string id;
  };
};

Now create a new database and initialize

 #OdlParse -ifp test test.odl

Now write some python code to do stuff with these people

#!/usr/bin/python

#Every thing that is persisten must be done inside a transaction and open
database
from Ft.Ods import Database
db = Database.Database()
db.open('test')

tx = db.new()
tx.begin()

#Create a new instance of some objects
import person
import employee
dad = employee.new()
mom = person.new()
son1 = person.new()
son2 = person.new()
daughter = person.new()

#Set some attributes
dad.name = Pops
mom.name = Ma
son1.name = Joey
son2.name = Bobby
daughter.name = Betty
dad.weight = 240.50

#We can set attributes not defined in the ODL but they will not persist
mom.address = 1234 Error Way


#Set some relationships

#First set a one to one relationship
dad.spouse = mom

#Or we could have done it via the ODMG spec
#dad.form_spouse(mom)

#Add some children to the dad (our data model does not let mom have
children.  We'd need a family struct (left up to the reader)

dad.add_children(son1)
#We can create relationships both ways
son2.form_child_of(dad)

#Shortcut for adding
dad.children = daughter

#Now root the family to some top level object.
db.bind(dad,The Fam)

#Make it so
tx.commit()

#Out side of a transaction we can still access the objects.
#However, any changes we make will not persist.
#NOTE, because 4ODS caches relationships, any relationships that were not
traversed during the
#transaction, cannot be traversed now because an object cannot be loaded
from the db outside
#of a transaction.
print dad.name

#Start a new tx to fetch

tx = db.new()
tx.begin()

newDad = db.lookup(The Fam)

print newDad.name
print newDad.children[0].name
print newDad.spouse

#Discard this transaction
tx.abort()

Ft/Ods/test_suite and Ft/Ods/demo are good places to look for more examples
^^

See also:
http://www.xml.com/pub/a/2000/10/11/rdf/

Some other relevant references are:

Extraction of DBMS catalogs to XML using python.
http://hyperschema.sourceforge.net./

PostgreSQL as XML repository
http://zvon.org/index.php?nav_id=61
http://hopla.sourceforge.net/doc/README

Note that none of this has much to do with the original topic of
Object-*Relational* Mapping.

*Essential* background for understanding what an object-relational
persistence layer looks like is:

http://www.ambysoft.com/persistenceLayer.html

It isn't very long and there *absolutely* isn't any point discussing
how to design such an OR persistence layer without first reading
and fully understanding it. (I say that after having carefully
studied all the messages in this discussion - though I also said
so before ;-)

The rest of that web site has 

Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread Joachim Werner

 Now, it may be useful to provide a management interface for defining the
 schema mapping.  I haven't approached that yet; AFAICT this is where the
 work done on SmartObjects and DBObjects would be very useful.  Initially
 I was planning for people to code the mapping purely in Python so we
 could gain experience and find common patterns before inventing a UI.

Our initial plans for SmartObjects (if I get Stephan right) are to define
all objects in Python code (as DBObjects do now), but to also provide two
alternative ways of creating the necessary code:

a) by importing and compiling an XML object description (which might come
from a UML tool)
b) via a ZMI interface similar to the ZClass GUI (which may or may not use
the XML schemas, depending on the actual implementation)

Of course it would be necessary to dynamically refresh the newly generated
filesystem-based code. The other issue that applies in general to all
filesystem-based code is that it is not directly available over ZEO as it is
not in the ZODB. But that should not be a very large problem ...


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread Joachim Werner

 My thought on this, is that you will be forced to explicitly consider the
 nature of this relationship and its storage at the application level.  If
 you write explicitly for ZODB, you might use a BTree, for example.  Or
 perhaps you'd have some kind of global container for the Tasks, with a
 Catalog to index them, and a method on TaskPerformer to query the Catalog.

 How would you propose to map this to an RDMS?  Well, substituting catalog
 queries might be relatively straightforward, but perhaps rather time
 consuming to develop in a generic way.  The BTree might give you a bit
more
 trouble.  Probably you'd have to end up using some sort of generic
 queryable containers that translated to BTrees or Catalogs or RDBMS
tables,
 depending...

 And lo!  You now have an application framework.  Oops.

I'll try to somehow get your and Shane's opinions together a bit: I think
Shane's approach does most of the job (of OR mapping) if the objects we are
talking about are relatively simple and the application domain is relatively
typical for what Zope can do well. E.g. the composite objects we have in
mind for the groupware (user credentials coming from LDAP, rest in the ZODB,
or file stored in the filesystem metadata in SQL DB, rest in ZODB) can
easily be modelled this way. And the existing Zope API (manage_AddX,
manage_changeProperties etc.) would do most of the job.

But the problems start with querying. Can the ZCatalog do it? Does it make
sense to implement a SQLZCatalog instead of just offering native SQL query
objects or an abstract query language like SkinSkript? To put it simply: The
reason why we want to develop a framework (or framework extension) is that
the Zope framework just doesn't do the stuff we need yet. BTW, the reason
why we didn't just use ZPatterns was very similar: ZPatterns makes it easy
to write an abstract implementation of a project, but you still have to
write all the glue code between the storage and the application layer by
hand. What we want (and what TransWarp is about, with a slightly different
focus) is that we can just map an attribute to a data source and all the
glue code (either ZODB or SQL statements) is auto-generated.

If the new framework just extends the existing Zope API, I don't see a
problem. ZPatterns did not, but maybe it is just not that easy to do so (and
not all parts of the Zope API really should bear this name ...).

The really good thing about the whole discussion is that at the end we will
see much better where we should put things. We (the SmartObjects people at
iuveno) do not really want to have a completely new world of SmartObjects
that will require a new API to learn. We just want to add new functionality
to Zope, be it in the core or as mix-in classes, or through-the-web
products.

Joachim


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread Joachim Werner

 My thought on this, is that you will be forced to explicitly consider the
 nature of this relationship and its storage at the application level.  If
 you write explicitly for ZODB, you might use a BTree, for example.  Or
 perhaps you'd have some kind of global container for the Tasks, with a
 Catalog to index them, and a method on TaskPerformer to query the Catalog.

 How would you propose to map this to an RDMS?  Well, substituting catalog
 queries might be relatively straightforward, but perhaps rather time
 consuming to develop in a generic way.  The BTree might give you a bit
more
 trouble.  Probably you'd have to end up using some sort of generic
 queryable containers that translated to BTrees or Catalogs or RDBMS
tables,
 depending...

 And lo!  You now have an application framework.  Oops.

I'll try to somehow get your and Shane's opinions together a bit: I think
Shane's approach does most of the job (of OR mapping) if the objects we are
talking about are relatively simple and the application domain is relatively
typical for what Zope can do well. E.g. the composite objects we have in
mind for the groupware (user credentials coming from LDAP, rest in the ZODB,
or file stored in the filesystem metadata in SQL DB, rest in ZODB) can
easily be modelled this way. And the existing Zope API (manage_AddX,
manage_changeProperties etc.) would do most of the job.

But the problems start with querying. Can the ZCatalog do it? Does it make
sense to implement a SQLZCatalog instead of just offering native SQL query
objects or an abstract query language like SkinSkript? To put it simply: The
reason why we want to develop a framework (or framework extension) is that
the Zope framework just doesn't do the stuff we need yet. BTW, the reason
why we didn't just use ZPatterns was very similar: ZPatterns makes it easy
to write an abstract implementation of a project, but you still have to
write all the glue code between the storage and the application layer by
hand. What we want (and what TransWarp is about, with a slightly different
focus) is that we can just map an attribute to a data source and all the
glue code (either ZODB or SQL statements) is auto-generated.

If the new framework just extends the existing Zope API, I don't see a
problem. ZPatterns did not, but maybe it is just not that easy to do so (and
not all parts of the Zope API really should bear this name ...).

The really good thing about the whole discussion is that at the end we will
see much better where we should put things. We (the SmartObjects people at
iuveno) do not really want to have a completely new world of SmartObjects
that will require a new API to learn. We just want to add new functionality
to Zope, be it in the core or as mix-in classes, or through-the-web
products.

Joachim


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] manage_workspace = index_html

2001-05-14 Thread Michel Pelletier

On Mon, 14 May 2001, The Doctor What wrote:

 What's going on?  Is this a Mozilla problem? 

Yes.

 If so, why is lynx
 doing the same?

Because lynx is broken too.

 How should I go about trouble shooting it?

The problem is the client does not provide the right Basic authentication
credentials for the frame like it's supposed to.  Zope then assumes that
you are just an anonymous user, and shows you the only 'managment tab' you
have permission to see, 'View'.

If I recall, this have been broken an fixed in mozilla a few times
now.  Other browsers do this wrong too, like w3m.

-Michel


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] Proposal: AuthenticatedRole

2001-05-14 Thread Chris McDonough

Hi all,

If you're concerned or curious, please review this proposal on the fishbowl
regarding adding a default role to Zope named 'Authenticated':

http://dev.zope.org/Wikis/DevSite/Proposals/AuthenticatedRole


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Experiments with ORMapping

2001-05-14 Thread ender

Hello,

i want to thank DC (jim, shane, and paul) for inviting me to come to the new 
DC offices. i had a great time and learned a bunch... and met 
the BFDL.

i gave a quick overview of the smartobjects design/framework and  jim and 
shane presented some ideas for an or mapping for the zodb.

to be honest it seemed both extremely elegant and slightly disturbing at the 
sametime. I've grown accustomed to treating the ZODB as a magic black box, 
that just works. integrating an or layer at this low level disturbs me as it 
meant that application developers would need to tinker at the lowest level of 
the zope framework, plus the dangers that to properly support application 
development the zodb would need to be bloated with accessory apis. at the 
same time it offers a really elegant way to make the or mapping available to 
python programmers on a pure python level (which is something i'm also 
interested in). 

for both approaches there are some similiar problems that must be tackled, 
namely definition of the or mapping, generation of stub python classes, 
integration with zope notions (permission, etc). so hopefully discussion here 
will flesh out, to the point of making a choice clear.

i'm primarily focused on doing the or mapping definition in xml, as its 
language neutral and allows me to leverage other tools (like torque 
java.apache.org/turbine/torque.html) to assist in application 
development/porting.

i've thought about it for a few days, and while i'm not convinced completely
that a zodb approach is the best method, i'm going to start work on the some 
of the support classes nesc. for moving forward, namely some of  the xml 
model/generators outlined in my design (embryonic)
(http://www.zope.org/Members/k_vertigo/smartobjects)

most of my concerns echo the issues that philip has brought up re the 
need of application developers. i do feel some if not most of these issues
can be solved, i'm just wondering if for zope this is the best solution. for
pure python development it seems alright. for zope, i wonder if a higher 
level application framework is more suitable. 

to clarify some existing points in this thread to date.

- (IMO) zpatterns is about two things, delegation and data abstraction at
the application level. the zpatterns framework doesn't assist in automating
this and its up to the integrator/developer to manually wire together all the 
app/business objects. the smartobjects framework is about automating
development and presenting easier development interfaces for both 
developers and designers. this isn't to say that such automation and 
interfaces couldn't be built on top of zpatterns, just to say that there are 
some different design goals. 

- one of the primary goals of smartobjects is easy integration with zope in a
manner that is natural for zope developers. this does not mandate a new db 
access api, its about facilitating automation of this api so that the objects 
can be treated as normaly zope objects. both approaches (smartobjects and or 
zodb) must still tackle similiar fundamental issues to integrate with zope. 
on the zodb level this seems complicated to me (which betrays my incomplete 
understanding of the zodb internals), plus it requires some additions to the 
zodb interface to allow effective application usage (namely a generic query 
mechanism, and probably interogation of a class information). 

- one of my main objectives in developing an or mapping is to use the
acs4 (www.arsdigita.com) applications natively (or as close as possible) 
within zope.

- my smartobjects design (www.zope.org/Members/k_vertigo/smartobjects) is
mine, in that smartobjects is primarily a iuveno project this does not mean 
it is smartobjects. stephen richter (of iuveno) liked parts of the design and 
will likely use it parts of it in a iuveno implementation but what comes out 
of iuveno is not limited or beholden in anyway by my proposal. the key focus 
of the proposal is a metamodel (schema) based approach.

questions

- i think any integration of or layer should be complementary to the zope 
framework. shane's existing examples and discussion of an or layer seem to be 
more geared towards replacing the zodb, than complementing. is the idea to 
use some sort of zodb product as a mounted storage?, or is this a replacement 
strategy.

cheers

kapil

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )