Re: [Zope] persistence and dictionaries
Matt [EMAIL PROTECTED] wrote: Chris, this was my original confusion the two places below where you say "You can put instances which do not inherit from Persistence.Persistent in your database. They just won't "stick". They'll hang around until the server is restarted or for an undetermined amount of time during normal operations." "No. It'll work for "a while" or until the server is restarted. :-)" actually do persist after restarts ... that's what confused me, they wouldn't go away and they should!! I think that the confusion here lies in the word, "Peristent". Because it uses the standard Python pickling mechanism, the ZODB can store all kinds of objects (integers, strings, etc., as well as class instances). The ZODB is, however, broken up into a collection of individual records, each of which contains a series of pickles (one per transaction). Objects which are not "ZODB aware" get stored in the pickle of their container (actually, the nearest one which *is* persistence-aware). Deriving your class from Persistence.Persistent says to the ZODB, "put me in my *own* pickle, not it the pickle of my container". The other thing Persistence does is hook the '__setattr__' method, so that changes you make to the object's attributes automatically register it with Zope's transaction machinery; this registration means that a new pickle gets saved to the ZODB at the end of the transaction. Objects which do not derive from Persistence must handle this registration themselves, or else risk having such changes be lost whenever they are reloaded. Globals.PersistentDictionary is a wrapper class which does does this kind of registration in its '__setitem__' method. HTH, Tres. -- === Tres Seaver[EMAIL PROTECTED] Digital Creations "Zope Dealers" http://www.zope.org ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] persistence and dictionaries
This makes sense with some of my experimentation. So I am correct to assume now that any member classes that I include into my Product that don't themselves derive from Persistence.Persistent will become pickled into my products pickle, and will do whenever __setattr__ method of my product is called. I am wondering if this also answers my question in my very last email titled Re: [Zope] persistence and dictionaries - new light where I found that anything called within my manage_edit(self, title, REQUEST=None) method was persisted, but that if I did the same things in another method(also in the Product) and called that method from a dtmlMethod then things would not be persisted. thanks for all the help from everyone, this is really invaluable. regards Matt Tres Seaver wrote: Matt [EMAIL PROTECTED] wrote: Chris, this was my original confusion the two places below where you say "You can put instances which do not inherit from Persistence.Persistent in your database. They just won't "stick". They'll hang around until the server is restarted or for an undetermined amount of time during normal operations." "No. It'll work for "a while" or until the server is restarted. :-)" actually do persist after restarts ... that's what confused me, they wouldn't go away and they should!! I think that the confusion here lies in the word, "Peristent". Because it uses the standard Python pickling mechanism, the ZODB can store all kinds of objects (integers, strings, etc., as well as class instances). The ZODB is, however, broken up into a collection of individual records, each of which contains a series of pickles (one per transaction). Objects which are not "ZODB aware" get stored in the pickle of their container (actually, the nearest one which *is* persistence-aware). Deriving your class from Persistence.Persistent says to the ZODB, "put me in my *own* pickle, not it the pickle of my container". The other thing Persistence does is hook the '__setattr__' method, so that changes you make to the object's attributes automatically register it with Zope's transaction machinery; this registration means that a new pickle gets saved to the ZODB at the end of the transaction. Objects which do not derive from Persistence must handle this registration themselves, or else risk having such changes be lost whenever they are reloaded. Globals.PersistentDictionary is a wrapper class which does does this kind of registration in its '__setitem__' method. HTH, Tres. -- === Tres Seaver[EMAIL PROTECTED] Digital Creations "Zope Dealers" http://www.zope.org ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] persistence and dictionaries
Thanks for the reply, that is really useful. There are a couple of things
though that still don't add up. Firstly, you say below, as do all the
ZODB
documents that "Custom" classes can certainly persist, they just need to
mix in
the "Persistence.Persistent" class as a base class. Well, in my example I
attached in my first email, my product certainly has
Persistence.Persistent,
but my second class that I add to this one does not, yet it still
persists.
There was an email sometime ago on the mailing list that told someone that
this
was why their product instances disappearing from the ZODB.
(the ref for the original email is :
http://www.egroups.com/message/zope/44263
... I can't find the reply again.)
You can put instances which do not inherit from Persistence.Persistent in
your database. They just won't "stick". They'll hang around until the
server is restarted or for an undetermined amount of time during normal
operations.
So my current understanding would be that any classes you want to add in
do not
need to derive from Persistence.Persistent, and if it is pickleable then
all
should be fine if you call on instances of that object within you product.
No. It'll work for "a while" or until the server is restarted. :-)
The next part that worried me came from the "python product tutorial"
http://www.zope.org/Members/hathawsh/PythonProductTutorial
This stated that the class dictionary self.votes = {} needed to be changed
to
self._votes = Globals.PersistentMapping() so that updates to it persist.
Hence my query about dictionaries.
This was for convenience, I'd imagine.
I also noticed your comment about __setstate__ . What is it about this
that is
dangerous.
Nothing implicitly dangerous, but it can get confusing if you have multiple
revisions of your product and you use variables caused by __setstate__.
Also, once you add a __setstate__ which modifies the object in-place,
there's a likelihood that it can never go away (you're can never be sure if
all instances have been updated).
Recently I built a product out of some python classes I wrapped
around 4DOM, and since 4DOM documents do not seem to persist(well the
document
does, but it loses all its children), then I persisted them to the local
file
system, since I needed to do that anyway for what I was doing. Setstate
seemed
to work nicely to bring them back, though watching its behaviour I noticed
that
it was called very often by zope.
Sure, that works... although at that point you're creating your own object
database. :-)
Chris McDonough wrote:
All pickleable Python primitive types (strings, dictionaries, lists,
Nones,
integers, floats, longs, etc.) can live in the ZODB. They can persist
just
like instances that inherit from the Persistent class.
I think you read a little too much in to the fact that you need to
"treat
mutable objects immutably" when working with them in the ZODB. This
statement doesn't mean that these kinds of objects can't be saved in the
ZODB, it just means you need to treat them specially when putting them
in
the database.
For instance, if you were doing this inside of an external method:
def amethod(self):
self.mydict = {}
self.mydict['a'] = 1
(where self is the persistent object that is usually the external
method's
"container")
It wouldn't work as you expected. Although you'd see an 'a' in mydict
for a
little while in further accesses to it, 'mydict' would eventaully show
up as
an empty dictionary on the first access of it after it was expired from
the
RAM cache (after it was 'ghosted'), because the last thing that the ZODB
"saw" (via the __setattr__ on 'self' and a subsequent transaction) was
you
setting a empty dictionary.
Persistent objects (like "self" in the above example) are only smart
enough
to notice changes to themselves that happen through their __setattr__
(e.g.
self.mydict = {} calls self's __setattr__). Mutating the attribute
'mydict'
above "in-place" (via self.mydict['a'] = 1) does not trigger self's
__setattr__, so the ZODB never notices that "mydict" got changed.
There are two ways to handle this. The first is to treat mutable
attributes
"immutably" via assigning to a temporary variable and then making sure
the
persistent container's __setattr__ gets called:
def amethod(self):
dict = {}
dict['a'] = 1
self.mydict = dict # trigger persistence mechanism implicitly
The second is to use the _p_changed attribute of the persistent object
on
which the primitive is set. This explcitly tells the persistence system
to
include the object on which it's set into the current transaction:
def amethod(self):
self.mydict = {}
self.mydict['a'] = 1
self._p_changed = 1 # trigger persistence mechanism manually
Variations on this theme extend to list methods too (e.g. list.append,
list.pop, etc.)
"Custom" classes can certainly persist, they just need
Re: [Zope] persistence and dictionaries
Huh. If they do, it's by chance only. I'd be hard-pressed to explain it.
Do they inherit from *anything*?
- Original Message -
From: "Matt" [EMAIL PROTECTED]
To: "Chris McDonough" [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Friday, December 08, 2000 2:30 PM
Subject: Re: [Zope] persistence and dictionaries
Chris, this was my original confusion the two places below where you
say
"You can put instances which do not inherit from Persistence.Persistent in
your database. They just won't "stick". They'll hang around until the
server is restarted or for an undetermined amount of time during normal
operations."
"No. It'll work for "a while" or until the server is restarted. :-)"
actually do persist after restarts ... that's what confused me, they
wouldn't
go away and they should!!
regards
Matt
Chris McDonough wrote:
Thanks for the reply, that is really useful. There are a couple of
things
though that still don't add up. Firstly, you say below, as do all the
ZODB
documents that "Custom" classes can certainly persist, they just need
to
mix in
the "Persistence.Persistent" class as a base class. Well, in my
example I
attached in my first email, my product certainly has
Persistence.Persistent,
but my second class that I add to this one does not, yet it still
persists.
There was an email sometime ago on the mailing list that told someone
that
this
was why their product instances disappearing from the ZODB.
(the ref for the original email is :
http://www.egroups.com/message/zope/44263
... I can't find the reply again.)
You can put instances which do not inherit from Persistence.Persistent
in
your database. They just won't "stick". They'll hang around until the
server is restarted or for an undetermined amount of time during normal
operations.
So my current understanding would be that any classes you want to add
in
do not
need to derive from Persistence.Persistent, and if it is pickleable
then
all
should be fine if you call on instances of that object within you
product.
No. It'll work for "a while" or until the server is restarted. :-)
The next part that worried me came from the "python product tutorial"
http://www.zope.org/Members/hathawsh/PythonProductTutorial
This stated that the class dictionary self.votes = {} needed to be
changed
to
self._votes = Globals.PersistentMapping() so that updates to it
persist.
Hence my query about dictionaries.
This was for convenience, I'd imagine.
I also noticed your comment about __setstate__ . What is it about
this
that is
dangerous.
Nothing implicitly dangerous, but it can get confusing if you have
multiple
revisions of your product and you use variables caused by __setstate__.
Also, once you add a __setstate__ which modifies the object in-place,
there's a likelihood that it can never go away (you're can never be sure
if
all instances have been updated).
Recently I built a product out of some python classes I wrapped
around 4DOM, and since 4DOM documents do not seem to persist(well the
document
does, but it loses all its children), then I persisted them to the
local
file
system, since I needed to do that anyway for what I was doing.
Setstate
seemed
to work nicely to bring them back, though watching its behaviour I
noticed
that
it was called very often by zope.
Sure, that works... although at that point you're creating your own
object
database. :-)
Chris McDonough wrote:
All pickleable Python primitive types (strings, dictionaries, lists,
Nones,
integers, floats, longs, etc.) can live in the ZODB. They can
persist
just
like instances that inherit from the Persistent class.
I think you read a little too much in to the fact that you need to
"treat
mutable objects immutably" when working with them in the ZODB. This
statement doesn't mean that these kinds of objects can't be saved in
the
ZODB, it just means you need to treat them specially when putting
them
in
the database.
For instance, if you were doing this inside of an external method:
def amethod(self):
self.mydict = {}
self.mydict['a'] = 1
(where self is the persistent object that is usually the external
method's
"container")
It wouldn't work as you expected. Although you'd see an 'a' in
mydict
for a
little while in further accesses to it, 'mydict' would eventaully
show
up as
an empty dictionary on the first access of it after it was expired
from
the
RAM cache (after it was 'ghosted'), because the last thing that the
ZODB
"saw" (via the __setattr__ on 'self' and a subsequent transaction)
was
you
setting a empty dictionary.
Persistent objects
Re: [Zope] persistence and dictionaries
Chris, this was my original confusion the two places below where you say
"You can put instances which do not inherit from Persistence.Persistent in
your database. They just won't "stick". They'll hang around until the
server is restarted or for an undetermined amount of time during normal
operations."
"No. It'll work for "a while" or until the server is restarted. :-)"
actually do persist after restarts ... that's what confused me, they wouldn't
go away and they should!!
regards
Matt
Chris McDonough wrote:
Thanks for the reply, that is really useful. There are a couple of things
though that still don't add up. Firstly, you say below, as do all the
ZODB
documents that "Custom" classes can certainly persist, they just need to
mix in
the "Persistence.Persistent" class as a base class. Well, in my example I
attached in my first email, my product certainly has
Persistence.Persistent,
but my second class that I add to this one does not, yet it still
persists.
There was an email sometime ago on the mailing list that told someone that
this
was why their product instances disappearing from the ZODB.
(the ref for the original email is :
http://www.egroups.com/message/zope/44263
... I can't find the reply again.)
You can put instances which do not inherit from Persistence.Persistent in
your database. They just won't "stick". They'll hang around until the
server is restarted or for an undetermined amount of time during normal
operations.
So my current understanding would be that any classes you want to add in
do not
need to derive from Persistence.Persistent, and if it is pickleable then
all
should be fine if you call on instances of that object within you product.
No. It'll work for "a while" or until the server is restarted. :-)
The next part that worried me came from the "python product tutorial"
http://www.zope.org/Members/hathawsh/PythonProductTutorial
This stated that the class dictionary self.votes = {} needed to be changed
to
self._votes = Globals.PersistentMapping() so that updates to it persist.
Hence my query about dictionaries.
This was for convenience, I'd imagine.
I also noticed your comment about __setstate__ . What is it about this
that is
dangerous.
Nothing implicitly dangerous, but it can get confusing if you have multiple
revisions of your product and you use variables caused by __setstate__.
Also, once you add a __setstate__ which modifies the object in-place,
there's a likelihood that it can never go away (you're can never be sure if
all instances have been updated).
Recently I built a product out of some python classes I wrapped
around 4DOM, and since 4DOM documents do not seem to persist(well the
document
does, but it loses all its children), then I persisted them to the local
file
system, since I needed to do that anyway for what I was doing. Setstate
seemed
to work nicely to bring them back, though watching its behaviour I noticed
that
it was called very often by zope.
Sure, that works... although at that point you're creating your own object
database. :-)
Chris McDonough wrote:
All pickleable Python primitive types (strings, dictionaries, lists,
Nones,
integers, floats, longs, etc.) can live in the ZODB. They can persist
just
like instances that inherit from the Persistent class.
I think you read a little too much in to the fact that you need to
"treat
mutable objects immutably" when working with them in the ZODB. This
statement doesn't mean that these kinds of objects can't be saved in the
ZODB, it just means you need to treat them specially when putting them
in
the database.
For instance, if you were doing this inside of an external method:
def amethod(self):
self.mydict = {}
self.mydict['a'] = 1
(where self is the persistent object that is usually the external
method's
"container")
It wouldn't work as you expected. Although you'd see an 'a' in mydict
for a
little while in further accesses to it, 'mydict' would eventaully show
up as
an empty dictionary on the first access of it after it was expired from
the
RAM cache (after it was 'ghosted'), because the last thing that the ZODB
"saw" (via the __setattr__ on 'self' and a subsequent transaction) was
you
setting a empty dictionary.
Persistent objects (like "self" in the above example) are only smart
enough
to notice changes to themselves that happen through their __setattr__
(e.g.
self.mydict = {} calls self's __setattr__). Mutating the attribute
'mydict'
above "in-place" (via self.mydict['a'] = 1) does not trigger self's
__setattr__, so the ZODB never notices that "mydict" got changed.
There are two ways to handle this. The first is to treat mutable
attributes
"immutably" via assigning to a temporary variable and then making sure
the
persistent container's __setattr__ gets called:
Re: [Zope] persistence and dictionaries
Ok, here are some of the offending bits out of my boringplus product
which I can send again if people want. I have
another product that keeps a dictionary of cookies so I can see how many
times someone uses the back button to
access the same forms page again. This worked as expected, where I had
to issue an _p_changed=1 to get them to
persist. But I am doing a similar thing below, or so I thought, once
with a dictionary in the product and once in a
dictionary within a class that does not inherit Persistent. This is
what is confusing. Through restarts etc, everything
perisists
class Boring(
OFS.SimpleItem.Item, # A simple Principia object. Not Folderish.
Persistent,# Make us persistent. Yaah!
Acquisition.Implicit, # Uh, whatever.
AccessControl.Role.RoleManager # Security manager.
):
def __init__(self, id, title=''):
self._things = {}
self._more = myClass.MoreStuff()
def manage_edit(self, title, REQUEST=None):
if REQUEST is not None:
if REQUEST.has_key('thing'):
self._things[REQUEST['thing']] = REQUEST['thing_value']
self.addToMore(REQUEST['thing'],REQUEST['thing_value'])
def addToMore(self,name,value):
self._more.addStuff(name,value)
return ""
class MoreStuff:
def __init__(self):
self._stuff = {'animal':'monkey'}
def addStuff(self,stuff_name,stuff_item):
On Sat, 09 Dec 2000, Jonothan Farr wrote:
Maybe I'm mistaken, but it seems like you can put an instance of an
object that
doesn't inherit from Persistent into the ZODB just fine, but its
contents won't
persist, so you'll always end up with a copy of the object as it was
first added
to the database.
--jfarr
- Original Message -
From: "Chris McDonough" [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Friday, December 08, 2000 11:57 AM
Subject: Re: [Zope] persistence and dictionaries
Huh. If they do, it's by chance only. I'd be hard-pressed to
explain it.
Do they inherit from *anything*?
- Original Message -
From: "Matt" [EMAIL PROTECTED]
To: "Chris McDonough" [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Friday, December 08, 2000 2:30 PM
Subject: Re: [Zope] persistence and dictionaries
Chris, this was my original confusion the two places below
where you
say
"You can put instances which do not inherit from
Persistence.Persistent in
your database. They just won't "stick". They'll hang around
until the
server is restarted or for an undetermined amount of time during
normal
operations."
"No. It'll work for "a while" or until the server is restarted.
:-)"
actually do persist after restarts ... that's what confused me,
they
wouldn't
go away and they should!!
regards
Matt
Chris McDonough wrote:
Thanks for the reply, that is really useful. There are a
couple of
things
though that still don't add up. Firstly, you say below, as do
all the
ZODB
documents that "Custom" classes can certainly persist, they
just need
to
mix in
the "Persistence.Persistent" class as a base class. Well, in
my
example I
attached in my first email, my product certainly has
Persistence.Persistent,
but my second class that I add to this one does not, yet it
still
persists.
There was an email sometime ago on the mailing list that told
someone
that
this
was why their product instances disappearing from the ZODB.
(the ref for the original email is :
http://www.egroups.com/message/zope/44263
... I can't find the reply again.)
You can put instances which do not inherit from
Persistence.Persistent
in
your database. They just won't "stick". They'll hang around
until the
server is restarted or for an undetermined amount of time during
normal
operations.
So my current understanding would be that any classes you want
to add
in
do not
need to derive from Persistence.Persistent, and if it is
pickleable
then
all
should be fine if you call on instances of that object within
you
product.
No. It'll work for "a while" or until the server is restarted.
:-)
The next part that worried me came from the "python product
tutorial"
http://www.zope.org/Members/hathawsh/PythonProductTutorial
This stated that the class dictionary self.votes = {} needed
to be
changed
to
self._votes = Globals.PersistentMapping() so that updates to
it
persist.
Hence my query about dictionaries.
This was for convenience, I'd imagine.
I also noticed your comment about __setstate__ . What is it
about
this
that is
dangerous.
Nothing implicitly dangerous, but it can get confusing if you
have
multiple
revisions of y
Re: [Zope] persistence and dictionaries
Maybe I'm mistaken, but it seems like you can put an instance of an object that
doesn't inherit from Persistent into the ZODB just fine, but its contents won't
persist, so you'll always end up with a copy of the object as it was first added
to the database.
--jfarr
- Original Message -
From: "Chris McDonough" [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Friday, December 08, 2000 11:57 AM
Subject: Re: [Zope] persistence and dictionaries
Huh. If they do, it's by chance only. I'd be hard-pressed to explain it.
Do they inherit from *anything*?
- Original Message -
From: "Matt" [EMAIL PROTECTED]
To: "Chris McDonough" [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Friday, December 08, 2000 2:30 PM
Subject: Re: [Zope] persistence and dictionaries
Chris, this was my original confusion the two places below where you
say
"You can put instances which do not inherit from Persistence.Persistent in
your database. They just won't "stick". They'll hang around until the
server is restarted or for an undetermined amount of time during normal
operations."
"No. It'll work for "a while" or until the server is restarted. :-)"
actually do persist after restarts ... that's what confused me, they
wouldn't
go away and they should!!
regards
Matt
Chris McDonough wrote:
Thanks for the reply, that is really useful. There are a couple of
things
though that still don't add up. Firstly, you say below, as do all the
ZODB
documents that "Custom" classes can certainly persist, they just need
to
mix in
the "Persistence.Persistent" class as a base class. Well, in my
example I
attached in my first email, my product certainly has
Persistence.Persistent,
but my second class that I add to this one does not, yet it still
persists.
There was an email sometime ago on the mailing list that told someone
that
this
was why their product instances disappearing from the ZODB.
(the ref for the original email is :
http://www.egroups.com/message/zope/44263
... I can't find the reply again.)
You can put instances which do not inherit from Persistence.Persistent
in
your database. They just won't "stick". They'll hang around until the
server is restarted or for an undetermined amount of time during normal
operations.
So my current understanding would be that any classes you want to add
in
do not
need to derive from Persistence.Persistent, and if it is pickleable
then
all
should be fine if you call on instances of that object within you
product.
No. It'll work for "a while" or until the server is restarted. :-)
The next part that worried me came from the "python product tutorial"
http://www.zope.org/Members/hathawsh/PythonProductTutorial
This stated that the class dictionary self.votes = {} needed to be
changed
to
self._votes = Globals.PersistentMapping() so that updates to it
persist.
Hence my query about dictionaries.
This was for convenience, I'd imagine.
I also noticed your comment about __setstate__ . What is it about
this
that is
dangerous.
Nothing implicitly dangerous, but it can get confusing if you have
multiple
revisions of your product and you use variables caused by __setstate__.
Also, once you add a __setstate__ which modifies the object in-place,
there's a likelihood that it can never go away (you're can never be sure
if
all instances have been updated).
Recently I built a product out of some python classes I wrapped
around 4DOM, and since 4DOM documents do not seem to persist(well the
document
does, but it loses all its children), then I persisted them to the
local
file
system, since I needed to do that anyway for what I was doing.
Setstate
seemed
to work nicely to bring them back, though watching its behaviour I
noticed
that
it was called very often by zope.
Sure, that works... although at that point you're creating your own
object
database. :-)
Chris McDonough wrote:
All pickleable Python primitive types (strings, dictionaries, lists,
Nones,
integers, floats, longs, etc.) can live in the ZODB. They can
persist
just
like instances that inherit from the Persistent class.
I think you read a little too much in to the fact that you need to
"treat
mutable objects immutably" when working with them in the ZODB. This
statement doesn't mean that these kinds of objects can't be saved in
the
ZODB, it just means you need to treat them specially when putting
them
in
the database.
For instance, if you were doing this inside of an external method:
def amethod(self):
self.mydict = {}
Re: [Zope] persistence and dictionaries
Oops. You snipped the contents of MoreStuff.addStuff(). No way to tell what's
going on without that!
--jfarr
- Original Message -
From: "Matt" [EMAIL PROTECTED]
To: "Jonothan Farr" [EMAIL PROTECTED]; [EMAIL PROTECTED]
Sent: Friday, December 08, 2000 2:39 PM
Subject: Re: [Zope] persistence and dictionaries
Ok, here are some of the offending bits out of my boringplus product
which I can send again if people want. I have
another product that keeps a dictionary of cookies so I can see how many
times someone uses the back button to
access the same forms page again. This worked as expected, where I had
to issue an _p_changed=1 to get them to
persist. But I am doing a similar thing below, or so I thought, once
with a dictionary in the product and once in a
dictionary within a class that does not inherit Persistent. This is
what is confusing. Through restarts etc, everything
perisists
class Boring(
OFS.SimpleItem.Item, # A simple Principia object. Not Folderish.
Persistent,# Make us persistent. Yaah!
Acquisition.Implicit, # Uh, whatever.
AccessControl.Role.RoleManager # Security manager.
):
def __init__(self, id, title=''):
self._things = {}
self._more = myClass.MoreStuff()
def manage_edit(self, title, REQUEST=None):
if REQUEST is not None:
if REQUEST.has_key('thing'):
self._things[REQUEST['thing']] = REQUEST['thing_value']
self.addToMore(REQUEST['thing'],REQUEST['thing_value'])
def addToMore(self,name,value):
self._more.addStuff(name,value)
return ""
class MoreStuff:
def __init__(self):
self._stuff = {'animal':'monkey'}
def addStuff(self,stuff_name,stuff_item):
On Sat, 09 Dec 2000, Jonothan Farr wrote:
Maybe I'm mistaken, but it seems like you can put an instance of an
object that
doesn't inherit from Persistent into the ZODB just fine, but its
contents won't
persist, so you'll always end up with a copy of the object as it was
first added
to the database.
--jfarr
- Original Message -
From: "Chris McDonough" [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Friday, December 08, 2000 11:57 AM
Subject: Re: [Zope] persistence and dictionaries
Huh. If they do, it's by chance only. I'd be hard-pressed to
explain it.
Do they inherit from *anything*?
- Original Message -
From: "Matt" [EMAIL PROTECTED]
To: "Chris McDonough" [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Friday, December 08, 2000 2:30 PM
Subject: Re: [Zope] persistence and dictionaries
Chris, this was my original confusion the two places below
where you
say
"You can put instances which do not inherit from
Persistence.Persistent in
your database. They just won't "stick". They'll hang around
until the
server is restarted or for an undetermined amount of time during
normal
operations."
"No. It'll work for "a while" or until the server is restarted.
:-)"
actually do persist after restarts ... that's what confused me,
they
wouldn't
go away and they should!!
regards
Matt
Chris McDonough wrote:
Thanks for the reply, that is really useful. There are a
couple of
things
though that still don't add up. Firstly, you say below, as do
all the
ZODB
documents that "Custom" classes can certainly persist, they
just need
to
mix in
the "Persistence.Persistent" class as a base class. Well, in
my
example I
attached in my first email, my product certainly has
Persistence.Persistent,
but my second class that I add to this one does not, yet it
still
persists.
There was an email sometime ago on the mailing list that told
someone
that
this
was why their product instances disappearing from the ZODB.
(the ref for the original email is :
http://www.egroups.com/message/zope/44263
... I can't find the reply again.)
You can put instances which do not inherit from
Persistence.Persistent
in
your database. They just won't "stick". They'll hang around
until the
server is restarted or for an undetermined amount of time during
normal
operations.
So my current understanding would be that any classes you want
to add
in
do not
need to derive from Persistence.Persistent, and if it is
pickleable
then
all
should be fine if you call on instances of that object within
you
product.
No. It'll work for "a while" or until the server is restarted.
:-)
The next part that worried me came from the "python product
tutorial"
http://www.zope.org/Members/hathawsh/PythonProductTutorial
Re: [Zope] persistence and dictionaries
um, nope, that's all there is to that class I'll attach the whole product
Matt
Jonothan Farr wrote:
Oops. You snipped the contents of MoreStuff.addStuff(). No way to tell what's
going on without that!
--jfarr
- Original Message -
From: "Matt" [EMAIL PROTECTED]
To: "Jonothan Farr" [EMAIL PROTECTED]; [EMAIL PROTECTED]
Sent: Friday, December 08, 2000 2:39 PM
Subject: Re: [Zope] persistence and dictionaries
Ok, here are some of the offending bits out of my boringplus product
which I can send again if people want. I have
another product that keeps a dictionary of cookies so I can see how many
times someone uses the back button to
access the same forms page again. This worked as expected, where I had
to issue an _p_changed=1 to get them to
persist. But I am doing a similar thing below, or so I thought, once
with a dictionary in the product and once in a
dictionary within a class that does not inherit Persistent. This is
what is confusing. Through restarts etc, everything
perisists
class Boring(
OFS.SimpleItem.Item, # A simple Principia object. Not Folderish.
Persistent,# Make us persistent. Yaah!
Acquisition.Implicit, # Uh, whatever.
AccessControl.Role.RoleManager # Security manager.
):
def __init__(self, id, title=''):
self._things = {}
self._more = myClass.MoreStuff()
def manage_edit(self, title, REQUEST=None):
if REQUEST is not None:
if REQUEST.has_key('thing'):
self._things[REQUEST['thing']] = REQUEST['thing_value']
self.addToMore(REQUEST['thing'],REQUEST['thing_value'])
def addToMore(self,name,value):
self._more.addStuff(name,value)
return ""
class MoreStuff:
def __init__(self):
self._stuff = {'animal':'monkey'}
def addStuff(self,stuff_name,stuff_item):
On Sat, 09 Dec 2000, Jonothan Farr wrote:
Maybe I'm mistaken, but it seems like you can put an instance of an
object that
doesn't inherit from Persistent into the ZODB just fine, but its
contents won't
persist, so you'll always end up with a copy of the object as it was
first added
to the database.
--jfarr
- Original Message -
From: "Chris McDonough" [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Friday, December 08, 2000 11:57 AM
Subject: Re: [Zope] persistence and dictionaries
Huh. If they do, it's by chance only. I'd be hard-pressed to
explain it.
Do they inherit from *anything*?
- Original Message -
From: "Matt" [EMAIL PROTECTED]
To: "Chris McDonough" [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Friday, December 08, 2000 2:30 PM
Subject: Re: [Zope] persistence and dictionaries
Chris, this was my original confusion the two places below
where you
say
"You can put instances which do not inherit from
Persistence.Persistent in
your database. They just won't "stick". They'll hang around
until the
server is restarted or for an undetermined amount of time during
normal
operations."
"No. It'll work for "a while" or until the server is restarted.
:-)"
actually do persist after restarts ... that's what confused me,
they
wouldn't
go away and they should!!
regards
Matt
Chris McDonough wrote:
Thanks for the reply, that is really useful. There are a
couple of
things
though that still don't add up. Firstly, you say below, as do
all the
ZODB
documents that "Custom" classes can certainly persist, they
just need
to
mix in
the "Persistence.Persistent" class as a base class. Well, in
my
example I
attached in my first email, my product certainly has
Persistence.Persistent,
but my second class that I add to this one does not, yet it
still
persists.
There was an email sometime ago on the mailing list that told
someone
that
this
was why their product instances disappearing from the ZODB.
(the ref for the original email is :
http://www.egroups.com/message/zope/44263
... I can't find the reply again.)
You can put instances which do not inherit from
Persistence.Persistent
in
your database. They just won't "stick". They'll hang around
until the
server is restarted or for an undetermined amount of time during
normal
operations.
So my current understanding would be that any classes you want
to add
in
do not
need to derive from Persistence.Persistent, and if it is
pickleable
then
all
should be fine if you call on instances of that object with
Re: [Zope] persistence and dictionaries - new light
I tried a few things out it seems the following is happening :
the methods that seemed to be activating persistence when I thought they
shouldn't are called through my handle to the manage_edit ... i.e. in the
product I sent this was anything called within
def manage_edit(self, title, REQUEST=None):
blah
If I call the same methods from another method in the product, and then call
that method from say a dtmlMethod then persotence failed unless I use
_p_changed. This would go along with what Chris mentioned.
So it seems that def manage_edit has a special behaviour, possible linked to
it's binding to
manage_main = HTMLFile('boringEdit', globals()) # Management Interface
which is linked to
manage_options = (
{'label': 'Edit', 'action': 'manage_main'},
hopefully someone has an answer to that.
regards
Matt
Jonothan Farr wrote:
Oops. You snipped the contents of MoreStuff.addStuff(). No way to tell what's
going on without that!
--jfarr
- Original Message -
From: "Matt" [EMAIL PROTECTED]
To: "Jonothan Farr" [EMAIL PROTECTED]; [EMAIL PROTECTED]
Sent: Friday, December 08, 2000 2:39 PM
Subject: Re: [Zope] persistence and dictionaries
Ok, here are some of the offending bits out of my boringplus product
which I can send again if people want. I have
another product that keeps a dictionary of cookies so I can see how many
times someone uses the back button to
access the same forms page again. This worked as expected, where I had
to issue an _p_changed=1 to get them to
persist. But I am doing a similar thing below, or so I thought, once
with a dictionary in the product and once in a
dictionary within a class that does not inherit Persistent. This is
what is confusing. Through restarts etc, everything
perisists
class Boring(
OFS.SimpleItem.Item, # A simple Principia object. Not Folderish.
Persistent,# Make us persistent. Yaah!
Acquisition.Implicit, # Uh, whatever.
AccessControl.Role.RoleManager # Security manager.
):
def __init__(self, id, title=''):
self._things = {}
self._more = myClass.MoreStuff()
def manage_edit(self, title, REQUEST=None):
if REQUEST is not None:
if REQUEST.has_key('thing'):
self._things[REQUEST['thing']] = REQUEST['thing_value']
self.addToMore(REQUEST['thing'],REQUEST['thing_value'])
def addToMore(self,name,value):
self._more.addStuff(name,value)
return ""
class MoreStuff:
def __init__(self):
self._stuff = {'animal':'monkey'}
def addStuff(self,stuff_name,stuff_item):
On Sat, 09 Dec 2000, Jonothan Farr wrote:
Maybe I'm mistaken, but it seems like you can put an instance of an
object that
doesn't inherit from Persistent into the ZODB just fine, but its
contents won't
persist, so you'll always end up with a copy of the object as it was
first added
to the database.
--jfarr
- Original Message -
From: "Chris McDonough" [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Friday, December 08, 2000 11:57 AM
Subject: Re: [Zope] persistence and dictionaries
Huh. If they do, it's by chance only. I'd be hard-pressed to
explain it.
Do they inherit from *anything*?
- Original Message -
From: "Matt" [EMAIL PROTECTED]
To: "Chris McDonough" [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Friday, December 08, 2000 2:30 PM
Subject: Re: [Zope] persistence and dictionaries
Chris, this was my original confusion the two places below
where you
say
"You can put instances which do not inherit from
Persistence.Persistent in
your database. They just won't "stick". They'll hang around
until the
server is restarted or for an undetermined amount of time during
normal
operations."
"No. It'll work for "a while" or until the server is restarted.
:-)"
actually do persist after restarts ... that's what confused me,
they
wouldn't
go away and they should!!
regards
Matt
Chris McDonough wrote:
Thanks for the reply, that is really useful. There are a
couple of
things
though that still don't add up. Firstly, you say below, as do
all the
ZODB
documents that "Custom" classes can certainly persist, they
just need
to
mix in
the "Persistence.Persistent" class as a base class. Well, in
my
example I
attached in my first email, my product certainly has
Persistence.Persistent,
but my second class that I add to this one does not, yet it
still
persists.
There was an email sometime ago on the mailing list that told
someone
that
this
[Zope] persistence and dictionaries
Hi I am trying to get a handle on how I should handle peristence in my python products. I have read all the ZODB documents and all the Product tutorials, which all led me to believe that 1) mutable objects such as lists and dictionaries are not persistent and that updates to these will not be implicitly saved into the ZODB, and 2) that custom classes would certainly not persist. That all seemed fine, and I used persitent mapping and __setstate__ to fill things back in where necessary. But then I decided to demonstrate that persitence does break as suggested. I used boring product, added a dicitonary and a custom class that contains it's own dictionary let the user update name : value pairs for both, and print the contents through index.dtml The problem is that everything persists fine through restarts, everything? Why does it work??? shouldn't it not? I have attached the modified boring product boringplus . it's dead simple to follow if you have made products before. Any explanation would be really nice. regards Matt boringplus.tar.gz
Re: [Zope] persistence and dictionaries
All pickleable Python primitive types (strings, dictionaries, lists, Nones,
integers, floats, longs, etc.) can live in the ZODB. They can persist just
like instances that inherit from the Persistent class.
I think you read a little too much in to the fact that you need to "treat
mutable objects immutably" when working with them in the ZODB. This
statement doesn't mean that these kinds of objects can't be saved in the
ZODB, it just means you need to treat them specially when putting them in
the database.
For instance, if you were doing this inside of an external method:
def amethod(self):
self.mydict = {}
self.mydict['a'] = 1
(where self is the persistent object that is usually the external method's
"container")
It wouldn't work as you expected. Although you'd see an 'a' in mydict for a
little while in further accesses to it, 'mydict' would eventaully show up as
an empty dictionary on the first access of it after it was expired from the
RAM cache (after it was 'ghosted'), because the last thing that the ZODB
"saw" (via the __setattr__ on 'self' and a subsequent transaction) was you
setting a empty dictionary.
Persistent objects (like "self" in the above example) are only smart enough
to notice changes to themselves that happen through their __setattr__ (e.g.
self.mydict = {} calls self's __setattr__). Mutating the attribute 'mydict'
above "in-place" (via self.mydict['a'] = 1) does not trigger self's
__setattr__, so the ZODB never notices that "mydict" got changed.
There are two ways to handle this. The first is to treat mutable attributes
"immutably" via assigning to a temporary variable and then making sure the
persistent container's __setattr__ gets called:
def amethod(self):
dict = {}
dict['a'] = 1
self.mydict = dict # trigger persistence mechanism implicitly
The second is to use the _p_changed attribute of the persistent object on
which the primitive is set. This explcitly tells the persistence system to
include the object on which it's set into the current transaction:
def amethod(self):
self.mydict = {}
self.mydict['a'] = 1
self._p_changed = 1 # trigger persistence mechanism manually
Variations on this theme extend to list methods too (e.g. list.append,
list.pop, etc.)
"Custom" classes can certainly persist, they just need to mix in the
"Persistence.Persistent" class as a base class.
As long as you obey this (slightly annoying, but necessary) rule, you
shouldn't need to use PersistentMapping (it's really just a convenient
wrapper that does this "magic" for you) or make any other wrappers for other
mutable objects.
I don't know why you're using __setstate__, but u.. I won't go there.
:-) I didn't look at your product, but I don't think I need to to answer
your question...
Hi I am trying to get a handle on how I should handle peristence in my
python products. I have read all the ZODB documents and all the Product
tutorials, which all led me to believe that 1) mutable objects such as
lists and dictionaries are not persistent and that updates to these will
not be implicitly saved into the ZODB, and 2) that custom classes would
certainly not persist. That all seemed fine, and I used persitent
mapping and __setstate__ to fill things back in where necessary. But
then I decided to demonstrate that persitence does break as suggested.
I used boring product, added a dicitonary and a custom class that
contains it's own dictionary let the user update name : value pairs
for both, and print the contents through index.dtml
The problem is that everything persists fine through restarts,
everything?
Why does it work??? shouldn't it not?
I have attached the modified boring product boringplus . it's
dead simple to follow if you have made products before.
Any explanation would be really nice.
regards
Matt
___
Zope maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] persistence and dictionaries
Thanks for the reply, that is really useful. There are a couple of things
though that still don't add up. Firstly, you say below, as do all the ZODB
documents that "Custom" classes can certainly persist, they just need to mix in
the "Persistence.Persistent" class as a base class. Well, in my example I
attached in my first email, my product certainly has Persistence.Persistent,
but my second class that I add to this one does not, yet it still persists.
There was an email sometime ago on the mailing list that told someone that this
was why their product instances disappearing from the ZODB.
(the ref for the original email is : http://www.egroups.com/message/zope/44263
... I can't find the reply again.)
So my current understanding would be that any classes you want to add in do not
need to derive from Persistence.Persistent, and if it is pickleable then all
should be fine if you call on instances of that object within you product.
The next part that worried me came from the "python product tutorial"
http://www.zope.org/Members/hathawsh/PythonProductTutorial
This stated that the class dictionary self.votes = {} needed to be changed to
self._votes = Globals.PersistentMapping() so that updates to it persist.
Hence my query about dictionaries.
I also noticed your comment about __setstate__ . What is it about this that is
dangerous. Recently I built a product out of some python classes I wrapped
around 4DOM, and since 4DOM documents do not seem to persist(well the document
does, but it loses all its children), then I persisted them to the local file
system, since I needed to do that anyway for what I was doing. Setstate seemed
to work nicely to bring them back, though watching its behaviour I noticed that
it was called very often by zope.
regards
Matt
Chris McDonough wrote:
All pickleable Python primitive types (strings, dictionaries, lists, Nones,
integers, floats, longs, etc.) can live in the ZODB. They can persist just
like instances that inherit from the Persistent class.
I think you read a little too much in to the fact that you need to "treat
mutable objects immutably" when working with them in the ZODB. This
statement doesn't mean that these kinds of objects can't be saved in the
ZODB, it just means you need to treat them specially when putting them in
the database.
For instance, if you were doing this inside of an external method:
def amethod(self):
self.mydict = {}
self.mydict['a'] = 1
(where self is the persistent object that is usually the external method's
"container")
It wouldn't work as you expected. Although you'd see an 'a' in mydict for a
little while in further accesses to it, 'mydict' would eventaully show up as
an empty dictionary on the first access of it after it was expired from the
RAM cache (after it was 'ghosted'), because the last thing that the ZODB
"saw" (via the __setattr__ on 'self' and a subsequent transaction) was you
setting a empty dictionary.
Persistent objects (like "self" in the above example) are only smart enough
to notice changes to themselves that happen through their __setattr__ (e.g.
self.mydict = {} calls self's __setattr__). Mutating the attribute 'mydict'
above "in-place" (via self.mydict['a'] = 1) does not trigger self's
__setattr__, so the ZODB never notices that "mydict" got changed.
There are two ways to handle this. The first is to treat mutable attributes
"immutably" via assigning to a temporary variable and then making sure the
persistent container's __setattr__ gets called:
def amethod(self):
dict = {}
dict['a'] = 1
self.mydict = dict # trigger persistence mechanism implicitly
The second is to use the _p_changed attribute of the persistent object on
which the primitive is set. This explcitly tells the persistence system to
include the object on which it's set into the current transaction:
def amethod(self):
self.mydict = {}
self.mydict['a'] = 1
self._p_changed = 1 # trigger persistence mechanism manually
Variations on this theme extend to list methods too (e.g. list.append,
list.pop, etc.)
"Custom" classes can certainly persist, they just need to mix in the
"Persistence.Persistent" class as a base class.
As long as you obey this (slightly annoying, but necessary) rule, you
shouldn't need to use PersistentMapping (it's really just a convenient
wrapper that does this "magic" for you) or make any other wrappers for other
mutable objects.
I don't know why you're using __setstate__, but u.. I won't go there.
:-) I didn't look at your product, but I don't think I need to to answer
your question...
Hi I am trying to get a handle on how I should handle peristence in my
python products. I have read all the ZODB documents and all the Product
tutorials, which all led me to believe that 1) mutable objects such as
lists and dictionaries are not persistent and that updates to these will
not be implicitly saved into the ZODB, and 2) that custom
