Re: [Zope-dev] how bad are per-request-write-transactions
Paul Everitt wrote: > Let's say we had a queue in Zope. We could asynchronously send changes > into the queue. Later, based on some policy (e.g. idle time, clock > ticks, etc.), those changes would be enacted/committed. > > Imagine the queue itself is in a different storage, likely > non-versioned. Imagine that the queue is processed every N seconds. It > takes all the work to do and performs it, but in a subtransaction. > > Thus you might send the queue ten increments to a counter, but only one > will be committed to the main storage. > > To make programmers have to think less about the queue (send in the > object reference, the method to use, and the parameters), you could make > it look like a special form of subtransactions. That is, you say: > > tm.beginQueuingTransactions() > self.incrementCounter() > self.title='Simple change' > self.body = upload_file > tm.endQueueingTransactions() > > At the transaction level, all enclosed changes are queued for later > commit. You don't have to think any differently than rather object > state management. Wow, on the surface, that would be very easy to do. Transaction.register() might dump to a long-lived queue instead of the single-transaction queue. > This pattern applies better when you have a lot of document cataloging > to be done. A separate process can wake up, make a ZEO connection, and > process the queue. I don't think that indexing documents *has* to be a > transactional part of every document save. Right. Here's another way to think about it: we could use a catalog lookalike which, instead of updating indexes directly, asks a special ZEO client to perform the reindexing. The special client might decide to batch updates. > Under this cron-style approach, you also pay less of a conflict-error > penalty, as you can increase the backoff period. There's no web browser > on the other end, impatiently waiting for their flaming logo. :^) A variant on your idea is that when the transaction is finishing, if there are any regular objects to commit, the long-lived queue gets committed too. That would be beneficial for counters, logs, and objects like Python Scripts which have to cache the compiled code in ZODB, but not as beneficial for catalogs. Ok, thinking further... how about a Zope object called a "peer delegate" which can act like other Zope objects, but which actually calls out to another ZEO client to do the work? It could be very interesting... it might use some standard RPC or RMI mechanism. We would want to be careful to make it simple. > Ahh well, fun to talk about. Maybe this time next year we can repeat > the conversation. :^) I hope we'll be talking about what we did instead of what we'll do. :-) The change to transactions seems simple. Another thought: the long-lived queue might be committed only when there are regular objects to commit *and* a certain amount of time has passed since the last commit of the long-lived queue. That might work well for catalogs. Cool! Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Need to instantiate zclass in location other thancurrent
On 4/17/02 4:49 PM, "Max Slimmer" <[EMAIL PROTECTED]> wrote:
> I have created a zclass and want to create a new instance of this class and
> have it be child of some other know object in the tree.
> Given that we know the path (url) to the new prospective parent how do we do
> this.
I do this a lot, with my own custom add in screens that add objects in
special locations. I have it working like this in a python script::
REQUEST = container.REQUEST
target = context.content.documents # folder to put the document in
### Create a new instance.
new_item = target.manage_addProduct[
'MyProduct'].MyZClass.createInObjectManager(new_id, REQUEST)
## other operations on the new item
new_item.setTitle('Rockabilly Bob')
...
> A second question. I need to manage lists of these zobjects within other
> instances of the same, is there a reason to store the url vs the object(s).
> I have has some problems getting back a string repr of the object " of .."
> instead of the object? It would probably be better for me (the code) to
> store objects thus not need to get the object each time I want to call a
> method...
You'll want to store the path to the object. Making other 'hard' references
to a persistent object in Zope is still a scary proposition.
--
Jeffrey P Shell
www.cuemedia.com
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Beyond Acquisition
In article <[EMAIL PROTECTED]>,
Toby Dickenson <[EMAIL PROTECTED]> wrote:
>On Fri, 19 Apr 2002 15:28:52 + (UTC), [EMAIL PROTECTED] ()
>wrote:
>
>>It seems there is no way to get the real 'b' attribute from a, i.e. the
>>B instance set at the line "a.b = B()"
>
>Thats exactly right. your __of__ method means that *any* time you try
>to take a B object out of an A, you get a C object instead of a B.
>
>try a.__dict__['b']
That's not a good solution since the b attribute can be defined in the
classes of a...
With some tries, I think I got the solution:
_nodefault = []
def getrealattr(obj, name, default = _nodefault):
try:
return obj.__dict__[name]
except KeyError: pass
objclass = obj.__class__
try:
attr = getattr(objclass, name)
if hasattr(attr, '__of__'):
# a trick from extension class:
# whenever you get a class attribute,
# you will get a Python Method
# im_func get the function if it is a
# method and the real object if not
return attr.im_func
return attr
except AttributeError:
if default is _nodefault: raise
return default
if __name__ == '__main__':
# Tests
from Acquisition import Implicit
class C(Implicit):
def __init__(self, name):
self.name = name
class B(C):
def __of__(self, parent):
return C(self.name + ' (in object %s)' % parent.name).__of__(parent)
class A(C):
b = B('b in the class')
a = C('a')
b = B('b')
a.b = b
print 'a.b = %s' % a.b.name
print 'real a.b = %s' % getrealattr(a, 'b').name
a = A('a')
print 'a.b = %s' % a.b.name
print 'real a.b = %s' % getrealattr(a, 'b').name
print getrealattr(a, 'c', None)
--
Julien Jalon
http://www.nuxeo.com/
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Beyond Acquisition
On Fri, 19 Apr 2002 15:28:52 + (UTC), [EMAIL PROTECTED] () wrote: >It seems there is no way to get the real 'b' attribute from a, i.e. the >B instance set at the line "a.b = B()" Thats exactly right. your __of__ method means that *any* time you try to take a B object out of an A, you get a C object instead of a B. try a.__dict__['b'] Toby Dickenson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Beyond Acquisition
Here is a simple code: from Acquisition import Implicit, aq_base class A(Implicit): pass class C(Implicit): pass class B(Implicit): def __of__(self, parent): # controls the wrapper used for acquisition c = C() c._parent = parent return c.__of__(parent) a = A() a.b = B() print a.b # returns print aq_base(a.b) # returns the same It seems there is no way to get the real 'b' attribute from a, i.e. the B instance set at the line "a.b = B()" The problem is that I want to do something like this: aa = A() cc = aq_realget(a, 'b').__of__(aa) where aq_realget would return the B instance without to try to put it in the context of a. some idea? thanks in advance. -- Julien Jalon http://www.nuxeo.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] how bad are per-request-write-transactions
On Fri, 19 Apr 2002 08:18:47 -0400, Chris McDonough <[EMAIL PROTECTED]> wrote: >> Storing counter objects *only* in a non-undo storage would be more >> pleasant if ZODB supported cross-storage object references. > >Yup. I don't think this is anywhere on the radar, though... H. cross-storage 'symbolic links' would help too. I think we could implement that using the same trickery as mounted storages. Toby Dickenson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] ForkedStorage ;-)
Paul Everitt wrote: > > ForkedStorage, I like it simply for the coolness of the name. :^) And so easily mispronounced, especially when it's not working :-) *grinz* Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] how bad are per-request-write-transactions
Chris McDonough wrote: > > > Storing counter objects *only* in a non-undo storage would be more > > pleasant if ZODB supported cross-storage object references. > > Yup. I don't think this is anywhere on the radar, though... How hard would they be to add? cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] how bad are per-request-write-transactions
> A dual mode storage, or simply dual storages? The former as a long-term goal, the latter as a short-term goal. The proposal I mentioned would make it easier to build tools that allow you to mount storages. > Storing counter objects *only* in a non-undo storage would be more > pleasant if ZODB supported cross-storage object references. Yup. I don't think this is anywhere on the radar, though... -- Chris McDonoughZope Corporation http://www.zope.org http://www.zope.com "Killing hundreds of birds with thousands of stones" ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] how bad are per-request-write-transactions
On Wed, 17 Apr 2002 23:01:04 -0400, "Chris McDonough" <[EMAIL PROTECTED]> wrote: >It would be best to make make a dual-mode undoing and nonundoing storage on >a per-object basis. But a half step would be to make it easier to use >mounted storages ala >http://dev.zope.org/Wikis/DevSite/Proposals/StorageAndConnectionTypeRegistri >es. A dual mode storage, or simply dual storages? Storing counter objects *only* in a non-undo storage would be more pleasant if ZODB supported cross-storage object references. Toby Dickenson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] how bad are per-request-write-transactions
ForkedStorage, I like it simply for the coolness of the name. :^) But it sparked a different kind of idea, leveraging a pattern that might emerge in Zope 3. Let's say we had a queue in Zope. We could asynchronously send changes into the queue. Later, based on some policy (e.g. idle time, clock ticks, etc.), those changes would be enacted/committed. Imagine the queue itself is in a different storage, likely non-versioned. Imagine that the queue is processed every N seconds. It takes all the work to do and performs it, but in a subtransaction. Thus you might send the queue ten increments to a counter, but only one will be committed to the main storage. To make programmers have to think less about the queue (send in the object reference, the method to use, and the parameters), you could make it look like a special form of subtransactions. That is, you say: tm.beginQueuingTransactions() self.incrementCounter() self.title='Simple change' self.body = upload_file tm.endQueueingTransactions() At the transaction level, all enclosed changes are queued for later commit. You don't have to think any differently than rather object state management. This pattern applies better when you have a lot of document cataloging to be done. A separate process can wake up, make a ZEO connection, and process the queue. I don't think that indexing documents *has* to be a transactional part of every document save. Under this cron-style approach, you also pay less of a conflict-error penalty, as you can increase the backoff period. There's no web browser on the other end, impatiently waiting for their flaming logo. :^) Ahh well, fun to talk about. Maybe this time next year we can repeat the conversation. :^) --Paul Shane Hathaway wrote: > Jeremy Hylton wrote: > >>> "CM" == Chris McDonough <[EMAIL PROTECTED]> writes: >> >> >> >> >> Completely agreed. My disagreement is portraying the counter >> >> problem as impossible with the zodb. I think some people, as >> >> evidenced by some of the responses, are willing to live with the >> >> tradeoffs. Other people will find managing a log file on disk to >> >> be a more manageable solution. >> >> CM> It would be best to make make a dual-mode undoing and nonundoing >> CM> storage on a per-object basis. >> >> I'd really like to do this for ZODB4, but it seems hard to get it into >> FileStorage, without adding automatic incremental packing to >> FileStorage. >> >> Example: Object A is marked as save enough revisions to do a single >> undo. When a transaction updates A and makes older revisions >> unnecessary, there's no obvious way to remove them without doing a >> pack. We could write a garbage collector that removed unneeded things >> (as opposed to packing everything to a particular date), but it >> doesn't seem very useful if it needs to be run manually. > > > One idea I've been floating in my head is the idea of a "forked" > storage, where some objects are stored in an undoable storage and others > are stored in a non-undoable storage. I could try to explain it in > English but pseudocode is easier: > > > class ForkedStorage: > > def __init__(self, undoable_storage, non_undoable_storage): > self.undoable = undoable_storage > self.non_undoable = non_undoable_storage > > def store(self, oid, data, serial): > if not serial or serial == '\0' * 8: > # For new objects, choose a storage. > want_undo = self.wantUndoableStorage(data) > if want_undo: > storage = self.undoable > else: > storage = self.non_undoable > else: > # For existing objects, use the storage chosen previously. > if self.undoable.load(oid): > storage = self.undoable > else: > storage = self.non_undoable > storage.store(oid, data, serial) > > def load(self, oid): > data, serial = self.undoable.load(oid) > if not data: > data, serial = self.non_undoable.load(oid) > if not data: > raise POSException, 'data not found' > return data, serial > > def wantUndoableStorage(self, data): > u = cpickle.Unpickler() > module, name = u.loads(data) > class_ = getattr(__import__(module), name) > if getattr(class_, '_p_undoable', 1): > return 1 > else: > return 0 > > > Only a simple idea. :-) > >> Also, how would you specifiy the object's packing policy? I'm >> thinking an _p_revision_control attribute or something like that. If >> the attribute exists on an object, it sets a particular policy for >> that object. > > > > Do individual transactions need to play in this game, too? I'm > >> imagining a use case where an object is marked as "no revisions" but >> you want to be able to undo a particular transaction. I'm n
[Zope-dev] (no subject)
confirm 962940 ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] _v_ and ZEO
On Thu, 18 Apr 2002 17:35:17 + (UTC), Florent Guillaume <[EMAIL PROTECTED]> wrote: >I'll investigate clearing the _v_ caches at the end of the transaction, >using the REQUEST._hold hack mentionned earlier. Below is the class I use for this. Just call attribute_cleaner(self,'_v_my_attribute') before assigning to _v_my_attribute, and it will be cleared at the end of the ZODB transaction. I wonder if this is generally useful enough to go in the ZODB distribution? class attribute_cleaner: def __init__(self,client,attr): self.client = client self.attr = attr get_transaction().register(self) def ClearCache(self,*args): try: delattr(self.client, self.attr) except AttributeError: pass except KeyError: pass tpc_finish = tpc_abort = abort = abort_sub = ClearCache def tpc_begin(self,transaction,subtransaction=None): pass def commit(self,object,transaction): pass def tpc_vote(self,transaction): pass def commit_sub(self,transaction):pass Toby Dickenson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope 2.3.3 with Python 1.5.2 asyncore/asynchat
Hi, I have to fix the asyncore/asynchat Broken Pipe bug in the Python 1.5.2 version. Can anybody tell me how to do that ? I know that the asyncore/asynchat in Zope 2.4.4 Zserver/medusa has this fixed. Thanks, Dirk BEGIN:VCARD VERSION:2.1 N:Datzert;Dirk FN:Dirk Datzert ORG:;Andernach TEL;WORK;VOICE:+49 2631 81 4595 TEL;WORK;FAX:+49 2631 81 15 4595 ADR;WORK:;;Andernach LABEL;WORK:Andernach KEY;X509;ENCODING=BASE64: MIIDVTCCAr6gAwIBAgIPAM2aAkJs9V5JP7ynMA0GCSqGSIb3DQEBBAUAMIG8MQswCQYD VQQGEwJERTEQMA4GA1UECBMHSGFtYnVyZzEQMA4GA1UEBxMHSGFtYnVyZzE6MDgGA1UEChMx VEMgVHJ1c3RDZW50ZXIgZm9yIFNlY3VyaXR5IGluIERhdGEgTmV0d29ya3MgR21iSDEiMCAG A1UECxMZVEMgVHJ1c3RDZW50ZXIgQ2xhc3MgMSBDQTEpMCcGCSqGSIb3DQEJARYaY2VydGlm aWNhdGVAdHJ1c3RjZW50ZXIuZGUwHhcNMDExMTEyMTIwMTM1WhcNMDMwMTAyMTIwMTM1WjBp MQswCQYDVQQGEwJERTEQMA4GA1UEBxMHTmV1d2llZDEVMBMGA1UEAxMMRGlyayBEYXR6ZXJ0 MTEwLwYJKoZIhvcNAQkBFiJEaXJrLkRhdHplcnRAcmFzc2Vsc3RlaW4taG9lc2NoLmRlMIGf MA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCog/5TyCRdMjACIM/U23qq0n0LUEeb+X3WpIn5 2UHosqQMDdLDEtlUOMSKTrqcsp3lHTOQWwj65/T3cI85brb7PkJDtbCI57MVTtMIKByd946g uIOe6xti0H+lJAscIO1a6FqFjjJLhudWpDqaWgW95qGaFVT1me2xPEaoiampxQIDAQABo4Gq MIGnMDMGCWCGSAGG+EIBCAQmFiRodHRwOi8vd3d3LnRydXN0Y2VudGVyLmRlL2d1aWRlbGlu ZXMwEQYJYIZIAYb4QgEBBAQDAgWgMF0GCWCGSAGG+EIBAwRQFk5odHRwczovL3d3dy50cnVz dGNlbnRlci5kZS9jZ2ktYmluL2NoZWNrLXJldi5jZ2kvQ0Q5QTAwMDAwMDAyNDI2Q0Y1NUU0 OTNGQkNBNz8wDQYJKoZIhvcNAQEEBQADgYEAOHB5sq4AdTAwizcmGPgdnfPo9btBu1d/PRNO D/xoIhXlI4A0ehRZCBGuOup0L78YBj+DZUMDOK3H6+5M8rOfCazkcYlcfdxZ9LeNRkMy+RRx PbmXelKB7UjM6IfJyZiQzm/jTwxFle3L+W8TMSPwmAdTJOo0YkikHCAlxNLMn4y= EMAIL;PREF;INTERNET:[EMAIL PROTECTED] REV:20020419T090523Z END:VCARD smime.p7s Description: application/pkcs7-signature
