Re: [ZODB-Dev] getExtensionMethods

2006-05-15 Thread Toby Dickenson
On Sunday 14 May 2006 21:52, Jim Fulton wrote:
 
 Toby, this is almost certainly a question for you. :)
 
 Can someone explain to me why getExtensionMethods is needed in the
 storage API?  It's not used by anything in ZODB. 

ZEO proxies the standard storage API.

The original requirement is for ZEO to support storages which expose extended 
functionality via an extended API. DirectoryStorage uses this to allow 
control of its snapshot mode.

 I'm incluned to remove it,

It would be mildly disappointing if there was no way of accessing an extended 
storage API over ZEO. However I have no concerns about seeing 
getExtensionMethods replaced with some alternative mechanism (possibly more 
complex for the consumer(s) of this extended API, if that makes it more 
sustainable within ZEO)



References:
First proposed here...
http://mail.zope.org/pipermail/zodb-dev/2002-May/002704.html
and an implementation discussed in the thread starting here...
http://mail.zope.org/pipermail/zodb-dev/2002-September/003209.html



-- 
Toby Dickenson
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] DirectoryStorage 1.1.17

2006-01-04 Thread Toby Dickenson
I am pleased to announce the release of DirectoryStorage, version 
1.1.17. 

Over the last few months DirectoryStorage has been languishing at 'release 
candidate' status, so this is the first release badged 'stable' to include 
Zope 3 support (thanks to Philipp von Weitershausen), NTFS support (thanks to 
Enfold Systems and Mark Hammond), and numerous small improvements to 
replication and backup tools.

http://dirstorage.sourceforge.net/

What is DirectoryStorage? An alternative storage for ZODB that uses ordinary 
files and directories to store revisions of ZODB objects; one file per 
revision per object. Designed to be robust, manageable, fault tolerant, and 
scalable.



___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] DirectoryStorage 1.1.16 (release candidate)

2005-11-14 Thread Toby Dickenson
I am pleased to announce the release candidate for DirectoryStorage version 
1.1.16. This version includes support for Zope 2.8 and Zope 3.

http://dirstorage.sourceforge.net/

What is DirectoryStorage? An alternative storage for ZODB that uses ordinary 
files and directories to store revisions of ZODB objects; one file per 
revision per object. 

Many thanks to Philipp von Weitershausen for the Zope 3 support.

This is a release candidate, aiming for final release before the end of the 
month. works-for-me emails to [EMAIL PROTECTED] are 
appreciated.
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] [Enhancement] Common cache for mounted connections

2005-09-30 Thread Toby Dickenson
On Friday 30 September 2005 11:40, Dieter Maurer wrote:
 We have the need to mount dozens, if not hundreds of ZODBs in a single
 Zope instance.
 
 With the current implementation, each ZODB connection has its own
 independent cache. This makes it very difficult to use the
 available memory efficiently for caching. As the caches are independent,
 a heavily used ZODB cannot use memory assigned to a lightly used ZODB.

Each ZODB version also has its own cache. There is already a mechanism for 
ensuring that lightly used versions do not hold on to memory indefinitely. 
That should be much easier, and might just be adequate.

One big disadvantage is that it does not enforce a strict upper limit on 
cache/memory usage, so memory usage will grow if all storages are frequently 
accessed.

-- 
Toby Dickenson
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB memory problems

2005-05-31 Thread Toby Dickenson
On Tuesday 31 May 2005 18:43, Shane Hathaway wrote:
 Jeremy Hylton wrote:
  It's really too bad that ZEO only allows a single outstanding request.
   Restructuring the protocol to allow multiple simulatenous requests
  was on the task list years ago, but the protocol implementation is so
  complex I doubt it will get done :-(.  I can't help but think building
  on top of an existing message/RPC layer would be profitable.  (What's
  twisted's RPC layer?

pb; perspective broker. I thought that suggestion was crazy when Jeremey first 
presented it in his blog, but increasing exposure to other twisted tools 
makes it seem more reasonable.

In short, Ive not yet been disappointed by anything coming from the twisted 
camp.

  )  Or at least something less difficult to use 
  than asyncore.
 
 Do you think the RPC layer is the source of the problem?  I feel like
 the way ZODB uses references in pickles is the main thing that slows it
 down.  Even if you have a protocol that can request many objects at
 once, the unpickling machinery only asks for one at a time.

Both problems seem interestingly non-trivial; a storage layer that will 
prefetch related pickles, and an rpc/storage layer that can express those 
prefetching requests with appropriate security, concurrency, and 
prioritization.


-- 
Toby Dickenson
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Hanging ZEO-client hangs all other ZEO-clients?

2005-04-14 Thread Toby Dickenson
On Thursday 14 April 2005 08:38, Dario Lopez-Kästen wrote:
 Chris Withers wrote:
  You sure you're using the latest 
  DirectoryStorage? Can you reproduce this using just plain FileStorage?
 
 No, I am not using the latest, because of my *n*l sysadmin (actually 
 there are two of them, but only one whines :-). I'll try beating him in 
 the head a few times, er, I mean, discuss the issue with him to make 
 him change his mind.

If it helps, Im pretty sure that no deadlocking bugs have been fixed in 
directorystorage since version 1.1.4.

 5) POSKeyErrors. We where getting quite a few of those, and that
 scared me. with DirStor, I do not see them as much as before.

I would be interested to see if you are getting *any* unexpected POSKeyErrors 
with DirectoryStorage.

-- 
Toby Dickenson
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev