Re: [ZODB-Dev] transaction timeouts raise no exceptions and cause strange errors on clients

2010-11-04 Thread Jim Fulton
;t really know whether the transaction committed or not. It's much worse if multiple storages are involved. > I restarted the >   client to be on the safe side, but it'd be good to know for sure. Restarting wouldn't necessarily make any differen

Re: [ZODB-Dev] transaction timeouts raise no exceptions and cause strange errors on clients

2010-11-04 Thread Jim Fulton
On Thu, Nov 4, 2010 at 10:31 AM, Chris Withers wrote: > On 04/11/2010 13:43, Jim Fulton wrote: >>> I'm curious: >>> >>> - Why is the timeout not logged at ERROR or even CRITICAL? >> >> Because it was considered to be neither.  It's also not

Re: [ZODB-Dev] transaction timeouts raise no exceptions and cause strange errors on clients

2010-11-04 Thread Jim Fulton
On Thu, Nov 4, 2010 at 11:03 AM, Chris Withers wrote: > On 04/11/2010 14:50, Jim Fulton wrote: >> >> Actually, I think it should be used, at least for applications with >> high availability requirements. For applications like our's (ZC's) >> it amounts to a

Re: [ZODB-Dev] Sharing (persisted) strings between threads

2010-12-08 Thread Jim Fulton
tainly thoughtr about it a lot. In practice, I doubt the benefit will be worth the extra overhead (let alond the effort :). Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] Sharing (persisted) strings between threads

2010-12-08 Thread Jim Fulton
On Wed, Dec 8, 2010 at 7:45 AM, Malthe Borch wrote: > On 8 December 2010 13:28, Jim Fulton wrote: >>> With 20 active threads, each having rendered the Plone 4 front page, >>> this approach reduced the memory usage with 70 MB. >> >> Out of a total of what? > &

Re: [ZODB-Dev] When will the Data.fs.index will be touched

2010-12-09 Thread Jim Fulton
use the zdaemon backoff-limit option to configure zdaemon to wait much longer before doing a hard kill. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] ZODB packing and possible side effects?

2010-12-09 Thread Jim Fulton
kup processes, I don't use repozo myself, but I believe it detects packs and makes a full backup after a pack. > a pack removing weakly referenced objects, I'm not sure what you're referring to or if you have a question. > impact on the Data.fs index, ... A new index file i

Re: [ZODB-Dev] Some tests do not pass load's version parameter

2010-12-13 Thread Jim Fulton
I tend to think > interface should be fixed). We should probably change the interface. I didn't want to do that because that would require changes to existing storages. For better or worse, I made another required storage change lately to add the checkCurrentSerialInTransaction method, and I sho

Re: [ZODB-Dev] [RFC] Allow serial to be returned as late as tpc_finish.

2010-12-16 Thread Jim Fulton
t or somethink like it will go into that release. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] Lazy Instantiation

2011-01-22 Thread Jim Fulton
s query should not >> trigger the instantiation. I assume you're talking about activation (instantiation isn't a good word here) of the child. Simply getting the child won't activate it, but almost any operation on it will. a.child is None won't cause the child to

Re: [ZODB-Dev] Lazy Instantiation

2011-01-22 Thread Jim Fulton
t. ... > Perhaps too much to ask: can I observe the state of the object (without > changing it)? For example indirectly by logging data base access and > noting that attribute access does cause data base access for a ghost. I don't understand the question. What problem are you trying

Re: [ZODB-Dev] Increasing MAX_BUCKET_SIZE for IISet, etc

2011-01-26 Thread Jim Fulton
for someone who's willing to grok the Python C APIs. Changing the default sizes for the II ad LL BTrees is pretty straightforward. We were more interested in LO (and similar) BTrees. For those, it's much harder to guess sizes because you don't know generally how big the objects will b

Re: [ZODB-Dev] RelStorage, history-free, pack causes POSKeyError with BTreeFolder2

2011-01-31 Thread Jim Fulton
ode above is broken. Note that the storage could validate the reference to x when self is saved, but no storage that I know of does, for performance reasons. In some future version of ZODB, it might be nice to change the database record structure to make this sort of check easier. Jim -- Jim Fult

Re: [ZODB-Dev] Issue with Blobstorage for a third client in Plone

2011-02-11 Thread Jim Fulton
Mac, Windows > ## http://LibreOffice.org > ## Support the Document Foundation (http://documentfoundation.org) > ## Meine Seite: http://www.amantke.de > ___ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.or

[ZODB-Dev] ZODB 3.10.2 released

2011-02-12 Thread Jim Fulton
I've released ZODB 3.10.2. http://pypi.python.org/pypi/ZODB3/3.10.2#bugs-fixed This release primarily removes an optimization that caused severe problems for some applications. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For

Re: [ZODB-Dev] Large ZODB packing and progress

2011-02-25 Thread Jim Fulton
operations. I don't know about relstorage, but for FileStorage, commit locks are only held in the very last stage of packing after the vast majority of the work is done. Even then, the commit lock is released periodically to let new transactions proceed. Jim --

Re: [ZODB-Dev] Wrong blob returned in one of the zeo clients

2011-03-01 Thread Jim Fulton
stores multiple blobs, and the OIDs of some of those > blobs differ by a multiple of 997, then cache filenames will collide. I'm at a loss as to why you would think this. The the blob file names use the full oid. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton _

Re: [ZODB-Dev] Wrong blob returned in one of the zeo clients

2011-03-02 Thread Jim Fulton
blob directories and the non-shared blob implementation suffered from neglect, which I at least tried to rectify in ZODB 3.9. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/

Re: [ZODB-Dev] Feedback on the packing process

2011-03-04 Thread Jim Fulton
FileStorage It writes progress information to it's own logs. Not sure how easy it is to figure out. > This is because packing usually takes very long time and it might hard > to observe when the process might complete. I agree that this would be a useful feature. I suggest making a b

Re: [ZODB-Dev] packing ZODB

2011-03-31 Thread Jim Fulton
as there are records remaining, but not all of the records stay around. I'm not sure what you meant above, but if you think all of the records in the transaction stay around, the answers is no. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton __

Re: [ZODB-Dev] savepoint not doing cacheGC?

2011-03-31 Thread Jim Fulton
emory (as they're saved in temporary storage). If you aren't writing data, savepoints don't buy you anything and you should simply call cacheGC yourself periodically. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more inf

[ZODB-Dev] ZODB 3.10.3 released

2011-04-12 Thread Jim Fulton
cache miss rate by 25-50%, depending on the cache size. It also contains some small optimizations to the storage server code. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zop

Re: [ZODB-Dev] Backports for ZODB 3.9

2011-04-18 Thread Jim Fulton
e these bugfixes in Plone 4.0.6. I plan to release this today. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.

Re: [ZODB-Dev] Backports for ZODB 3.9

2011-04-18 Thread Jim Fulton
On Mon, Apr 18, 2011 at 11:29 AM, David Glick wrote: > > > On Apr 18, 2011, at 8:16 AM, "Jim Fulton" wrote: > >> On Fri, Apr 15, 2011 at 2:16 PM, Eric Steele wrote: >>> Jim (et al), >>> >>> Could I trouble you for backports of >>>

Re: [ZODB-Dev] Speeding up ZODB (was "redis cache for RelStorage")

2011-05-05 Thread Jim Fulton
ments that can be done at the OS or even application > layer that might improve performance? Look at how your application is using data. If you have requests that have to load a lot of data, maybe you can refactor your application to load fewer. Jim -- Jim Fulton http://www.linkedin.com/in/jimfult

Re: [ZODB-Dev] Speeding up ZODB (was "redis cache for RelStorage")

2011-05-06 Thread Jim Fulton
ng data. If you have requests that >> have to load a lot of data, maybe you can refactor your application to >> load fewer. > > A quick check with nethogs shows values of network usage oscillating > between 100 and 200 KB/s. But I guess that if I were loading an > excessive amount o

Re: [ZODB-Dev] How to check for setting the same values on persistent objects?

2011-05-06 Thread Jim Fulton
lasses of persistent which overwrite > various hooks. > > Maybe option one is the easiest here, but it would need some > documentation about this being a best practice. Until now I didn't > realize the implications of setting attributes to unchanged values. I think the best app

Re: [ZODB-Dev] Speeding up ZODB (was "redis cache for RelStorage")

2011-05-06 Thread Jim Fulton
ur ZEO cache and store the ZEO cache on either a RAM disk (try mount > -t tmpfs none /some/path) or a solid state disk.  Remember that seek > time is 5-10 ms with spinning drives, so putting a ZEO cache on a > spinning drive can actually kill performance. If this on Linux and you have enoug

Re: [ZODB-Dev] Speeding up ZODB (was "redis cache for RelStorage")

2011-05-06 Thread Jim Fulton
On Fri, May 6, 2011 at 4:21 PM, Shane Hathaway wrote: > On 05/06/2011 02:14 PM, Jim Fulton wrote: >>> >>> It sounds like you primarily need a bigger and faster cache.  If you >>> want to make minimal changes to your setup, try increasing the size of >>> yo

Re: [ZODB-Dev] Immutable blobs?

2011-05-09 Thread Jim Fulton
ant to implement a restriction like this, do it in a wrapper class. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] I would like append an indexer with uuid key at root of my ZODB, how can I append an function to commit event to update this uuid catalog ?

2011-06-28 Thread Jim Fulton
ough they typically use BTrees. For example, indexes typically use forward and reverse mappings to facilitate inindexing without having to store old document values. - Events to signal when objects should be added, updated, or removed from the index(es). Jim -- Jim Fulton http://www.linkedin.c

[ZODB-Dev] RFC: Blobs in S3

2011-07-06 Thread Jim Fulton
tate would be set on the storage server. - The win depends on being able to load a blob file independently of loading blob objects, although the ZEO blob cache implementation already depends on this. -- Jim Fulton http://www.linkedin.com/in/jimfulton _

Re: [ZODB-Dev] RFC: Blobs in S3

2011-07-07 Thread Jim Fulton
G, I sent this before I was done. Had some sort of gmail missfire, I thought the email was lost. :/ On Wed, Jul 6, 2011 at 2:44 PM, Jim Fulton wrote: > We're evaluating AWS for some of our applications and I'm thinking of adding > some options to support using S3 to st

Re: [ZODB-Dev] RFC: Blobs in S3

2011-07-07 Thread Jim Fulton
On Thu, Jul 7, 2011 at 9:42 AM, Alan Runyan wrote: > On Wed, Jul 6, 2011 at 1:44 PM, Jim Fulton wrote: >> We're evaluating AWS for some of our applications and I'm thinking of adding >> some options to support using S3 to store Blobs: > > Mind clarifying, is the a

Re: [ZODB-Dev] RFC: Blobs in S3

2011-07-07 Thread Jim Fulton
On Thu, Jul 7, 2011 at 9:13 AM, Laurence Rowe wrote: > On 6 July 2011 19:44, Jim Fulton wrote: ... > Adding the ability to store blobs in S3 would be an excellent feature > for AWS based deployments. I'm not convinced that presenting S3 urls > to the end users is terribly usef

Re: [ZODB-Dev] RFC: Blobs in S3

2011-07-07 Thread Jim Fulton
There is certainly some opportunity for doing things in parallel up until you get to tpc_vote. I wonder if renames in S3 take much time. I can image that they do. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more information about

Re: [ZODB-Dev] Wrong blob file being returned (similar to https://mail.zope.org/pipermail/zodb-dev/2011-February/014067.html )

2011-07-12 Thread Jim Fulton
to reporoduce the problem. I would think that if a request returns an incorrect Blob, it would continue to. If someone reports a bad blob, get the URL and see if you can reproduce by making the same request to each of the app servers, bypassing the load balencer. If one server is being bad, you

Re: [ZODB-Dev] Wrong blob file being returned (similar to https://mail.zope.org/pipermail/zodb-dev/2011-February/014067.html )

2011-07-13 Thread Jim Fulton
's a bug in ZODB, it's one I haven't seen and I'd have no idea what it's parameters might be. I suppose anything's possible. :) Note that, in the actually database file, blobs have no data. All that matters is their oid and serial. I can't imagine

Re: [ZODB-Dev] How to fix "CorruptedDataError: Error reading unknown oid. Found '' at 81036527"?

2011-07-14 Thread Jim Fulton
emoved during recovery > > Any more options? Use multi-zodb-check-refs with the -r option to get a database of reverse references. Use that do figure out what's refering to the missing object and fix that object(s) so that it no longer does. Jim -- Jim Fulton http://www.linkedin.com/in/jimfu

Re: [ZODB-Dev] How to fix "CorruptedDataError: Error reading unknown oid. Found '' at 81036527"?

2011-07-14 Thread Jim Fulton
On Thu, Jul 14, 2011 at 10:12 AM, Andreas Jung wrote: > > > Jim Fulton wrote: >>> >>> 2011-07-14 13:36:22 INFO Z2 Shutting down >> >> That error message seems rather weird. WTF SIGINT? > > Likely caused during the Zope shutdown amd unrelated). I find

Re: [ZODB-Dev] How to fix "CorruptedDataError: Error reading unknown oid. Found '' at 81036527"?

2011-07-14 Thread Jim Fulton
ter(a) > app._p_jar._added[a._p_oid] = a > transaction.commit() > > At Jarn we've used that trick many times to repair broken internals in > the intid/keyreference data structures. Do you have any theories why objects are going away for you? The only time we've seen

Re: [ZODB-Dev] How to fix "CorruptedDataError: Error reading unknown oid. Found '' at 81036527"?

2011-07-14 Thread Jim Fulton
On Thu, Jul 14, 2011 at 3:58 PM, Hanno Schlichting wrote: > On Thu, Jul 14, 2011 at 9:40 PM, Jim Fulton wrote: >> On Thu, Jul 14, 2011 at 3:23 PM, Hanno Schlichting wrote: >>> At Jarn we've used that trick many times to repair broken internals in >>> the in

Re: [ZODB-Dev] How to fix "CorruptedDataError: Error reading unknown oid. Found '' at 81036527"?

2011-07-15 Thread Jim Fulton
On Fri, Jul 15, 2011 at 3:58 AM, Hanno Schlichting wrote: > On Thu, Jul 14, 2011 at 10:30 PM, Jim Fulton wrote: >> Is the symptom you're seeing POSKeyError? > > Yes, ... This is disturbing. It would be good to better understand the reasons. I've thought of checking

Re: [ZODB-Dev] I would like to know if api exists nowadays to get list of all objects which where modified in a transaction ?

2011-07-26 Thread Jim Fulton
On Tue, Jul 26, 2011 at 3:55 AM, Stéphane Klein wrote: > Hi, > > some time ago, someone ask how to get list of all objects which where > modified in a transaction : > http://article.gmane.org/gmane.comp.web.zope.zodb/5734 > > Jim Fulton said « there isn't a public API fo

Re: [ZODB-Dev] I would like to know if api exists nowadays to get list of all objects which where modified in a transaction ?

2011-07-26 Thread Jim Fulton
ibers to events. It would be great if this kind of feature existed so > you could have an overview of what happens during a transaction. You don't need a public API for debugging. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For

Re: [ZODB-Dev] I would like to know if api exists nowadays to get list of all objects which where modified in a transaction ?

2011-07-28 Thread Jim Fulton
On Thu, Jul 28, 2011 at 9:38 AM, Stéphane Klein wrote: > Le 26/07/2011 09:55, Stéphane Klein a écrit : >> Hi, >> >> some time ago, someone ask how to get list of all objects which where >> modified in a transaction : >> http://article.gmane.org/gmane.comp.web.zop

Re: [ZODB-Dev] zc.zodbdgc bug - undefined variable

2011-08-05 Thread Jim Fulton
It should be: oldtid = f.read(8) Of course, this needs to be verified with a test. This seems to be a (obviously untested) case where there are multiple records of a garbage object. I'll have to ponder how this could actually happen. Jim -- Jim Fulton http://www.linked

Re: [ZODB-Dev] detect zodb with the unix tool 'file' (contrib)

2011-09-06 Thread Jim Fulton
quot; magic string for ZEO cache files. Not sure what the best description for that might be. Maybe "Zope Object Database Client Cache File". > Thanks for all your work, Thanks for following up on this. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton __

Re: [ZODB-Dev] detect zodb with the unix tool 'file' (contrib)

2011-09-07 Thread Jim Fulton
EC3    Zope Object Database Client Cache File (data) Question: What is the significance of "(data)"? Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZO

Re: [ZODB-Dev] detect zodb with the unix tool 'file' (contrib)

2011-09-07 Thread Jim Fulton
2011/9/7 Stéphane Blondon : > 2011/9/7 Jim Fulton : >> 2011/9/6 Stéphane Blondon : >> ... >>> # Cache file for the database of Zope (done by ClientStorage) >>> 0       string  ZEC3    Zope Object Database Client Cache File (data) >> >> Question: Wha

Re: [ZODB-Dev] notes re trying to run ZODB on PyPy

2011-09-25 Thread Jim Fulton
a great place for someone to help out. :) Just needs finishing IIRC. :) ... > At this point I was able to start up the Pyramid ZODB scaffold and add > an object to the DB root. ... > That's as far as I got. Hope this is helpful to the next guy. :) Very cool. Jim -- Jim Fulton http:

Re: [ZODB-Dev] the difference between normal class object and the ZODB Persistent subclass object

2011-09-27 Thread Jim Fulton
lication, two persistet objects refer to the same non-persistent object, after being stored and reloaded, they non-persistent object will be duplicated and unshared. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more information abou

Re: [ZODB-Dev] Is there any convenient way to reduce the storage file size after delete something in database?

2011-09-27 Thread Jim Fulton
to keep old (unpacked) files. If set to False, the old file will be automatically removed. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mai

Re: [ZODB-Dev] the difference between normal class object and the ZODB Persistent subclass object

2011-09-27 Thread Jim Fulton
On Tue, Sep 27, 2011 at 12:45 PM, Vincent Pelletier wrote: > Le mardi 27 septembre 2011 12:47:42, Jim Fulton a écrit : >> Generally speaking, clases and methods are not stored in the database, >> only class names. > > (This mail is out of topic... Probably only inte

Re: [ZODB-Dev] the difference between normal class object and the ZODB Persistent subclass object

2011-09-27 Thread Jim Fulton
On Tue, Sep 27, 2011 at 2:26 PM, Stephan Richter wrote: ... > Also, how does the ZODB handle circular references within non-persistent > objects? I assume pickle handles this? Yes. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ Fo

[ZODB-Dev] Call for volunteers: help w finishing Python BTrees

2011-10-04 Thread Jim Fulton
, both of which I'd like to see. svn+ssh://svn.zope.org/repos/main/ZODB/branches/jim-python-btrees Jim P.S. Much thanks to Tres for his work on the Python version of persistence. -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For

[ZODB-Dev] RFC: oodb namespace package

2011-10-04 Thread Jim Fulton
a namespace package. Thoughts? I'd have prefered "odb", but that's sort of taken. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-De

Re: [ZODB-Dev] Call for volunteers: help w finishing Python BTrees

2011-10-04 Thread Jim Fulton
On Tue, Oct 4, 2011 at 11:36 AM, David Glick wrote: > On 10/4/11 8:33 AM, Jim Fulton wrote: >> >> Someone recently told me I should be more agressive about asking for help. >> >> If someone is looking for an opportunity to help, finishing the Python >> version of

Re: [ZODB-Dev] RFC: oodb namespace package

2011-10-05 Thread Jim Fulton
On Tue, Oct 4, 2011 at 1:07 PM, David Glick wrote: > On 10/4/11 9:59 AM, Jim Fulton wrote: >> >> Based on some discussions with Alan Runyan over the last few days, >> I've created an oodb "package" as a namespace package: >> >>   http://pypi.python.

Re: [ZODB-Dev] RFC: oodb namespace package

2011-10-05 Thread Jim Fulton
On Tue, Oct 4, 2011 at 1:45 PM, Vincent Pelletier wrote: > Le mardi 04 octobre 2011 18:59:32, Jim Fulton a écrit : >> I've created an oodb "package" as a namespace package: > > Sounds good to me. > >> Thoughts? > > What would be the structure ? First

Re: [ZODB-Dev] zeo.memcache

2011-10-05 Thread Jim Fulton
On Wed, Oct 5, 2011 at 1:40 PM, Pedro Ferreira wrote: > Hello all, > > While doing some googling on ZEO + memcache I came across this: > > https://github.com/eleddy/zeo.memcache Interesting. I'll review it. Thanks for pointing it out. Jim -- Jim Fulton http://www.linke

Re: [ZODB-Dev] zeo.memcache

2011-10-09 Thread Jim Fulton
#x27;t think it > would be very difficult.  How many people would be interested in such a > thing? This would be of broad interest! Can you briefly describe the strategy? How do you arrange that the client sees a consistent view of the current tid

Re: [ZODB-Dev] Using demostorage + blobs with ZEO

2011-10-17 Thread Jim Fulton
rk, even if the client is read only. To get around this, we typically use a before storage, which freezes the client storage at a point in time. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more information about ZODB, see http://z

Re: [ZODB-Dev] ZODB on ironPython?!

2011-10-20 Thread Jim Fulton
modules. Is that at the > moment possible running ZODB on .NET ironpython?! As of now, there are some required C extensions. We're in the process of trying to make these optional. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more i

Re: [ZODB-Dev] ZODB on ironPython?!

2011-10-20 Thread Jim Fulton
On Thu, Oct 20, 2011 at 2:34 PM, Tamer Higazi wrote: > You told amount of work had been done?! Yes!!! > In what stage are they now?! They aren't done yet!!! > Where can I get the latest informations according these cases? Here. You just saw it. :) Jim -- Jim Fulton http://ww

Re: [ZODB-Dev] ZODB on ironPython?!

2011-10-26 Thread Jim Fulton
27;re used to. If a performance is an issue, you may find yourself wanting to write c# versions of these. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] ZODB on ironPython?!

2011-10-26 Thread Jim Fulton
On Wed, Oct 26, 2011 at 2:09 PM, Tamer Higazi wrote: > dear Jim, > Thank your for your quick answer. > > Am 26.10.2011 19:34, schrieb Jim Fulton: >> On Wed, Oct 26, 2011 at 12:59 PM, Tamer Higazi wrote: >>> Hey Jim! >>> Hey just cool down. >> >>

Re: [ZODB-Dev] Error 'error: uncaptured python exception, closing channel ... ', When use ZEO in python client( not zope...)

2011-10-29 Thread Jim Fulton
[/usr/lib/python2.6/asyncore.py|recv|362] > [/usr/lib/python2.6/asyncore.py|recv|588]) This looks like some sort of system problem, otherwise, I have no idea. :) Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] Transaction notes

2011-11-15 Thread Jim Fulton
. > > My question is: Will using transaction 'note' in a non-writing transaction > cause a record to be written out when using FileStorage and/or ZEO? No. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more infor

[ZODB-Dev] RFC: reduce ZEO default min-disconnect-poll to 1 and default max-disconnect-poll count to min-disconnect-poll.

2011-12-09 Thread Jim Fulton
nnect-poll > >). If there are no objections, I'll make this change in 3.11. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://

Re: [ZODB-Dev] Question about zeo client cache

2011-12-11 Thread Jim Fulton
e and lots of server loads makes the server even more heavily loaded. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] Question about zeo client cache

2011-12-11 Thread Jim Fulton
On Sun, Dec 11, 2011 at 7:44 AM, Malthe Borch wrote: > On 11 December 2011 13:22, Jim Fulton wrote: ... > If it's in the disk cache, to my understanding, it might as well be in > the ZEO client's pickle cache. You mean the ZODB object cache? Compressed pickles in the zeo cli

[ZODB-Dev] RFC: Remove KeyError as base class of POSKeyError

2011-12-15 Thread Jim Fulton
While this is a backward incompatible change, I can't imagine unbroken code that intentionally catches POSKeyErrors via KeErrors. If there are no objections, I'll make this change in 3.11. JIm -- Jim Fulton http://www.linkedin.com/in/jimfulton _

Re: [ZODB-Dev] Can iterate thru records - but cannot access same record directly by key

2011-12-28 Thread Jim Fulton
> tom > >        base = db.dataConn["1292289873"]["books"]["1292289873"] What is db? What version of ZODB are you using? What Storage? Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more informat

Re: [ZODB-Dev] [Zope-DB] How to extract users info from ZODB?

2012-01-25 Thread Jim Fulton
thread is about Zope, not ZODB. Please move the discussion to zope-dev. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/m

Re: [ZODB-Dev] zeopack error

2012-01-30 Thread Jim Fulton
structure > that prevents zeopack to run correctly? There is no verbose mode with zeopack > as far as I know that would provide me with more helpful information (like > the transaction id?). Maybe I can use any of the other tools to get to the > source of the problem? > > Any experi

Re: [ZODB-Dev] zeopack error

2012-02-09 Thread Jim Fulton
, >> > line 328, in findrefs >> >     return self.referencesf(self._file.read(dh.plen)) >> >   File >> > "/usr/local/lib/python2.6/dist-packages/ZODB3-3.10.3-py2.6-linux-i686.egg/ZODB/serialize.py", >> >

Re: [ZODB-Dev] [RFC] Allow serial to be returned as late as tpc_finish.

2012-02-09 Thread Jim Fulton
On Tue, Feb 7, 2012 at 12:05 PM, Vincent Pelletier wrote: > Le jeudi 16 décembre 2010 17:19:33, Jim Fulton a écrit : >> Cool. I'll review this for 3.11 and it or somethink like it will go >> into that release. > > bump :) Um. Thanks. Jim -- Jim Fulton http://www

Re: [ZODB-Dev] zeopack error

2012-02-09 Thread Jim Fulton
#x27;re packing all the storages together, presumably. zeopack only works on a single storage and thus can't take cross-database references into account. See http://pypi.python.org/pypi/zc.zodbdgc Perhaps this should be included in ZODB 3.11. Jim -- Jim Fulton http://www.linkedin.com/i

Re: [ZODB-Dev] Server-side caching

2012-02-10 Thread Jim Fulton
r if it's not faster to ask the DB for an oid that is already in > memory instead of retrieving it from the client cache? In general, I'd say no. It can depend on lots of details, including: - database size - active set size - network speed - memory and disk speeds on clie

Re: [ZODB-Dev] Server-side caching

2012-02-13 Thread Jim Fulton
lly figure out how to use memcached to implement a shared ZEO cache, as has been done for relstorage. At PyCon, I'll be presenting work I've been doing on a load balancer that seeks to avoid sharing the same data in multiple caches by assigning different kinds of work to different wor

[ZODB-Dev] Build compression into ZODB 3.11?

2012-03-14 Thread Jim Fulton
I'm pretty happy with how zc.zlibstorage has worked out. Should I build this into ZODB 3.11? BTW, lz4 compression looks interesting. The Python binding (at least from PyPI) is broken. I submitted an issue. Hopefully it will be fixed. Jim -- Jim Fulton http://www.linkedin.com/in/jimf

Re: [ZODB-Dev] Build compression into ZODB 3.11?

2012-03-15 Thread Jim Fulton
On Wed, Mar 14, 2012 at 7:46 PM, Stephan Richter wrote: > On Wednesday, March 14, 2012 01:47:54 PM Jim Fulton wrote: >> I'm pretty happy with how zc.zlibstorage has worked out. >> >> Should I build this into ZODB 3.11? > > Hi Jim, > > I am +1 as well.

Re: [ZODB-Dev] Build compression into ZODB 3.11?

2012-03-15 Thread Jim Fulton
On Wed, Mar 14, 2012 at 1:47 PM, Jim Fulton wrote: ... > BTW, lz4 compression looks interesting. Mainly because it (claims that it) makes compression and decompression much faster. While zlibstorage is a significant net win, the cpu load it adds is noticeable. > The Python binding (at

Re: [ZODB-Dev] How to update an object in a multithreading application?

2012-03-19 Thread Jim Fulton
ZEO sometimes didn't have it's own thread. Remember that with multi-version concurrency control, you always see the database as it was at the beginning of the transaction. If you want to see updates, you need to start a new transaction. Jim -- Jim Fulton

Re: [ZODB-Dev] Build compression into ZODB 3.11?

2012-03-20 Thread Jim Fulton
On Thu, Mar 15, 2012 at 11:09 AM, Jim Fulton wrote: > On Wed, Mar 14, 2012 at 1:47 PM, Jim Fulton wrote: ... > At some point soonish, I'll do some > tests against a large database. On a database with 180 million records, 150 million of which are compressable:

Re: [ZODB-Dev] database conflict fail

2012-03-22 Thread Jim Fulton
ly retrying is fine. If your application wants to update the same value from multiple threads often, then that's a design problem you should solve. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] transaction "Attempts" class

2012-03-28 Thread Jim Fulton
retry #. I do plan to do this, um, when I get around to it. Might be a nice small project for someone who wants to contribute. The with statement is really wanting when you might want to execute code other than once. Jim -- Jim Fulton http://www.linkedin.c

Re: [ZODB-Dev] transaction "Attempts" class

2012-03-28 Thread Jim Fulton
On Wed, Mar 28, 2012 at 4:37 PM, Chris McDonough wrote: > On Wed, 2012-03-28 at 14:21 -0400, Jim Fulton wrote: ... > A decorator for running some code in the context of a txn and retrying > retryable exceptions would be nice higher level behavior.  I'd be > willing to do th

Re: [ZODB-Dev] transaction "Attempts" class

2012-03-29 Thread Jim Fulton
On Thu, Mar 29, 2012 at 2:50 AM, Chris McDonough wrote: > On Wed, 2012-03-28 at 18:37 -0400, Chris McDonough wrote: >> On Wed, 2012-03-28 at 17:06 -0400, Jim Fulton wrote: >> > On Wed, Mar 28, 2012 at 4:37 PM, Chris McDonough wrote: >> > > On Wed, 2012-03-28 at

Re: [ZODB-Dev] transaction "Attempts" class

2012-03-29 Thread Jim Fulton
On Wed, Mar 28, 2012 at 6:37 PM, Chris McDonough wrote: > On Wed, 2012-03-28 at 17:06 -0400, Jim Fulton wrote: >> On Wed, Mar 28, 2012 at 4:37 PM, Chris McDonough wrote: >> > On Wed, 2012-03-28 at 14:21 -0400, Jim Fulton wrote: >> ... >> > A decorator for runn

Re: [ZODB-Dev] transaction "Attempts" class

2012-03-31 Thread Jim Fulton
On Thu, Mar 29, 2012 at 6:59 PM, Chris McDonough wrote: > On Thu, 2012-03-29 at 07:18 -0400, Jim Fulton wrote: >> On Wed, Mar 28, 2012 at 6:37 PM, Chris McDonough wrote: >> > On Wed, 2012-03-28 at 17:06 -0400, Jim Fulton wrote: >> >> On Wed, Mar 28, 2012 at 4

Re: [ZODB-Dev] Storing blob data out of zodb into amazon s3?

2012-07-05 Thread Jim Fulton
level, rather than going through ZODB. Alternatively, it would be nice to have a way to get the S3 location for a blob. Maybe this could be done through some extension API. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm _

Re: [ZODB-Dev] zope.generations to standalone ZODB ?

2012-08-27 Thread Jim Fulton
ne ZODB database ? It should be. If it isn't it's a bug. >* Are there other, similar, facilities which would track change to >application classes (or at least accept notifications of them) and >automatically 'fixup' the corresponding classes already in the &

Re: [ZODB-Dev] [Zope-dev] [announce] NEO 1.0 - scalable and redundant storage for ZODB

2012-08-29 Thread Jim Fulton
h, if at all. The strategy should provide greater transaction throughput and reduce latency. It's a strategy I'd like to implement for ZEO at some point. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm

Re: [ZODB-Dev] [announce] NEO 1.0 - scalable and redundant storage for ZODB

2012-08-29 Thread Jim Fulton
rk that way, since there's no relstorage server. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope

[ZODB-Dev] RFC: ZODB 4.0 (without persistent)

2012-10-14 Thread Jim Fulton
roject and that doesn't include ZEO. - ZEO 4 (4.0.0a1 initially) - ZODB3 3.11 (3.11.0.a1 initially) that depends on ZODB and ZEO and is otherwise empty. If there are no objections, I'll release these in a few days. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is bet

Re: [ZODB-Dev] RFC: ZODB 4.0 (without persistent)

2012-10-14 Thread Jim Fulton
On Sun, Oct 14, 2012 at 5:03 PM, Hanno Schlichting wrote: > On Sun, Oct 14, 2012 at 10:21 PM, Jim Fulton wrote: >> Goal: Give persistent, ZODB and ZEO their own release cycles. > > In which distribution would BTrees end up in? Still ZODB. > I think the pure-Python work fo

Re: [ZODB-Dev] RFC: ZODB 4.0 (without persistent)

2012-10-14 Thread Jim Fulton
(I don't think > anything depends on BTrees). FileStorage uses BTrees for it's in-memory index. MappingStorage used BTrees. There are ZODB tests that use BTrees, but I suppose they could be fixed. I just don't think the win is that great in separating BTrees at this time. Ji

Re: [ZODB-Dev] RFC: ZODB 4.0 (without persistent)

2012-10-14 Thread Jim Fulton
On Sun, Oct 14, 2012 at 6:07 PM, Laurence Rowe wrote: > On 14 October 2012 22:49, Jim Fulton wrote: >> On Sun, Oct 14, 2012 at 5:28 PM, Tres Seaver wrote: >> ... >>>> Well, I don't have time to chase BTrees. This could always be done in >>>> ZODB 5.

<    1   2   3   4   5   6   7   8   9   10   >