Re: [ZODB-Dev] Big catalog indexes/btrees, reads, and the object cache

2012-12-15 Thread Stefan H. Holek
On 14.12.2012, at 01:26, Jeff Shell wrote:

> The document counts of these indexes are around 17 items each, and tend 
> to cover every major-ish content object. I'm wondering how this is impacting 
> the object caches and how we should be sizing them.
> 
> Are we hitting a limit on BTree and/or catalog index size?
> Do we need to keep upping the zodb-object-cache size? (What is a good metric 
> for setting that cache size?)

As a rule of thumb, the cache size should be about the same as the catalog 
size, i.e. at least 170k in your case. I don't know of any other limit in 
catalog or indexes.

Stefan

-- 
Stefan H. Holek
ste...@epy.co.at

___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] copying objects from one database to another

2012-10-04 Thread Stefan H. Holek
On 20.09.2012, at 12:27, Sanjay Rao wrote:

> I am trying to copy selective objects from one database to another database. 
> we are using following functions to accomplish this.
> 
> ZODB.Connection.Connection.ExportFile
> ZODB.Connection.Connection.ImportFile
> 
> In this process destination file become very large after copy operation, plus 
> it takes a lot of memory.
> for example 
> source database size is 500 MB
> initial destination database size is 700 MB
> finally destination database size become around 20GB. After packing the final 
> database it came to normal size but 
> 
> - Sometime it creates disk space issue on machine
> - It consumes a lot of memory
> 
> Is somebody aware of any solution to this problem ? 
> Thanks a lot in advance.

I think the solution is to provide enough disk space and memory. You may also 
want to check out RelStorage which has different resource usage patterns than 
FileStorage.

Stefan

-- 
Stefan H. Holek
ste...@epy.co.at

___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Is any function called when an object is loaded from the database?

2012-06-20 Thread Stefan H. Holek
One thing I seem to remember is that changes made in __setstate__ do not mark 
the object as dirty. This is because after being loaded an object is up-to-date 
by definition. 

I suggest going with zope.generations.

Stefan


On 19.06.2012, at 23:42, Malthe Borch wrote:

> On 19 June 2012 23:38, Claudiu Saftoiu  wrote:
>> Is there any hook to call *after* the instance attributes get set/loaded
>> from the database?
> 
> True. I guess you'll have to go with Marius' suggestion – ``__setstate__``.

-- 
Stefan H. Holek
ste...@epy.co.at

___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] RelStorage - Make memcached timeouts configurable

2011-10-17 Thread Stefan H. Holek
Hi,

Memcached allows to specify a variety of timeouts in its "behaviors", in 
particular there are connect_timeout, send_timeout and receive_timeout. [1]

Since we recently had to customize relstorage.pylibmc_wrapper.Client to 
accommodate the timeout values, I was wondering if it made sense to make them 
configurable in the relstorage section of zope.conf?

Anybody else has the need for timeouts?

Stefan

[1] http://sendapatch.se/projects/pylibmc/behaviors.html

-- 
Stefan H. Holek
ste...@epy.co.at





___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Increasing MAX_BUCKET_SIZE for IISet, etc

2011-01-26 Thread Stefan H. Holek
On 26.01.2011, at 21:15, Matt Hamilton wrote:

> I'm not sure how the current MAX_BUCKET_SIZE values were determined,
> but looks like they have been the same since the dawn of time, and I'm
> guessing might be due a tune?


Go Matt go! :-)

-- 
Stefan H. Holek
ste...@epy.co.at

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] RelStorage: Clearing temp_store in replication-friendly way

2008-07-30 Thread Stefan H. Holek

On 27. Jul 2008, at 18:48, Shane Hathaway wrote:

I would expect that when a slave goes down, the slave must replay  
all of the statements since the beginning of a transaction,  
including the statements that create temporary tables.  Does it not  
work that way?


Not in MySQL anyway. MySQL knows transactions at table-level only.  
For replication, the slave maintains the name of the master log file  
and a pointer to the next line to be read from it. That's all.


Do slaves respect transaction boundaries?  If they don't, then ZODB  
clients of slaves will miss object updates, leading to sporadically  
inconsistent ZODB caches, especially under load.


Slaves replay BEGIN, COMMIT, and ROLLBACK statements issued on the  
master.


We can do that, but I hope it doesn't impact performance too much.   
It seems a shame to not use a RAM-based temporary table.  OTOH,  
I've tried to structure RelStorage to allow changes like this  
without too much effort.


I am primarily aiming for redundancy here, not performance.

And -BTW- you have and thanks for that.

Stefan

--
Anything that, in happening, causes itself to happen again,
happens again.  --Douglas Adams


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] RelStorage: Clearing temp_store in replication-friendly way

2008-07-27 Thread Stefan H. Holek
This is not a problem of the ZODB or relstorage, but specific to how  
MySQL handles a) replication and b) temporary tables.


MySQL employs a log-based replication mechanism. This means the  
replication slave replays the commands performed on the master to  
keep its copy of the database in sync.


In MySQL, temporary tables live in RAM. So when a slave goes down,  
its copy of the table vanishes. When the slave comes back up the log  
may still contain commands using the temporary table however, causing  
execution to barf (and replication to stop). To fix this condition,  
and get replication going again, we have to perform a manual copy of  
the master's database to the affected slave.


We believe the solution is to avoid temporary tables altogether, and  
to recreate the needed semantics in a replication-safe way. The  
refactoring for temp_store could look like:


1) Create 'temp_store' as a permanent table.
2) Add a 'connid' column, storing the MySQL connection id.
3) Use CONNECTION_ID() in all inserts to populate 'connid'.
4) Qualify all updates and queries using 'temp_store' with
   WHERE connid = CONNECTION_ID() or equivalent.
5) Clear entries from 'temp_store' at transaction boundaries with
   DELETE FROM temp_store WHERE connid = CONNECTION_ID().

I plan to work on this in the near future.

Stefan


On 24. Jul 2008, at 18:33, Shane Hathaway wrote:

We hope to tackle the main issue (a.k.a. better-not-use-temporary-  
tables-with-mysql-replication-at-all) in a later installment.


Ok.  Conceptually, what we need is a way for each connection to  
write to a scratch table that no other connection can see.  Is  
there a better way to do that than temporary tables?


--
Anything that, in happening, causes something else to happen,
causes something else to happen.  --Douglas Adams


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] RelStorage: SVN trunk not up-to-date

2008-07-27 Thread Stefan H. Holek
Most projects on svn.zope.org have the policy to perform new work on  
the trunk, cutting maintenance branches from it when appropriate.  
RelStorage appears to forge ahead on its 1.1 branch, allowing the  
trunk to go stale.


What's the policy here? If I wanted to create a branch to work on  
something new, would I branch off 1.1?


Thanks,
Stefan

--
Anything that, in happening, causes something else to happen,
causes something else to happen.  --Douglas Adams

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] RelStorage: Clearing temp_store in replication-friendly way

2008-07-24 Thread Stefan H. Holek

I have made two checkins to relstorage 1.1 branch:

[1] http://svn.zope.org/relstorage/branches/1.1/?rev=88789&view=rev
[2] http://svn.zope.org/relstorage/branches/1.1/?rev=88790&view=rev

If someone wants to discuss them this is the thread. ;-)

Ad [1]:
This clearly is not the only problem with temporary tables and  
replication. What the change does, however, is turn a hard error -  
which stops replication in a way requiring operator intervention -  
into a recoverable failure. And all this in an application neutral  
way! 


We hope to tackle the main issue (a.k.a. better-not-use-temporary- 
tables-with-mysql-replication-at-all) in a later installment.


Cheers,
Stefan

P.S.: I am quite excited about the memcached support. Does it "just  
work"? I.e. can I run my ZODB in RAM now? ;-)


--
Anything that, in happening, causes itself to happen again,
happens again.  --Douglas Adams


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Re: Snapshotting state with DemoStorage

2008-05-28 Thread Stefan H. Holek

On 28.05.2008, at 09:48, Martin Aspeli wrote:

I think the main problem is that I'm not terribly familiar with how  
DemoStorage works, or how the Zope server (in a test case) ends up  
getting its database. I'm worried that I may be asking the "wrong"  
questions here, but to my mind, the three questions would be:


The tests get their database from Testing.custom_zodb.py

- how can I use DemoStorage to snapshot a known state and then  
return to it at all


See ZopeLite.sandbox


- how can I ensure that this is the storage that the Zope server sees


See sandbox.Sandboxed and the AppZapper class

- how can I actually access a server that's running inside a  
ZopeTestCase over a URL, i.e. ensure that it binds to a port and  
serves requests for the duration of the test run


ZopeTestCase.utils.startZServer()

And then something like urllib.urlopen(self.portal.absolute_url())

--
Stefan H. Holek
[EMAIL PROTECTED]

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Snapshotting state with DemoStorage

2008-05-28 Thread Stefan H. Holek
You can base DemoStorage on another storage. This could be simple with  
ZTC, haven't tried in a while. See ZopeLite.sandbox() and sandbox.py.  
You may be able to get away with minor mods to the base classes, e.g.  
overriding the _app() method of Sandboxed/Functional.


Stefan


On 27.05.2008, at 23:22, Martin Aspeli wrote:

Is it possible to create a fast "snapshot" with DemoStorage e.g. in  
a ZopeTestCase layer, and then have the tear-down after each test  
revert to this state? Doing the full setup (creating a new Plone  
site, populating it) between each test run is prohibitively expensive!


--
Stefan H. Holek
[EMAIL PROTECTED]




___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] RelStorage: Which tables to replicate?

2008-05-22 Thread Stefan H. Holek

Hi Russ,

This is correct. I should have mentioned that the two servers are  
configured so that server A creates odd, server B even sequence  
numbers. This makes the duplicate id error pretty mysterious, IMO ;-)


Stefan

On 22.05.2008, at 13:15, Russ Ferriday wrote:

I have not read the reference, but it seems that this scheme can  
never work without keys (OIDs) being qualified by which master they  
originated on. Otherwise one of the mySQLs would need to be  
responsible for handing out unique oids.  You need an OID  
origination namespace for each master.


--
Stefan H. Holek
[EMAIL PROTECTED]




___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] RelStorage: Which tables to replicate?

2008-05-22 Thread Stefan H. Holek
We have put up two boxes in a MySQL master-master replication setup  
[1]. As long as we only write to one of the masters all is fine.  
Writing to both masters (and expecting MySQL to sort it out) is giving  
us trouble. We see things like:


Last_Errno: 1062
Last_Error: Error 'Duplicate entry '41860' for key 1' on query.
Default database: 'prod_zodb'.
Query: 'INSERT INTO new_oid VALUES ()'

and

Last_Errno: 1050
Last_Error: Error 'Table 'temp_store' already exists' on query.
Default database: 'prod_zodb'.
Query: 'CREATE TEMPORARY TABLE temp_store
   (
zoidBIGINT NOT NULL PRIMARY KEY,
prev_tidBIGINT NOT NULL,
md5 CHAR(32),
state   LONGBLOB
   ) ENGINE MyISAM'

Stefan

[1] http://www.howtoforge.com/mysql_master_master_replication


On 20.05.2008, at 18:36, Shane Hathaway wrote:

That is mostly correct.  You need to replicate transaction,  
object_state, current_object, and do something with new_oid.  There  
is no requirement to replicate the other tables, since they are used  
only for packing, although replicating object_ref and  
object_refs_added may save some work.


The new_oid table is special to the MySQL adapter.  RelStorage uses  
the new_oid table as a kind of sequence.  Like any sequence, it is  
intentionally non-transactional.  Perhaps there would be no ill  
effects in switching it to the InnoDB engine.


Which MySQL replication method are you looking into?


--
Stefan H. Holek
[EMAIL PROTECTED]

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] RelStorage: Which tables to replicate?

2008-05-20 Thread Stefan H. Holek
Let me rephrase this. Would it be ok to change *all* tables to use the  
InnoDB engine (except new_oid)?


Stefan

On 19.05.2008, at 10:55, Stefan H. Holek wrote:

I am playing with database replication (MySQL) and have found the  
following caveats:


- don't mix engine types (MyISAM, InnoDB)
- don't use temporary tables

Now, RelStorage does both and so I was wondering how to proceed with  
replication. Would it be prudent to assume that all I ever need to  
replicate are the InnoDB tables "transaction", "object_state", and  
"current_object"? All other tables don't seem to be used across  
transaction boundaries, but I may be missing something here.


--
Stefan H. Holek
[EMAIL PROTECTED]




___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] RelStorage: Which tables to replicate?

2008-05-19 Thread Stefan H. Holek
Thanks Tino. I am aware of alternatives. I am however looking for an  
answer for the specific case of MySQL master-master replication.  
Exciting times!


Stefan

On 19.05.2008, at 12:04, Tino Wildenhain wrote:


Stefan H. Holek wrote:
I am playing with database replication (MySQL) and have found the  
following caveats:

   - don't mix engine types (MyISAM, InnoDB)
   - don't use temporary tables
Now, RelStorage does both and so I was wondering how to proceed  
with replication. Would it be prudent to assume that all I ever  
need to


Maybe use postgres and slony instead? The mysql shortcomings on
replication and integrity should be well known.

It should also be possible to replicate on ZODB layer instead.
At least I read something in the ML beside ZRE of course :-)

Cheers
Tino


--
Stefan H. Holek
[EMAIL PROTECTED]




___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] RelStorage: Which tables to replicate?

2008-05-19 Thread Stefan H. Holek
I am playing with database replication (MySQL) and have found the  
following caveats:


- don't mix engine types (MyISAM, InnoDB)
- don't use temporary tables

Now, RelStorage does both and so I was wondering how to proceed with  
replication. Would it be prudent to assume that all I ever need to  
replicate are the InnoDB tables "transaction", "object_state", and  
"current_object"? All other tables don't seem to be used across  
transaction boundaries, but I may be missing something here.


Thanks,
Stefan

--
Stefan H. Holek
[EMAIL PROTECTED]




___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] RelStorage: Unable to acquire commit lock

2008-05-10 Thread Stefan H. Holek

yeas and yes ;-)

Srefan


On 9. Mai 2008, at 18:31, Shane Hathaway wrote:


StorageError: Unable to acquire commit lock


It takes a few moments for that error to appear, doesn't it?

Perhaps the scope of the commit lock in MySQL is the entire  
database server rather than just the database, which would explain  
this.  We should make sure each database has its own commit lock  
and pack lock.


--
Anything that, in happening, causes itself to happen again,
happens again.  --Douglas Adams


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] RelStorage: Unable to acquire commit lock

2008-05-08 Thread Stefan H . Holek
I try to use two relstorages, roughly following Evan's ZopeLabs recipe  
[1]. This is how my zope.conf looks like:



%import relstorage


db test_zodb
host localhost


mount-point /



%import relstorage


db test_sessions
host localhost


mount-point /sessions_folder


However, when I want to mount the /sessions_folder I get this:

2008-05-08 21:13:15 ERROR Zope.SiteErrorLog 
http://localhost:8080/manage_addProduct/ZODBMountPoint/manage_addMounts
Traceback (innermost last):
  Module ZPublisher.Publish, line 125, in publish
  Module Zope2.App.startup, line 238, in commit
  Module transaction._manager, line 96, in commit
  Module transaction._transaction, line 395, in commit
  Module transaction._transaction, line 498, in _commitResources
  Module ZODB.Connection, line 730, in tpc_vote
  Module relstorage.relstorage, line 529, in tpc_vote
  Module relstorage.relstorage, line 513, in _vote
  Module relstorage.relstorage, line 420, in _prepare_tid
  Module relstorage.adapters.mysql, line 406, in start_commit
StorageError: Unable to acquire commit lock

Thanks,
Stefan

[1] http://www.zopelabs.com/cookbook/1061234337

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] What's the deal with _p_independent?

2007-07-07 Thread Stefan H. Holek

https://bugs.launchpad.net/zodb/+bug/124613

Stefan


On 7. Jul 2007, at 19:18, Jim Fulton wrote:


Would you mind creating a launchpad issue to that effect?


--
Anything that, in happening, causes itself to happen again,
happens again.  --Douglas Adams


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] What's the deal with _p_independent?

2007-07-07 Thread Stefan H. Holek
BTrees.Length is used in many places to maintain the length of  
BTrees. Just the other day it was added to zope.app.container.btree.  
While I am happy about the speed improvements, I am concerned about  
the fact that BTrees.Length declares itself _p_independent. I'd like  
some clarification about what happens in a conflict resolution  
situation, when the Length is _p_independent but the BTree itself is  
not. I *think* that with MVCC this means a read-conflict will reset  
the BTree, but not it's Length.


All I could google up is this from 2004: http://mail.zope.org/ 
pipermail/zodb-dev/2004-April/007269.html


Now, do we need another Length class or is BTrees.Length just fine  
and dandy?


Thanks,
Stefan


--
Anything that happens, happens.  --Douglas Adams


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Concurrent transactions

2007-06-01 Thread Stefan H. Holek
Counting in the ZODB is more or less a no-go. You will get write  
conflicts, and your ZODB will grow (too) quickly. That said, you may  
want to look at BTrees.Length for a "counter" with built-in conflict  
resolution.


Stefan


On 1. Jun 2007, at 09:47, Kai Diefenbach wrote:


After reading some documentation about transactions, concurrency and
atomicity, I don't think that the following issue is a problem, but  
I'd
be nice to get confirmation from some exports. Or - of course -  
that it

doesn't behave in the way I think.

Within an adapter I increase a counter (e.g. for every download of an
file) by doing:

self.counter[0] += 1

self.counter is a PersistentList stored as annotation.

Could there be a problem with concurrent access to the counter? With
other words:


--
Anything that, in happening, causes something else to happen,
causes something else to happen.  --Douglas Adams


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.8.0a1 released

2007-05-13 Thread Stefan H. Holek
You would now use transaction.savepoint(True). Note also that you  
don't need to abort a subtransaction or optimistic savepoint, it will  
be aborted/committed as part of the "full" transaction.


Stefan


On 12. Mai 2007, at 19:00, Paul Winkler wrote:


I belatedly realized I have some old test code that uses a
"subtransaction hack" to get a _p_jar and _p_oid:

try:
transaction.commit(1)
# do something needing _p_jar or _p_oid
finally:
transaction.abort(1)


--
It doesn't necessarily do it in chronological order, though.
  --Douglas Adams


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] more lockup information / zope2.9.6+zodb 3.6.2

2007-04-11 Thread Stefan H. Holek
Are you, by chance, using Intel E1000 NICs? See "Fighting the  
hardware" in Jodok's write-up: http://www.lovelysystems.com/batlogg/ 
2007/03/30/the-decathlon-of-computer-science/


Stefan


On 10. Apr 2007, at 20:19, Alan Runyan wrote:


I am seeing something *very* strange in zeo.log:
2007-04-10T12:20:53 ERROR ZEO.zrpc.Connection(S)  
(172.16.235.120:49351) Erro

r caught in asyncore
Traceback (most recent call last):
 File "/usr/local/python-2.4.4/lib/python2.4/asyncore.py", line 69,  
in read

   obj.handle_read_event()
 File "/usr/local/python-2.4.4/lib/python2.4/asyncore.py", line  
391, in han

dle_read_event
   self.handle_read()
 File "/usr/local/zope/lib/python/ZEO/zrpc/smac.py", line 147, in  
handle_re

ad
   d = self.recv(8192)
 File "/usr/local/python-2.4.4/lib/python2.4/asyncore.py", line  
343, in rec

v
   data = self.socket.recv(buffer_size)
error: (113, 'No route to host')


but several more of these:
2007-04-10T13:55:36 ERROR ZEO.zrpc.Connection(S)  
(172.16.235.119:44322) Erro

r caught in asyncore
Traceback (most recent call last):
 File "/usr/local/python-2.4.4/lib/python2.4/asyncore.py", line 69,  
in read

   obj.handle_read_event()
 File "/usr/local/python-2.4.4/lib/python2.4/asyncore.py", line  
391, in han

dle_read_event
   self.handle_read()
 File "/usr/local/zope/lib/python/ZEO/zrpc/smac.py", line 147, in  
handle_re

ad
   d = self.recv(8192)
 File "/usr/local/python-2.4.4/lib/python2.4/asyncore.py", line  
343, in rec

v
   data = self.socket.recv(buffer_size)
error: (110, 'Connection timed out')


--
Anything that, in happening, causes itself to happen again,
happens again.  --Douglas Adams


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] History-less FileStorage?

2006-12-29 Thread Stefan H. Holek

Do we have a history-less (i.e. no-grow) FileStorage?

Thanks,
Stefan

--
Anything that, in happening, causes something else to happen,
causes something else to happen.  --Douglas Adams


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] ZODB.tests.testZODB.checkResetCachesAPI definitely bad

2006-09-12 Thread Stefan H. Holek
These errors are *also* due to  
ZODB.tests.testZODB.checkResetCachesAPI. The cache reset appears to  
leave the Application object and/or transactions in a borked state.


All is well when I suppress checkResetCachesAPI:

$ python2.4 test.py -q -m '!^(ZEO|zope[.]app[.])' -t '! 
checkResetCachesAPI' --all

Running unit tests:
  Ran 7062 tests with 0 failures and 0 errors in 494.930 seconds.
Running  
Testing.ZopeTestCase.zopedoctest.testLayerExtraction.TestLayer tests:
  Set up  
Testing.ZopeTestCase.zopedoctest.testLayerExtraction.TestLayer in  
0.005 seconds.

  Running:
..
  Ran 2 tests with 0 failures and 0 errors in 0.025 seconds.
Tearing down left over layers:
  Tear down  
Testing.ZopeTestCase.zopedoctest.testLayerExtraction.TestLayer in  
0.003 seconds.

Total: 7064 tests, 0 failures, 0 errors

Stefan


testInsideOutVirtualHosting  
(webdav.tests.testPUT_factory.TestPUTFactory)

testNoVirtualHosting (webdav.tests.testPUT_factory.TestPUTFactory)

Error in test testNoVirtualHosting  
(webdav.tests.testPUT_factory.TestPUTFactory)

Traceback (most recent call last):
  File "/usr/lib/python2.4/unittest.py", line 251, in run
self.setUp()
  File "/home/buildbot/slave-zope.org/zc-buildbot--Linux--Zope--- 
branches---2.9--2.4/build/lib/python/webdav/tests/ 
testPUT_factory.py", line 22, in setUp

self.app.manage_addFolder('folder', '')
  File "/home/buildbot/slave-zope.org/zc-buildbot--Linux--Zope--- 
branches---2.9--2.4/build/lib/python/OFS/Folder.py", line 49, in  
manage_addFolder

self._setObject(id, ob)
  File "/home/buildbot/slave-zope.org/zc-buildbot--Linux--Zope--- 
branches---2.9--2.4/build/lib/python/OFS/ObjectManager.py", line 300,  
in _setObject

v = self._checkId(id)
  File "/home/buildbot/slave-zope.org/zc-buildbot--Linux--Zope--- 
branches---2.9--2.4/build/lib/python/OFS/ObjectManager.py", line 95,  
in checkValidId

raise BadRequest, (
BadRequest: The id "folder" is invalid - it is already in use.



On 12. Sep 2006, at 02:52, [EMAIL PROTECTED] wrote:

The Buildbot has detected a failed build of Zope branches 2.9 2.4  
Linux zc-buildbot.


Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 7595
Blamelist: adamg,ctheune,flindner,flox,jim,tseaver

BUILD FAILED: failed test

sincerely,
 -The Buildbot


--
It doesn't necessarily do it in chronological order,
though.  --Douglas Adams


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Before-abort hook?

2006-03-24 Thread Stefan H. Holek
Not an option in this case I am afraid. I am more after a general  
solution for transaction participants that need to manage external  
(to ZODB) resources. The fact that persistent attributes are  
destroyed before a registered TM gets a chance to do its work is  
unfortunate.


Stefan


On 24. Mär 2006, at 11:24, Tino Wildenhain wrote:

Do you remember, when you use the tempfile module, you get a  
filehandle

with a "nameless" file which automatically vanishes if you close the
handle. Maybe this solves the problem for you?


--
Anything that happens, happens.  --Douglas Adams


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] Before-abort hook?

2006-03-24 Thread Stefan H. Holek

Hi all,

The ExtFile product I maintain keeps its payload on the filesystem.  
All operations are performed using temporary files which are then  
saved or deleted on transaction commit or abort respectively.


To this end an ExtFile provides _finish and _abort methods which are  
hooked up to a TM. Since Zope 2.8 (MVCC) I have a problem with  
aborts. On abort the machinery apparently nukes an ExtFile's  
persistent attributes, and I lose all information I need to clean up  
the external resources (filename, path, etc).


What I need is a way to guarantee my cleanup code is called *before*  
the persistent part of my object vanishes. I currently manipulate the  
transaction's _resources list directly [1], but that's obviously not  
desirable. I have seen synchronizers, but I am unable to tell whether  
a beforeCompletion will end up to be a commit or abort.


We now have before-commit hooks, and it seems what I need is a before- 
abort hook as well.


Thoughts appreciated,

Stefan

[1] http://dev.plone.org/collective/browser/ExtFile/trunk/TM.py


--
Anything that happens, happens.  --Douglas Adams


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] Typo in ZODB.ZApplication of ZODB 3.4!

2005-05-06 Thread Stefan H. Holek
I am pretty sure this should say "isinstance(connection, basestring)", 
not "type".

def __call__(self, connection=None):
db, aname, version_support = self._stuff
if connection is None:
connection=db.open()
elif isinstance(type, basestring):
connection=db.open(connection)
return connection.root()[aname]
--
Software Engineering is Programming when you can't. --E. W. Dijkstra
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] Commit/abort before connection close required?

2005-05-03 Thread Stefan H. Holek
Hi,
I found that I now need to explicitly abort (or commit) the transaction 
before closing a connection, as otherwise the connection pool would not 
be freed?

Is this a recent change? I am pretty sure this was not always necessary 
but I may be wrong. Zope 2.7.6 here.

Thanks,
Stefan
--
Software Engineering is Programming when you can't. --E. W. Dijkstra
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev