Re: [ZODB-Dev] RelStorage - high ConflictError rate in newly converted DB

2011-11-10 Thread Darryl Dixon - Winterhouse Consulting 
Hi Shane,

> On 11/07/2011 03:15 PM, Darryl Dixon - Winterhouse Consulting wrote:
>> Hi All,
>>
>> We have just converted a site with a handful of editors over from ZEO to
>> RelStorage on Oracle and and the rate of ConflictErrors has gone through
>> the roof (5 an hour average continuously for days).
[...snip...]
>> Currently we are using a very basic
>> RelStorage configuration without any additional bits like memcache.
>>
> I have a few thoughts:
>
> - You aren't using poll-interval, are you? Make sure that is disabled.
>
> - When you switched from ZEO to Oracle without turning on memcache, you
> lost the ZEO cache and didn't replace it with anything, so now loading
> objects takes longer, which may lead to more conflicts. I would try
> turning on memcache.
>
> - Do you have multiple storages / databases? Make sure each uses a
> different Oracle lock ID.

We are not using the poll-interval, so clear on that one. I wasn't clear
that memcache would assist in this situation - I will see about enabling
it in this environment, but there are a few questions I need a better
understanding for with this around what exactly are the characteristics of
memcache integration:

* What is the situation with using multiple memcache instances (eg, one
per machine - will this lead to cache incoherency - inconsistency between
instances, etc?

* What is the failure scenario for a memcache instance dying - is it
handled gracefully by RelStorage or will there be
hanging/blocking/tracebacks until it returns?

* Will this potentially exacerbate the ancient ZEO-Client-Race condition
that occurs with load-balancing when data changes? (To be clear, I am
referring to when one Zope instance updates/commits some data, and then
the client browser makes a new request which is directed to a second Zope
instance, which has not yet received an invalidation message for the
changed objects, and it returns, eg, a 404 to the client).

Any guidance on the above once again very gratefully received.

With regards the the commit lock - yes, we use multiple storages and each
uses a different commit-lock-id.

Thanks heaps for your help,

regards,
Darryl Dixon
Winterhouse Consulting Ltd
http://www.winterhouseconsulting.com
___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] Relstorage PostgreSQL and blobs garbage collection

2011-11-10 Thread Shane Hathaway 
On 11/10/2011 08:56 AM, Александр Неганов wrote:
> We had large history-free database with lot of "deleted" Blobs, so we
> packed and gc-ed it. Many rows from object_state as well as
> corresponding blob_chunk rows has been deleted, but postgresql large
> objects stays untouched. It seems that blob_chunk_delete_trigger is
> not executed in this case, but works fine when i manually delete rows
> from blob_chunk table. PostgreSQL problem? PostgreSQL server version
> is 9.0.4, RelStorage version is 1.5.0.

Hmm, that trigger is essential for deleting the large objects. There is 
probably some special case in Postgres that RelStorage is accidentally 
hitting when it deletes the rows. It has always worked for me.

Shane
___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev

[ZODB-Dev] Relstorage PostgreSQL and blobs garbage collection

2011-11-10 Thread Александр Неганов 
Hello.

We had large history-free database with lot of "deleted" Blobs, so we
packed and gc-ed it. Many rows from object_state as well as
corresponding blob_chunk rows has been deleted, but postgresql large
objects stays untouched. It seems that blob_chunk_delete_trigger is
not executed in this case, but works fine when i manually delete rows
from blob_chunk table. PostgreSQL problem? PostgreSQL server version
is 9.0.4, RelStorage version is 1.5.0.

-- 
Alexander.
___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev