subject:"\[Zope\-dev\] ZODB"

Re: [Zope-dev] [ZODB-Dev] when did transaction lose the ability to be usable as a context manager?

2013-02-06 Thread Arfrever Frehtes Taifersar Arahesis

2013-02-06 09:57:14 Chris Withers napisał(a):
> I used to do this:
> 
>  >>> import transaction
>  >>> with transaction:
> ...   print 'hello'
> ...
> Traceback (most recent call last):
>File "", line 1, in 
> AttributeError: __exit__
> 
> When did that stop working and what should I now do instead?

http://bugs.python.org/issue9220

-- 
Arfrever Frehtes Taifersar Arahesis


signature.asc
Description: This is a digitally signed message part.
___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] [ZODB-Dev] Progress report: porting persistent, BTrees to Python3

2012-12-17 Thread Lennart Regebro

On Mon, Dec 17, 2012 at 10:05 AM, Chris McDonough  wrote:
> On Sun, 2012-12-16 at 22:10 +0100, Godefroid Chapelle wrote:
>> Le 15/12/12 01:52, Tres Seaver a écrit :
>> > I fixed the remainig issues in persistent and released 4.0.5 today:  its
>> > tests properly exercise the C extensions Under Python 3.2 / 3.3.
>>
>> I want to express my thanks to you, Tres, for taking care of that work !
>>
>> This port of ZODB to Python 3 is really a crucial step for the ZTK
>> ecosystem. After the work already done on zope.interface and zope.component.
>>
>> Further, I'd like to also thank Jim for his work on porting buildout.
>>
>> When this will be finished, porting the rest of the ZTK should be much
>> easier, which hopefully implies that more of us will be able to
>> participate.
>
> Hear, hear!

+lots
___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] [ZODB-Dev] Progress report: porting persistent, BTrees to Python3

2012-12-17 Thread Chris McDonough

On Sun, 2012-12-16 at 22:10 +0100, Godefroid Chapelle wrote:
> Le 15/12/12 01:52, Tres Seaver a écrit :
> > I fixed the remainig issues in persistent and released 4.0.5 today:  its
> > tests properly exercise the C extensions Under Python 3.2 / 3.3.
> 
> I want to express my thanks to you, Tres, for taking care of that work !
> 
> This port of ZODB to Python 3 is really a crucial step for the ZTK 
> ecosystem. After the work already done on zope.interface and zope.component.
> 
> Further, I'd like to also thank Jim for his work on porting buildout.
> 
> When this will be finished, porting the rest of the ZTK should be much 
> easier, which hopefully implies that more of us will be able to 
> participate.

Hear, hear!

- C


___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] [ZODB-Dev] [announce] NEO 1.0 - scalable and redundant storage for ZODB

2012-08-29 Thread Jim Fulton

On Tue, Aug 28, 2012 at 12:31 PM, Vincent Pelletier  wrote:
...
> I forgot in the original mail to mention that NEO does all conflict
> resolutions on client side rather than server side. The same happens in
> relStorage, but this is different from ZEO.

That's good.  I'd like to move ZEO in this direction.  I'd also
like to stop hanging conflict-resolution on classes and have
some kind of registry, so that people can set CR policies
independent of class implementation.

I didn't realize that relstorage did client side CR, but thinking
about it, it has to work that way, since there's no relstorage
server.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
Jerky is better than bacon! http://zo.pe/Kqm
___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] [ZODB-Dev] [announce] NEO 1.0 - scalable and redundant storage for ZODB

2012-08-29 Thread Jim Fulton

On Wed, Aug 29, 2012 at 2:29 AM, Marius Gedminas  wrote:
> On Tue, Aug 28, 2012 at 06:31:05PM +0200, Vincent Pelletier wrote:
>> On Tue, 28 Aug 2012 16:31:20 +0200,
>> Martijn Pieters  wrote :
>> > Anything else different? Did you make any performance comparisons
>> > between RelStorage and NEO?
>>
>> I believe the main difference compared to all other ZODB Storage
>> implementation is the finer-grained locking scheme: in all storage
>> implementations I know, there is a database-level lock during the
>> entire second phase of 2PC, whereas in NEO transactions are serialised
>> only when they alter a common set of objects.
>
> This could be a compelling point.  I've seen deadlocks in an app that
> tried to use both ZEO and PostgreSQL via the Storm ORM.  (The thread
> holding the ZEO commit lock was blocked waiting for the PostgreSQL
> commit to finish, while the PostgreSQL server was waiting for some other
> transaction to either commit or abort -- and that other transaction
> couldn't proceed because it was waiting for the ZEO lock.)

This sounds like an application/transaction configuration problem.
To avoid this sort of deadlock, you need to always commit in a
a consistent order.  You also need to configure ZEO (or NEO)
to time-out transactions that take too long to finish the second phase.

I don't think NEO's locking strategy mitigates the deadlock problem
much, if at all.

The strategy should provide greater transaction throughput and
reduce latency.  It's a strategy I'd like to implement for ZEO at some
point.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
Jerky is better than bacon! http://zo.pe/Kqm
___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] [ZODB-Dev] [announce] NEO 1.0 - scalable and redundant storage for ZODB

2012-08-28 Thread Marius Gedminas

On Tue, Aug 28, 2012 at 06:31:05PM +0200, Vincent Pelletier wrote:
> On Tue, 28 Aug 2012 16:31:20 +0200,
> Martijn Pieters  wrote :
> > Anything else different? Did you make any performance comparisons
> > between RelStorage and NEO?
> 
> I believe the main difference compared to all other ZODB Storage
> implementation is the finer-grained locking scheme: in all storage
> implementations I know, there is a database-level lock during the
> entire second phase of 2PC, whereas in NEO transactions are serialised
> only when they alter a common set of objects.

This could be a compelling point.  I've seen deadlocks in an app that
tried to use both ZEO and PostgreSQL via the Storm ORM.  (The thread
holding the ZEO commit lock was blocked waiting for the PostgreSQL
commit to finish, while the PostgreSQL server was waiting for some other
transaction to either commit or abort -- and that other transaction
couldn't proceed because it was waiting for the ZEO lock.)

Marius Gedminas
-- 
People who think, "Oh this is a one-off," need to be offed, or perhaps politely
removed from the project.
-- George Neville-Neil

signature.asc
Description: Digital signature
___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] [ZODB-Dev] [announce] NEO 1.0 - scalable and redundant storage for ZODB

2012-08-28 Thread Vincent Pelletier

On Tue, 28 Aug 2012 16:31:20 +0200,
Martijn Pieters wrote :
> Anything else different? Did you make any performance comparisons
> between RelStorage and NEO?

I believe the main difference compared to all other ZODB Storage
implementation is the finer-grained locking scheme: in all storage
implementations I know, there is a database-level lock during the
entire second phase of 2PC, whereas in NEO transactions are serialised
only when they alter a common set of objects.
This removes an argument in favour of splitting databases (ie, using
mountpoints): evading the tpc_vote..tpc_finish database-level locking.

Also, NEO distributes objects over several servers (aka, some or all
servers might not contain the whole database), for load balancing/
parallelism purposes. This is not possible if one relies on relational
database replication alone.

I forgot in the original mail to mention that NEO does all conflict
resolutions on client side rather than server side. The same happens in
relStorage, but this is different from ZEO. Packing on client side
makes it easier to get the setup right: with ZEO you will get more
conflicts than normal if it cannot load some class which implements
conflict resolution, and this might go unnoticed until someone worries
about a performance drop or so. With client-side resolution, if you
don't see Broken Objects, conflict resolution for those classes works.

Some comments on some points you mentioned:
> * NEO supports MySQL and sqlite, RelStorage MySQL, PostgreSQL and
> Oracle.

It should be rather easy to adapt to more back-ends.
We (Nexedi) are not interested in proprietary software, so we will
probably not implement Oracle support ourselves. For PostgreSQL, it's
just that we do not have a setup at hand and the experience to
implement a client properly. I expect that it would not take more than a
week to get PostgreSQL implemented by someone used to it and knowing
python, but new to NEO.

Just to demonstrate that NEO really does not rely on fancy features of
SQL servers, you may dig in older revisions in NEO's git repository. You
can find a "btree.py"[1] test storage, which is based on ZODB.BTree
class. It was just a toy, without persistence support (I initially
intended to provide it, but never finished it) and hence limited by
the available amount of RAM. But it was otherwise a fully functional NEO
storage backend. I think it took me a week-end to put it together,
while discovering ZODB.Btree API and adapting NEO's storage backend
API along the way (this was the first non-MySQL backend ever
implemented, so API was a bit too ad-hoc at that time).

sqlite was chosen as a way to get rid of the need to setup a
stand-alone SQL server in addition to NEO storage process. We are not
sure yet of how well our database schema holds when there are several
(10+) GB of data in each storage node.

> * RelStorage can act as a BlobStorage, NEO can not.

I would like to stress that this has nothing to do with design, rather
it's just not implemented. We do not wish to rely on filesystem-level
sharing, so we consider something along the lines of providing a
FUSE-based to share blob storage, which then can abstract the blobs
being distributed over several servers. This is just the general idea,
we don't have much experience with blob handling ourselves (which is
why we preferred to leave it asides rather than providing an
unrealistic - and hence unusable - implementation).

[1]http://git.erp5.org/gitweb/neoppod.git/blob/75d83690bd4a34cfe5ed83c949e4a32c7dec7c82:/neo/storage/database/btree.py

Regards,
--
Vincent Pelletier
ERP5 - open source ERP/CRM for flexible enterprises
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )

1 2 >

1 - 100 of 128 matches

Mail list logo