Hi Pedro
> Betreff: Re: AW: [ZODB-Dev] Data.fs size grows non-stop
>
>
> > If you need performance, take a look at the z3c.indexer
> package which
> > offers a much faster indexing concept. I implemented this package
> > because my app coudn't h
On Thu, Dec 10, 2009 at 4:06 AM, Pedro Ferreira
wrote:
> Jim Fulton wrote:
>>
>> On Wed, Dec 9, 2009 at 10:54 AM, Pedro Ferreira
>> wrote:
>> ...
>>
>>>
>>> Well, at least we won't have to rewrite the whole bucket... but still, it
>>> would be much nicer to fragment the list
>>> in smaller chunks
Hello,
> Are the values in your BTree implemented as persistent lists or some
> other subclass of Persistent? That makes a big difference in the size
> of each commit to the BTree.
>
Yes, that is exactly the problem. We are using simple Python lists, and
that implies rewriting the whole bucket
> If you need performance, take a look at the z3c.indexer package
> which offers a much faster indexing concept. I implemented
> this package because my app coudn't handle the implicit default
> indexing handling. The README.txt explains what is different and how
> to use it. It defently needs so
Hi Pedro
> Betreff: Re: [ZODB-Dev] Data.fs size grows non-stop
>
>
> > Why not store the events on some BTree or similar structure with a
> > simple unique integer id, and use a separate BTree to map dates to
> > these events? You can even use zc.catalog or
On 2009-12-10 10:48, Pedro Ferreira wrote:
>
>> Why not store the events on some BTree or similar structure with a
>> simple unique integer id, and use a separate BTree to map dates to
>> these events? You can even use zc.catalog or similar tools to create
>> that mapping - you would get a lot of q
> Why not store the events on some BTree or similar structure with a
> simple unique integer id, and use a separate BTree to map dates to these
> events? You can even use zc.catalog or similar tools to create that
> mapping - you would get a lot of query options for free with those as well.
>
>
Thanks!
But I still get an error:
pferreir
/usr/lib/python2.6/site-packages/ZODB3-3.9.0-py2.6-linux-i686.egg$
zodbbrowser ~/indico/db/last.fs
Traceback (most recent call last):
File "/usr/bin/zodbbrowser", line 8, in
load_entry_point('zodbbrowser==0.6.1', 'console_scripts',
'zodbbrowser'
On 2009-12-10 10:06, Pedro Ferreira wrote:
> The situation is simple: we have events (objects) that have to be
> indexed by date. We currently use an OOBTree with the days as keys and
> lists of objects as values. The objective is to avoid rewriting these
> lists each time something changes, since
Jim Fulton wrote:
> On Wed, Dec 9, 2009 at 10:54 AM, Pedro Ferreira
> wrote:
> ...
>
>> Well, at least we won't have to rewrite the whole bucket... but still, it
>> would be much nicer to fragment the list
>> in smaller chunks. We could use an OOBTree instead... but something less
>> complex wo
On Wed, Dec 09, 2009 at 06:09:36PM +0200, Marius Gedminas wrote:
> On Wed, Dec 09, 2009 at 02:42:58PM +0100, Pedro Ferreira wrote:
> > Hello,
> > > Just zodbbrowser with no prefix:
> > >
> > > http://pypi.python.org/pypi/zodbbrowser
> > > https://launchpad.net/zodbbrowser
> > >
> > > It's a web
On Wed, Dec 9, 2009 at 10:54 AM, Pedro Ferreira
wrote:
...
> Well, at least we won't have to rewrite the whole bucket... but still, it
> would be much nicer to fragment the list
> in smaller chunks. We could use an OOBTree instead... but something less
> complex would suffice... any suggestions?
On 12/09/2009 04:58 PM, Pedro Ferreira wrote:
>
>> OOBTrees are complex?
>>
> No, I meant something list-like, rather than key-value.
zc.blist implements a BTree-based list.
--
Christian Theune · c...@gocept.com
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocep
On Wed, Dec 09, 2009 at 02:42:58PM +0100, Pedro Ferreira wrote:
> Hello,
> > Just zodbbrowser with no prefix:
> >
> > http://pypi.python.org/pypi/zodbbrowser
> > https://launchpad.net/zodbbrowser
> >
> > It's a web-app: it can connect to your ZEO server so you can inspect the
> > DB while it's
> OOBTrees are complex?
>
No, I meant something list-like, rather than key-value.
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
https://mail.zope.org/mailman/listin
Am 09.12.09 16:54, schrieb Pedro Ferreira:
> We could use an OOBTree instead... but something less
> complex would suffice... any suggestions?
OOBTrees are complex?
-aj
<>___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/
Hello,
>
> In the case of BTrees, yes. I assume your OOBuckets are used within
> OOBTrees. (?)
>
>
Yes. I was not sure whether OOBTrees behaved exactly like a BTree.
>> So, since we are adding very large
>> lists as values, the tree remains unbalanced,
>>
>
> No, they trees tend to stay fa
On Wed, Dec 9, 2009 at 8:42 AM, Pedro Ferreira
wrote:
...
> We've modified
> Jim's script
Cool. Storage iterators are so simple and allow a wide variety of analyses.
> in order to find out which OIDs are being rewritten, and
> how much space they are taking, and this is a fragment of it:
>
> OID
2009/12/9 Pedro Ferreira :
> Hello,
>> Just zodbbrowser with no prefix:
>>
>> http://pypi.python.org/pypi/zodbbrowser
>> https://launchpad.net/zodbbrowser
>>
>> It's a web-app: it can connect to your ZEO server so you can inspect the
>> DB while it's being used.
>>
> We tried this, but we curre
Hello,
> Just zodbbrowser with no prefix:
>
> http://pypi.python.org/pypi/zodbbrowser
> https://launchpad.net/zodbbrowser
>
> It's a web-app: it can connect to your ZEO server so you can inspect the
> DB while it's being used.
>
We tried this, but we currently get an error related with the g
On Mon, Dec 07, 2009 at 10:58:20PM +0100, Roger wrote:
> what I whould todo as next,
>
> try z3c.zodbbrowser or the other ZODB introspector
> tool from mgedmin, don't remember the name of the tool.
Just zodbbrowser with no prefix:
http://pypi.python.org/pypi/zodbbrowser
https://launchpad.ne
Jose Benito Gonzalez Lopez wrote:
> Dear ZODB developers,
>
> Since some time ago (not sure since when) our database
> has passed from 15GB to 65GB so fast, and it keeps growing
> little by little (2 to 5 GB per day). It is clear that something is not
> correct in it.
>
> We would like to check w
> Have you tried packing the database?
>
Pack has just finished: 17GB.
Still a lot, some months ago was 6GB
> I recommend buying a bigger disk -- seriously, before you run out of space. :)
>
yes, but by the time the bigger disk arrives...we will be done :)
BTW Jim, your class_stats has
i would think it would be more useful immediately to account for the rate of
growth rather than trying to account for all objects in the zodb. i've
attached a quick script you can use to get some introspection on the last
few transactions of a large data.fs file. it operates in read only mode,
just
Hi Jose
> Betreff: Re: [ZODB-Dev] Data.fs size grows non-stop
>
> Hi,
>
> Actually, we have two different versions (current and beta)
> of the software, working against the same ZODB. We are just
> developing new features on top of the beta version.
> Fearing that it
On Mon, Dec 7, 2009 at 3:08 PM, Jose Benito Gonzalez Lopez
wrote:
> Hi,
>
> Actually, we have two different versions (current and beta) of the software,
> working against the same ZODB. We are just developing new features
> on top of the beta version.
> Fearing that it was due to some changes don
Hi,
Actually, we have two different versions (current and beta) of the software,
working against the same ZODB. We are just developing new features
on top of the beta version.
Fearing that it was due to some changes done by a programmer
we have disabled the "beta version" but the growing of the D
On 2009-12-7 17:34, Jim Fulton wrote:
> On Mon, Dec 7, 2009 at 11:00 AM, Alan Runyan wrote:
> ...
>> A design pattern for RDBMS is to have 2 pools. READ pool and WRITE pool.
>> Often the READ pool comes from some replica and WRITE is to the master.
>> I'm unsure this pattern would work for ZODB.
On 12/07/2009 05:00 PM, Alan Runyan wrote:
>> I'd just like to add that there's some changes that can be related to this:
>>
>> - we had some classes inheriting from Persistent that now inherit from
>> something else as well (but no extra arguments are being added, AFAIK);
>> - we added some zope.i
On Mon, Dec 7, 2009 at 11:00 AM, Alan Runyan wrote:
...
> A design pattern for RDBMS is to have 2 pools. READ pool and WRITE pool.
> Often the READ pool comes from some replica and WRITE is to the master.
> I'm unsure this pattern would work for ZODB. I know Malthe was thinking
> about this but u
> I'd just like to add that there's some changes that can be related to this:
>
> - we had some classes inheriting from Persistent that now inherit from
> something else as well (but no extra arguments are being added, AFAIK);
> - we added some zope.interface definitions to some Persistent classes;
Hello all,
> Since some time ago (not sure since when) our database
> has passed from 15GB to 65GB so fast, and it keeps growing
> little by little (2 to 5 GB per day). It is clear that something is not
> correct in it.
>
I'd just like to add that there's some changes that can be related to this
On Mon, Dec 7, 2009 at 8:31 AM, Jose Benito Gonzalez Lopez
wrote:
> Since some time ago (not sure since when) our database
> has passed from 15GB to 65GB so fast, and it keeps growing
> little by little (2 to 5 GB per day). It is clear that something is not
> correct in it.
Are you packing it reg
2009/12/7 Jose Benito Gonzalez Lopez :
> Dear ZODB developers,
>
> Since some time ago (not sure since when) our database
> has passed from 15GB to 65GB so fast, and it keeps growing
> little by little (2 to 5 GB per day). It is clear that something is not
> correct in it.
>
> We would like to chec
Dear ZODB developers,
Since some time ago (not sure since when) our database
has passed from 15GB to 65GB so fast, and it keeps growing
little by little (2 to 5 GB per day). It is clear that something is not
correct in it.
We would like to check which objects are taking most of the space
or just
35 matches
Mail list logo
