Re: Sample large datastore

2008-05-20 Thread Tomeu Vizoso
On Tue, May 20, 2008 at 3:11 AM, Martin Langhoff <[EMAIL PROTECTED]> wrote: > On Mon, May 19, 2008 at 9:26 PM, Tomeu Vizoso <[EMAIL PROTECTED]> wrote: >> On Mon, May 19, 2008 at 10:30 AM, Tomeu Vizoso <[EMAIL PROTECTED]> wrote: >>> >>>results, count = data_store.find({}, ['uid']) >> >> You may

Re: Sample large datastore

2008-05-19 Thread Martin Langhoff
On Mon, May 19, 2008 at 9:26 PM, Tomeu Vizoso <[EMAIL PROTECTED]> wrote: > On Mon, May 19, 2008 at 10:30 AM, Tomeu Vizoso <[EMAIL PROTECTED]> wrote: >> >>results, count = data_store.find({}, ['uid']) > > You may want to limit the amount of results returned by > > results, count = data_store.fin

Re: Sample large datastore

2008-05-19 Thread Tomeu Vizoso
On Mon, May 19, 2008 at 10:30 AM, Tomeu Vizoso <[EMAIL PROTECTED]> wrote: > >results, count = data_store.find({}, ['uid']) You may want to limit the amount of results returned by results, count = data_store.find({'limit': 20}, ['uid']) as trying to get too much info from the DS in one go can

Re: Sample large datastore

2008-05-19 Thread Tomeu Vizoso
On Mon, May 19, 2008 at 9:55 AM, Martin Langhoff <[EMAIL PROTECTED]> wrote: > n Mon, May 19, 2008 at 7:20 PM, Tomeu Vizoso <[EMAIL PROTECTED]> wrote: >> Do you think this would be enough? > > That'll be *perfect* for this :-) Ok, then something like this should do the job: --- import sys import

Re: Sample large datastore

2008-05-19 Thread Martin Langhoff
n Mon, May 19, 2008 at 7:20 PM, Tomeu Vizoso <[EMAIL PROTECTED]> wrote: > Do you think this would be enough? That'll be *perfect* for this :-) cheers, m -- [EMAIL PROTECTED] [EMAIL PROTECTED] -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff -

Re: Sample large datastore

2008-05-19 Thread Tomeu Vizoso
On Mon, May 19, 2008 at 7:01 AM, Martin Langhoff <[EMAIL PROTECTED]> wrote: > Working on ds-backup, I am concerned about performance of extracting > all of the datastore metadata. Both Tomeu and Ivan have warned me > about performance and memory impact. I want to repro the problem asap. > > If a fu

Sample large datastore

2008-05-18 Thread Martin Langhoff
Working on ds-backup, I am concerned about performance of extracting all of the datastore metadata. Both Tomeu and Ivan have warned me about performance and memory impact. I want to repro the problem asap. If a full metadata dump can be acceptably fast, a lot of complications and error conditions