Re: [ZODB-Dev] Indexing and dates/times

2010-07-13 Thread Jim Fulton
On Tue, Jul 13, 2010 at 4:35 AM, Pedro Ferreira jose.pedro.ferre...@cern.ch wrote: Hello, I am currently trying to devise a way to index and retrieve some millions of objects according to their modification date/time. One of the problems I'm facing is that of index granularity: I'd like to

Re: [ZODB-Dev] Indexing and dates/times

2010-07-13 Thread Benji York
On Tue, Jul 13, 2010 at 4:35 AM, Pedro Ferreira jose.pedro.ferre...@cern.ch wrote: Hello, I am currently trying to devise a way to index and retrieve some millions of objects according to their modification date/time. So, in a relational DB i would do something like: SELECT * FROM table

Re: [ZODB-Dev] Indexing and dates/times

2010-07-13 Thread Pedro Ferreira
So, the issue is that you have multiple items with the same key. This is simply handled by using sets as values ion a BTree. There are existing index implementations that do this. Hmm... no, in fact the problem is that most of the time I will have only one value per index entry.So, in a

Re: [ZODB-Dev] Indexing and dates/times

2010-07-13 Thread Pedro Ferreira
If you use the timestamp as the key and you want to retrieve all values between two timestamps (inclusive), you can do my_btree.values(min=start, max=end) Yes, but, as I mentioned in my answer to Jim's mail, my concern is the performance of this range operation for a very large

Re: [ZODB-Dev] Indexing and dates/times

2010-07-13 Thread Jim Fulton
On Tue, Jul 13, 2010 at 7:51 AM, Pedro Ferreira jose.pedro.ferre...@cern.ch wrote: ... Hmm... no, in fact the problem is that most of the time I will have only one value per index entry.So, in a relational DB i would do something like: As I mentioned in some text you didn't quote, you could use

[ZODB-Dev] Indexing and dates/times

2010-07-12 Thread Pedro Ferreira
Hello all, I am currently trying to devise a way to index and retrieve some millions of objects according to their modification date/time. One of the problems I'm facing is that of index granularity: I'd like to provide to the second granularity, but for that I need some structure that lets

Re: [ZODB-Dev] Indexing and dates/times

2010-07-12 Thread Jim Fulton
On Mon, Jul 12, 2010 at 11:47 AM, Pedro Ferreira jose.pedro.ferre...@cern.ch wrote: Hello all, I am currently trying to devise a way to index and retrieve some millions of objects according to their modification date/time. One of the problems I'm facing is that of index granularity: I'd like

Re: [ZODB-Dev] indexing and querying objects in the ZODB

2009-02-13 Thread Jim Fulton
On Feb 13, 2009, at 1:54 AM, Sebastian Wehrmann wrote: Hi, I finished my diploma thesis about indexing and querying objects [1] (only available in german - sorry) last year. With that thesis I developed some piece of software, which can be used to query for objects within the ZODB

Re: [ZODB-Dev] indexing and querying objects in the ZODB

2009-02-13 Thread Sebastian Wehrmann
Hi, On Friday, Feb 13, at 9:30am +0100, Jim Fulton wrote: Have you done any scale experiments? Indeed, I've done tests with some binary trees with a heigth from 2 till 20. The results pointed out, that filling the index structures required very much more time than the query itself (which

[ZODB-Dev] indexing and querying objects in the ZODB

2009-02-12 Thread Sebastian Wehrmann
Hi, I finished my diploma thesis about indexing and querying objects [1] (only available in german - sorry) last year. With that thesis I developed some piece of software, which can be used to query for objects within the ZODB with a XPath like syntax (its called regular path

Re: [ZODB-Dev] indexing

2008-07-24 Thread Tim Cook
On Wed, 2008-07-23 at 11:41 -0400, [EMAIL PROTECTED] wrote: I've found the ZODB website to be very disorganized an not nearly as helpful as repeated googlings. I have been collecting links and writing some new text for the new ZODB website. But alas there are only so many hours in the day.

Re: [ZODB-Dev] indexing

2008-07-23 Thread AFoglia
Sean Allen [EMAIL PROTECTED] wrote on 07/22/2008 11:44:11 PM: googling and the main zodb website seem to take me to a bunch of what appears to be severely dated websites, most not updated in at least 2 years ( some upwards of 6 ). this is probably because i simply don't know what to google

[ZODB-Dev] indexing

2008-07-22 Thread Sean Allen
googling and the main zodb website seem to take me to a bunch of what appears to be severely dated websites, most not updated in at least 2 years ( some upwards of 6 ). this is probably because i simply don't know what to google for, so, could someone be so kind as to shoot some links to

[ZODB-Dev] indexing object in ZODB

2007-03-01 Thread shahrzad khorrami
Hi all, What is the algorithm of indexing objects in ZODB? Is there any place where I can see the structure of ZODB ? Thanks alot ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list -

RE: [ZODB-Dev] Indexing: Query Optimization

2005-05-03 Thread Tim Peters
[Thomas Guettler] I developed a simple index using ZODB. Searching for single values is very fast. Searching big ranges is slow. Example: Search: customer_id=0815 date_start=2001-01-01 date_end=2004-12-31 The index holds a btree which maps values to docids. The search for customer_id is