Re: Best way to structure data for efficient searching

2012-04-03 Thread Roy Smith
> On 3/28/2012 11:39 AM, larry.mart...@gmail.com wrote: > > I have a set of data that is contains 3 fields, K1, K2 and a > > timestamp. There are duplicates in the data set, and they all have to > > processed. > > > > Then I have another set of data with 4 fields: K3, K4, K5, and a > > timestamp. T

Re: Best way to structure data for efficient searching

2012-04-03 Thread John Nagle
On 3/28/2012 11:39 AM, larry.mart...@gmail.com wrote: I have the following use case: I have a set of data that is contains 3 fields, K1, K2 and a timestamp. There are duplicates in the data set, and they all have to processed. Then I have another set of data with 4 fields: K3, K4, K5, and a tim

Re: Best way to structure data for efficient searching

2012-04-02 Thread Peter Otten
larry.mart...@gmail.com wrote: > I have the following use case: > > I have a set of data that is contains 3 fields, K1, K2 and a > timestamp. There are duplicates in the data set, and they all have to > processed. > > Then I have another set of data with 4 fields: K3, K4, K5, and a > timestamp.

Re: Best way to structure data for efficient searching

2012-04-02 Thread Asen Bozhilov
Larry.Mart wrote: > Since there are duplicates, I can't use a dict. And if I have any > extraneous data in the keys (i.e. something to make them unique) then > I still have to walk through the entire dict to find the matches. You can use slightly different approach. With double mapping you could

Re: Best way to structure data for efficient searching

2012-04-02 Thread Jon Clements
On Wednesday, 28 March 2012 19:39:54 UTC+1, larry@gmail.com wrote: > I have the following use case: > > I have a set of data that is contains 3 fields, K1, K2 and a > timestamp. There are duplicates in the data set, and they all have to > processed. > > Then I have another set of data with 4

Best way to structure data for efficient searching

2012-04-02 Thread larry.mart...@gmail.com
I have the following use case: I have a set of data that is contains 3 fields, K1, K2 and a timestamp. There are duplicates in the data set, and they all have to processed. Then I have another set of data with 4 fields: K3, K4, K5, and a timestamp. There are also duplicates in that data set, and

Re: Best way to structure data for efficient searching

2012-04-02 Thread larry.mart...@gmail.com
On Mar 28, 1:52 pm, Jon Clements wrote: > On Wednesday, 28 March 2012 19:39:54 UTC+1, larry@gmail.com  wrote: > > I have the following use case: > > > I have a set of data that is contains 3 fields, K1, K2 and a > > timestamp. There are duplicates in the data set, and they all have to > > proc