Re: Making searches inside a message file

2009-06-22 Thread Kenton Varda
Sorry, Protocol Buffers do not provide support for searching or indexing.
 All protobuf does is parse and serialize data structures.  Drizzle is
actually building database functionality *on top* of protocol buffers --
they implement their own searching and indexing, using protocol buffers to
implement the base serialization.

On Fri, Jun 19, 2009 at 11:25 PM, green enmanu...@gmail.com wrote:


 I am interested in using protocol buffers for the following scenario:

 I have a large number of small data points I need to serialize to a
 file. Once this serialization process is complete, the data will be
 strictly read only, accessed by multiple threads, until I need to
 create a new version of the file. The application will access the file
 at startup, and while executing will need to retrieve data points
 based on a given parameter. Pretty much behaves like tabular data you
 would find in a database table. I need to retrieve a row, based on a
 primary key. Yet a traditional database is not a good solution in
 this situation.

 My understanding is that it's not a good idea to have large messages,
 but you can have a large number of small messages within a file, and
 it shouldn't be a problem. I have seen the .proto file structure and
 how messages are determined. What I don't understand (maybe I missed
 that part in the documentation) is how to make searches for messages
 within a file. If I use the repeated qualifier, it will let me have
 more than one message in another one, but I'll just retrieve it as a
 list. I can't specify what subset of messages I want.

 Can I sort messages based on a given field?
 Can I request a subset of messages by index range, or some other
 criteria?
 Can I browse through a message file, given a particular search
 parameter?
 Can I have some sort of Map inside the .proto definition, where I
 organize elements in key - value fashion?

 Alternatively, are my assumptions of what I should be able to do with
 protocol buffers wrong from the start? I assumed this kind of thing
 was possible since the drizzle devs are using protocol buffers for
 their database implementation. Link below.

 http://drizzle.org/wiki/Table_Proto_Definition

 Thanks in advance.
 


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---



Making searches inside a message file

2009-06-20 Thread green

I am interested in using protocol buffers for the following scenario:

I have a large number of small data points I need to serialize to a
file. Once this serialization process is complete, the data will be
strictly read only, accessed by multiple threads, until I need to
create a new version of the file. The application will access the file
at startup, and while executing will need to retrieve data points
based on a given parameter. Pretty much behaves like tabular data you
would find in a database table. I need to retrieve a row, based on a
primary key. Yet a traditional database is not a good solution in
this situation.

My understanding is that it's not a good idea to have large messages,
but you can have a large number of small messages within a file, and
it shouldn't be a problem. I have seen the .proto file structure and
how messages are determined. What I don't understand (maybe I missed
that part in the documentation) is how to make searches for messages
within a file. If I use the repeated qualifier, it will let me have
more than one message in another one, but I'll just retrieve it as a
list. I can't specify what subset of messages I want.

Can I sort messages based on a given field?
Can I request a subset of messages by index range, or some other
criteria?
Can I browse through a message file, given a particular search
parameter?
Can I have some sort of Map inside the .proto definition, where I
organize elements in key - value fashion?

Alternatively, are my assumptions of what I should be able to do with
protocol buffers wrong from the start? I assumed this kind of thing
was possible since the drizzle devs are using protocol buffers for
their database implementation. Link below.

http://drizzle.org/wiki/Table_Proto_Definition

Thanks in advance.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---