Hi, I work in vertical search in Yahoo!, we would like to use for an application where it would be extremely beneficial to be able to extract single fields from binary Avro data without unpacking the entire datagram. This is hard to do efficiently, as far as I can tell. Avro supports skip information, but it is somewhat inefficient to skip across a block of an array, a record or a map, if any of these contain a variable length object. The headers only contain the number of objects contained, not the length in bytes.
Is the correct way to solve this kind of scenario to create a schema with extra meta data (which feels like kind of a kludge), or is there some other way? Hopefully, I have overlooked something. Regards, -- Steinar Knutsen <[email protected]> Yahoo! Technologies Norway AS
