you can use an approach like HADDOP's TFile https://issues.apache.org/jira/secure/attachment/12396286/TFile%20Specification%2020081217.pdf
basically they store compressed chunks in a file, and at the end of the file (hence T(ail)File) they store come kind of a sparse index to the chunks. Eyal. > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To view this discussion on the web visit https://groups.google.com/d/msg/protobuf/-/3AKfAtEVbkgJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
