Thanks Pradeep - none of our logs currently use Proto Buf/Thrift/Avro and
we were somewhat trying to stay away from these guys but they may be a good
option.
On Thu, Dec 12, 2013 at 6:35 PM, Pradeep Gollakota pradeep...@gmail.comwrote:
It seems like what you're asking for is Versioned Schema
We're playing around with options to what I'm sure is a common problem -
changing schemas in our log data.
Specifically we collect pixel data via nginx servers. These pixels
currently have a pretty static list of parameters in the query string. We
have eventual plans to change this and support
It seems like what you're asking for is Versioned Schema management. Pig is
not designed for that. Pig is only a scripting language to manipulate
datasets.
I'd recommend you look into Thrift, Protocol Buffers and Avro. They are
compact serialization libraries that do versioned schema management.
