Re: Encrypting only some of the fields.

2015-08-20 Thread Ryan Blue
Hi Niels, I think your assessment of the situation makes sense. Using the compression hooks would encrypt entire Avro blocks, so you need a way to change individual fields. I think it may make sense for you to take advantage of the recently-added logical types. That allows you a hook for

Encrypting only some of the fields.

2015-08-19 Thread Niels Basjes
Hi, I'm working on a project where I plan to put clickstream data into Kafka serialized using AVRO. In a later stage I want these records persisted into AVRO files so they can be used by people using PIG. So far this is no problem at all. Now some of those fields (not all) are privacy sensitive