If you define you records like this (this is in the Avro IDL lang. for brevity)
If you write your records with a schema like this:
record R {
Header header;
Body body;
}
Then you can read with a schema like this:
record RSansBody {
Header header;
}
And the Avro libraries will read the header part (in which your "type" would
reside) and effectively skip the body part.
________________________________
From: Anna Lahoud <[email protected]>
Sent: Friday, August 16, 2013 12:23 PM
To: [email protected]
Subject: Is there a way to conditionally read Avro data?
I am wondering if there is a way that I can avoid reading all of an item in an
Avro file, based on some of the data that I have already read. For instance,
say I have a datum where I know that if it's 'type' value is a 'ComputerVirus',
and that I do not want to touch the remaining fields. Is there a way to 'move
on' and get the next datum, without touching the remainder of the scary datum?
I would call it a 'conditional read' in that I only want to fully read the
datum if the datum meets some criteria.
Anna
