Hi Wei,
It looks like you are using a writer with one schema to write records
created with another. That doesn't work because Avro deconstructs
generic records by position. Is there a way you could change your code
so that you use the final write schema as the read schema for the file
where you data comes from? You could also construct the records using
the final schema, too. You just have to make sure that schema matches.
What are you trying to do?
rb
On 04/02/2015 05:46 PM, Julien Le Dem wrote:
Maybe Ryan or Tom can help
On Wed, Mar 18, 2015 at 4:59 PM, Wei Yan <ywsk...@gmail.com
<mailto:ywsk...@gmail.com>> wrote:
Hi, devs,
I’m a newbie of using parquet. I met a ParquetWriter problem and
wish anyone can help me.
I use ParquetWriter to write a GenericRecord to a file. And the
schema used to define the ParquetWriter has fewer fields than the
GenericRecord. e.g., The schema for the ParquetWriter:
{"type":"record","name":"r","fields":[{"name":"f1","type":"double","default":0}]}
, which only has one field "f1”. And the GenericRecord has two
fields: {"f2": null, "“"f1": 1.0}.
When I use that ParquetWriter to write that record, I thought it
would only write field “f1” and skip “f2”. However, I got this
exception “Null-value for required field: f1”. It looks like the
ParquetWriter considered the field sequence, and tried to match the
“f2” in the record to the “f1” to the schema. Is this by design?
Very appreciate for any help.
thanks,
Wei
--
Ryan Blue
Software Engineer
Cloudera, Inc.
