Two options: * DIfferent files per schema * One schema that is a union of all schemas you want in the file
Which is best depends on your use case. On 6/1/11 4:02 PM, "Yang" <[email protected]<mailto:[email protected]>> wrote: our use case is that we have many different types of "events", with different schemas. I was thinking to dump them into one file, for easier maintenance of the files. but then I found that all the DataFileWriter, JsonEncoder/Decoder require a schema to be present, so each file can have really only one schema. of course I can create a separate encoder/writer for each record I write. but then there would be no way to parse out the file later. such a mixed schema file can be useful only to humans at best. so generally what is your experience in dealing with serializing objects of different types? do you put them in different files? Thanks Yang
