Optional is the primary mechanism for adding and removing fields. Any field that wasn't in the first version of ORC 9 years ago has to be optional for the old readers to be able to read the file. *smile*
The general guidance for designing protobufs is that almost everything should be optional. To be required, you have to convince yourself that there is no circumstance where the record makes sense without the value. In hindsight, the footer length probably qualifies. I do understand your concern, but if you handle it the other direction it will be easier. What are you trying to accomplish? Which fields do you need to accomplish that? Are you trying to explore ORC files to display them? .. Owen On Fri, Nov 4, 2022 at 6:01 AM deshan xiao <deshanx...@gmail.com> wrote: > Hi Richard, > You can find detailed protocol information here: > https://github.com/apache/orc/tree/main/proto > > As you can see, many fields are optional. These restrictions you mentioned > are mainly added by the specific ORC Reader implementation. You know > Apache ORC > is the official standard implementation of ORC Reader/Writer but not the > only one. > Apache ORC will add some other restrictions on top of the protocol. > This problem does not exist if you use Apache ORC Writer to write sn orc > file. > > By the way, could you provide more detailed information or exception stack > for your use case? > > On 2022/11/01 09:30:36 Richard Evans wrote: > > Hello > > > > Reading the message specifications in the ORC file format > > (https://orc.apache.org/specification/ORCv1/) I see every field is > > marked optional. Yet some of the fields are always required (such as > > the footerLength in the file Postscript). > > > > So are all fields always present? Or is there any way to tell which > > fields are truly optional and which are always required? > > > > Thanks > > > > Richard > > > > >
