joellubi commented on issue #43443: URL: https://github.com/apache/arrow/issues/43443#issuecomment-2254099411
Thanks for opening this. It indeed is the case that the schema is logically a required parameter; the IPC stream must have a consistent schema. I agree that having it as an optional parameter may be misleading. The reason for this is because it passes through the general `ipc.Option`'s that are shared across the different IPC readers/writers. Often the schema is _optional_ because it can be inferred otherwise, such as from the footer of an IPC file in the case of reading. In that specific case the schema is not required, but if it is provided a check is performed to ensure it matches the footer schema. I think a similar approach could make sense here in the case of the `flight.Writer`: - If no schema is provided in the constructor, set it for the whole stream based off the first record written. All subsequent records must match this schema. - If a schema is provided, check the first record's schema matches and proceed as before. @zeroshade Do these semantics seem reasonable? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
