tscottcoombes1 commented on PR #206: URL: https://github.com/apache/arrow-go/pull/206#issuecomment-2501342379
@zeroshade can I get your thoughts on this? spark follows this pattern: https://spark.apache.org/docs/latest/sql-data-sources-protobuf.html#handling-circular-references-protobuf-fields ``` syntax = "proto3" message Person { string name = 1; Person bff = 2 } // The protobuf schema defined above, would be converted into a Spark SQL columns with the following // structure based on `recursive.fields.max.depth` value. 0: struct<name: string, bff: null> 1: struct<name string, bff: <name: string, bff: null>> 2: struct<name string, bff: <name: string, bff: struct<name: string, bff: null>>> ... ``` but parquet can't do null types my idea, is if we get to depth = x, then we drop the field? e.g. for 2 ``` 2: struct<name string, bff: <name: string, bff: struct<name: string>>> ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
