gszadovszky commented on PR #3272: URL: https://github.com/apache/parquet-java/pull/3272#issuecomment-3249354369
Thanks @gauravkhatri05 for the detailed clarification. So, we are trying to translate a recursive schema to a type system where recursive schemas are not supported. By using a max depth you translate the recursive Avro schema to a Parquet schema where the recursive types are simply repeated again. So the Avro schema A -> B -> C -> D -> A gets translated (with the default 10 max depth) to A -> B -> C -> D -> A -> B -> C -> D -> A -> B. What happens with the data having more depth? Do we fail or silently ignore the rest? Neither seems to be the best approach for systems where no issues are expected after a proper schema conversion. It seems this topic has some history: [PARQUET-129](https://issues.apache.org/jira/browse/PARQUET-129) Maybe bring up this topic on the dev list. Let's see what the community thinks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
