Artem created FLINK-34620:
-----------------------------
Summary: Process recursive protobuf schemas in PbToRowTypeUtil
Key: FLINK-34620
URL: https://issues.apache.org/jira/browse/FLINK-34620
Project: Flink
Issue Type: Improvement
Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
Reporter: Artem
The current version of _PbToRowTypeUtil.generateRowType_ doesn't support
recursive protobuf schemas like
message RecursiveMessage \{
int32 id = 1;
RecursiveMessage message = 2;
}
It throws _java.lang.StackOverflowError._
This could be implemented by trimming the recursion. Same approach is used in
Apache Spark, for example.
([link|[https://github.com/apache/spark/blob/master/connector/protobuf/src/main/scala/org/apache/spark/sql/protobuf/utils/SchemaConverters.scala#L178])]
then output Table schemas for the above proto-schema will be like this:
recursiveFieldMaxDepth=0: message: ROW< id: INTEGER >
recursiveFieldMaxDepth=1: message: ROW< id: INTEGER, message: ROW< id: INTEGER
> > recursiveFieldMaxDepth=2: message: ROW< id: INTEGER, message: ROW< id:
INTEGER, message: ROW< id: INTEGER > > >
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
