[
https://issues.apache.org/jira/browse/FLINK-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299769#comment-17299769
]
Jan Kriho edited comment on FLINK-20054 at 3/11/21, 5:54 PM:
-------------------------------------------------------------
{{Hi,}}
3 level lists with struct elements are still yielding errors even with the
patch, e. g.:
{{message spark_schema {}}
{{ required group foo (LIST) {}}
{{ repeated group list {}}
{{ required group element {}}
{{ optional int32 barId;}}
{{ optional int32 bazId;}}
{{ }}}
{{ }}}
{{ }}}
{{ {{}}}}}{{ }}
results in:
{{java.lang.UnsupportedOperationException: List field [optional int32 barId] in
List [foo] has to be required.}}
was (Author: erbureth):
{{Hi,}}
3 level lists with struct elements are still yielding errors even with the
patch, e. g.:
{{message spark_schema { }}
{{ required group foo (LIST) { }}
{{ repeated group list { }}
{{ required group element { }}
{{ optional int32 barId; }}
{{ optional int32 bazId; }}
{{ } }}
{{ } }}
{{ } }}
{{}}}{{}}
results in:
java.lang.UnsupportedOperationException: List field [optional int32 barId] in
List [foo] has to be required.
> 3 Level List is not supported in ParquetInputFormat
> ---------------------------------------------------
>
> Key: FLINK-20054
> URL: https://issues.apache.org/jira/browse/FLINK-20054
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
> Affects Versions: 1.11.2
> Reporter: Zhenqiu Huang
> Assignee: Zhenqiu Huang
> Priority: Minor
> Labels: pull-request-available
> Fix For: 1.13.0
>
>
> In flink-parquet module doesn't support
> reading a 3-level list type in parquet though it is able to process a
> 2-level list type.
> 3-level
> optional group my_list (LIST) {
> repeated group element {
> required binary str (UTF8);
> };
> }
> 2-level
> optional group my_list (LIST) {
> repeated int32 element;
> }
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
