[
https://issues.apache.org/jira/browse/SPARK-45334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean R. Owen resolved SPARK-45334.
----------------------------------
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 43119
[https://github.com/apache/spark/pull/43119]
> Remove misleading comment in parquetSchemaConverter
> ---------------------------------------------------
>
> Key: SPARK-45334
> URL: https://issues.apache.org/jira/browse/SPARK-45334
> Project: Spark
> Issue Type: Documentation
> Components: SQL
> Affects Versions: 3.5.0
> Reporter: Mengran Lan
> Assignee: Mengran Lan
> Priority: Trivial
> Labels: pull-request-available
> Fix For: 4.0.0
>
>
> I'm debugging a parquet issue and reading spark code as references. Happened
> to find a misleading comment which remains in the latest version as well.
> {code:java}
> Types
> .buildGroup(repetition).as(LogicalTypeAnnotation.listType())
> .addField(Types
> .buildGroup(REPEATED)
> // "array" is the name chosen by parquet-hive (1.7.0 and prior version)
> .addField(convertField(StructField("array", elementType, nullable)))
> .named("bag"))
> .named(field.name) {code}
> the comment above is misleading since Hive always uses "array_element" as the
> name.
> It is imported by this PR [https://github.com/apache/spark/pull/14399] and
> relates to this issue https://issues.apache.org/jira/browse/SPARK-16777
> Furthermore, the parquet-hive module has been removed from the parquet-mr
> project https://issues.apache.org/jira/browse/PARQUET-1676
> I suggest removing this piece of comment and will submit a PR later.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
