[
https://issues.apache.org/jira/browse/FLINK-32760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
luoyuxia resolved FLINK-32760.
------------------------------
Fix Version/s: 1.18.0
1.16.3
1.17.2
Resolution: Fixed
> Version Conflict in flink-sql-connector-hive for shaded.parquet prefix
> packages
> -------------------------------------------------------------------------------
>
> Key: FLINK-32760
> URL: https://issues.apache.org/jira/browse/FLINK-32760
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Hive
> Affects Versions: 1.17.1
> Reporter: dongwoo.kim
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.18.0, 1.16.3, 1.17.2
>
> Attachments: image-2023-08-05-14-50-47-806.png
>
>
> h2. Summary
> In https://issues.apache.org/jira/browse/FLINK-23074 it seems like shading
> parquet dependency from *hive-exec* is done.
> But I think this is not enough and causing errors like below when I try to
> read parquet file using sql-gateway which requires both *flink-parquet* and
> *flink-sql-connector-hive* dependencies.
> !image-2023-08-05-14-50-47-806.png|width=1392,height=909!
>
> h2. {color:#172b4d}Cause{color}
> {color:#172b4d}Parquet dependency not only includes *org.apache.parquet* but
> also *shaded.parquet* prefix dependencies.
> ([ref|https://github.com/apache/parquet-mr/blob/515734c373f69b5250e8b63eb3d1c973da893b63/pom.xml#L72]){color}
> {color:#172b4d}So we need to shade both.{color}
> {color:#172b4d}- flink-parquet depends on Parquet 1.12.3 with shaded Thrift
> 0.16.0 (prefix: {{{}shaded.parquet{}}}){color}
> {color:#172b4d}- flink-sql-connector-hive depends on hive-exec 3.1.3 with
> Parquet 1.10.0 and shaded Thrift 0.9.3 (prefix: {{{}shaded.parquet{}}}){color}
> {color:#172b4d}- Code compiled against Thrift 0.16.0 attempts to run against
> 0.9.3, causing the error.{color}
> h2. {color:#172b4d}Proposed solution{color}
> Adding new shading rule to flink-sql-connector-hive project.
> I have confirmed that adding this rule could resolve the above error.
> {code:xml}
> <relocation>
> <pattern>shaded.parquet</pattern>
> <shadedPattern>shaded.parquet.flink.hive.shaded</shadedPattern>
> </relocation>{code}
>
> I would be happy to implement it if the proposal is accepted. Thanks
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
