[
https://issues.apache.org/jira/browse/FLINK-23074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368004#comment-17368004
]
luoyuxia edited comment on FLINK-23074 at 6/23/21, 9:56 AM:
------------------------------------------------------------
[~ana4]
I'm not sure about what the confict is. According to the solution you proposed,
I guess you mean we need to shade the 'org.apache.parquet' included in
hive-exec to avoid flink-parquet and hive-exec refer to same
'org.apache.parquet' .
But after I build the module flink-connector-hive 2.3.6, the
'org.apache.parquet' included in hive-exec and flink-parquet actually belong to
different package as shown below.
!E8C394D1-F970-4825-82CD-3EFA74C65B27.png|width=268,height=297!
'org.apache.parquet' has been shaded to 'org.apache.flink.hive.shaded.parque'
in flink-connector-hive_2.11, and then when flink-connector-hive 2.3.6
including hive-exec, it's no need to shade 'org.apache.parquet' again.
was (Author: luoyuxia):
[~ana4]
I'm not sure about what the confict is. According to the solution you proposed,
I guess you mean we need to shade the 'org.apache.parquet' included in
hive-exec to avoid flink-parquet and hive-exec refer to same
'org.apache.parquet' .
But after I build the module flink-connector-hive 2.3.6, the
'org.apache.parquet' included in hive-exec and flink-parquet actually belong to
different package as shown below.
!E8C394D1-F970-4825-82CD-3EFA74C65B27.png!
'org.apache.parquet' has been shaded to 'org.apache.flink.hive.shaded.parque'
in flink-connector-hive_2.11, and then when flink-connector-hive 2.3.6
including hive-exec, it's no need to shade 'org.apache.parquet' again.
> There is a class conflict between flink-connector-hive and flink-parquet
> ------------------------------------------------------------------------
>
> Key: FLINK-23074
> URL: https://issues.apache.org/jira/browse/FLINK-23074
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Hive
> Affects Versions: 1.13.1, 1.12.4
> Reporter: Ada Wong
> Assignee: Ada Wong
> Priority: Major
> Attachments: E8C394D1-F970-4825-82CD-3EFA74C65B27.png,
> image-2021-06-23-17-26-32-559.png
>
>
> flink-connector-hive 2.3.6 include parquet-hadoop 1.8.1 version but
> flink-parquet include 1.11.1.
> org.apache.parquet.hadoop.example.GroupWriteSupport
> is different.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
