[
https://issues.apache.org/jira/browse/FLINK-15185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingsong Lee updated FLINK-15185:
---------------------------------
Description:
Now in hive HadoopFileSystemFactory, we use
org.apache.flink.runtime.fs.hdfs.HadoopFileSystem to get FileSystem.
But it should not work after we setting default child first class loader.
Because in standalone mode, the cluster has no hadoop dependency. So the
solution is:
# Add `flink-hadoop-fs` dependency to hive module, not work, because classes
with "org.apache.flink" prefix will always be loaded by parent class loader :(
# User add hadoop dependency to standalone cluster, it breaks out-of-the-box.
# Shade hadoop FileSystem in hive module, not complex, good.
was:
Now in hive HadoopFileSystemFactory, we use
org.apache.flink.runtime.fs.hdfs.HadoopFileSystem to get FileSystem.
But it should not work after we setting default child first class loader.
Because in standalone mode, the cluster has no hadoop dependency. So the
solution is:
# Add `flink-hadoop-fs` dependency to hive module, not work, because classes
with "org.apache.flink" prefix will always be loaded by parent class loader :(
# User add hadoop dependency to standalone cluster, it breaks out-of-the-box.
# Implement hive hadoop FileSystem, not complex, good.
> Hive sink can not run in standalone mode
> ----------------------------------------
>
> Key: FLINK-15185
> URL: https://issues.apache.org/jira/browse/FLINK-15185
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Hive
> Reporter: Jingsong Lee
> Priority: Major
> Fix For: 1.10.0
>
>
> Now in hive HadoopFileSystemFactory, we use
> org.apache.flink.runtime.fs.hdfs.HadoopFileSystem to get FileSystem.
> But it should not work after we setting default child first class loader.
> Because in standalone mode, the cluster has no hadoop dependency. So the
> solution is:
> # Add `flink-hadoop-fs` dependency to hive module, not work, because classes
> with "org.apache.flink" prefix will always be loaded by parent class loader :(
> # User add hadoop dependency to standalone cluster, it breaks out-of-the-box.
> # Shade hadoop FileSystem in hive module, not complex, good.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
