[
https://issues.apache.org/jira/browse/FLINK-17978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133964#comment-17133964
]
Yang Wang commented on FLINK-17978:
-----------------------------------
Update:
I setup two EMR clusters on AliCloud. The hadoop versions are 2.8.5 and 3.1.3.
Build the Flink binary from the latest
source(22098b27c32342f3ef74848a86d4b4d8d3c1dfb8).
Do not have {{flink-shaded-hadoop-*.jar}} in the lib directory. And {{export
HADOOP_CLASSPATH=`hadoop classpath`}} before submitting the job.
Running the following two Flink jobs.
* streaming WordCount.jar, write the result to HDFS, directly depends on hadoop
* A self developed simple demo to write result using {{StreamingFileSink}}
with orc format, indirectly depends on hadoop
Both the two jobs finished successfully and the result is expected. There are
no exceptions in the client, jobmanager, taskmanager logs.
> Test Hadoop dependency change
> -----------------------------
>
> Key: FLINK-17978
> URL: https://issues.apache.org/jira/browse/FLINK-17978
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Affects Versions: 1.11.0
> Reporter: Till Rohrmann
> Assignee: Yang Wang
> Priority: Critical
> Labels: release-testing
> Fix For: 1.11.0
>
>
> Test the Hadoop dependency change:
> * Run Flink with HBase/ORC (maybe add e2e test)
> * Validate meaningful exception message if Hadoop dependency is missing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
