Github user haohui commented on a diff in the pull request:
https://github.com/apache/flink/pull/4683#discussion_r140696992
--- Diff: flink-core/pom.xml ---
@@ -52,6 +52,12 @@ under the License.
<artifactId>flink-shaded-asm</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-shaded-hadoop2</artifactId>
+ <version>${project.version}</version>
+ </dependency>
--- End diff --
Internally we have several users that try Flink to read the files generated
by Hadoop (e.g. lz4 / gz / snappy). I think the support of Hadoop is quite
important.
I'm not sure supporting the xerial snappy format is a good idea. The two
file formats are actually incompatible -- it would be quite confusing for the
users to find out that they can't access the files using Spark / MR / Hive due
to a missed configuration.
I suggest at least we should make the Hadoop file format as the default --
or to just get rid of the xerial version of the file format.
Putting the dependency in provided sounds fine to me -- if we need even
tighter controls on the dependency, we can start thinking about having a
separate module for it.
What do you think?
---
