lamber-ken created CARBONDATA-3389:
--------------------------------------
Summary: Optimize the bundling of Hadoop libraries with CarbonData
Key: CARBONDATA-3389
URL: https://issues.apache.org/jira/browse/CARBONDATA-3389
Project: CarbonData
Issue Type: Improvement
Components: hadoop-integration
Affects Versions: 1.5.3
Reporter: lamber-ken
Fix For: NONE
For now, CarbonData provides archives bunding with hadoop-2.7.2, user needs to
build carbondata to fit their own hadoop env.
{code:java}
apache-carbondata-1.5.3-bin-spark2.1.0-hadoop2.7.2.jar
apache-carbondata-1.5.3-bin-spark2.1.0-hadoop2.7.2.jar.asc
apache-carbondata-1.5.3-bin-spark2.1.0-hadoop2.7.2.jar.sha512
apache-carbondata-1.5.3-bin-spark2.2.1-hadoop2.7.2.jar
apache-carbondata-1.5.3-bin-spark2.2.1-hadoop2.7.2.jar.asc
apache-carbondata-1.5.3-bin-spark2.2.1-hadoop2.7.2.jar.sha512
apache-carbondata-1.5.3-bin-spark2.3.2-hadoop2.7.2.jar
apache-carbondata-1.5.3-bin-spark2.3.2-hadoop2.7.2.jar.asc
apache-carbondata-1.5.3-bin-spark2.3.2-hadoop2.7.2.jar.sha512
{code}
I think it's better to split carbondata and hadoop. use can manually download a
pre-packaged Hadoop jar from the optional components, like bellow
{code:java}
CarbonData 1.6.0
CarbonData 1.6.0 for Scala 2.11
CarbonData 1.6.0 for Scala 2.12
Optional components
Pre-bundled Hadoop 2.4.1
Pre-bundled Hadoop 2.6.5
Pre-bundled Hadoop 2.7.5
Pre-bundled Hadoop 2.8.3{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
