[
https://issues.apache.org/jira/browse/CAMEL-7249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13914626#comment-13914626
]
Grzegorz Grzybek commented on CAMEL-7249:
-----------------------------------------
What I did so far is:
* a copy of {{camel-hdfs}} to {{camel-hdfs2}} component repackaged into
{{org.apache.camel.component.hdfs2}} and {{hdfs2://}} uri
* a change in POM - switch from single {{hadoop-core:1.2.1}} to two deps:
** {{hadoop-common:2.3.0}}
** {{hadoop-hdfs:2.3.0}}
* all tests passed without any changes to the Hadoop API used
Then came the OSGi checking time, I:
* asked Jean-Baptiste Onofré to prepare OSGified versions of hadoop client-side
JARs (he was working on that)
* prepared my own snapshot versions of OSGified:
** hadoop-annotations
** hadoop-auth
** hadoop-common
** hadoop-hdfs
* prepared {{camel-hdfs2}} feature to be able to install this into Karaf / test
with PAX-Exam. Here's the feature right now:
{code:xml}
<feature name='camel-hdfs2' version='${project.version}' resolver='(obr)'
start-level='50'>
<feature version='${project.version}'>camel-core</feature>
<bundle dependency='true'>mvn:commons-lang/commons-lang/2.6</bundle>
<bundle dependency='true'>mvn:com.google.guava/guava/14.0.1</bundle>
<bundle
dependency='true'>mvn:com.google.protobuf/protobuf-java/2.5.0</bundle>
<bundle
dependency='true'>mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.jsch/0.1.50_1</bundle>
<bundle
dependency='true'>mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.paranamer/2.4_1</bundle>
<bundle
dependency='true'>mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.avro/1.7.3_1</bundle>
<bundle
dependency='true'>mvn:org.apache.commons/commons-compress/1.5</bundle>
<bundle
dependency='true'>mvn:org.codehaus.jackson/jackson-core-asl/1.9.12</bundle>
<bundle
dependency='true'>mvn:org.codehaus.jackson/jackson-mapper-asl/1.9.12</bundle>
<bundle dependency='true'>mvn:org.xerial.snappy/snappy-java/1.1.0.1</bundle>
<bundle dependency='true'>mvn:commons-codec/commons-codec/1.8</bundle>
<bundle
dependency='true'>mvn:commons-collections/commons-collections/3.2.1</bundle>
<bundle dependency='true'>mvn:commons-io/commons-io/1.4</bundle>
<bundle dependency='true'>mvn:commons-net/commons-net/3.3</bundle>
<bundle dependency='true'>mvn:org.apache.zookeeper/zookeeper/3.4.5</bundle>
<bundle
dependency='true'>mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.xmlenc/0.52_1</bundle>
<bundle>mvn:org.apache.camel/camel-hdfs2/2.13-SNAPSHOT</bundle>
<bundle>mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.hadoop-annotations/2.3.0_1-SNAPSHOT</bundle>
<bundle>mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.hadoop-auth/2.3.0_1-SNAPSHOT</bundle>
<bundle>mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.hadoop-common/2.3.0_1-SNAPSHOT</bundle>
<bundle>mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.hadoop-hdfs/2.3.0_1-SNAPSHOT</bundle>
</feature>
{code}
* succesfully installed the feature in clean Karaf-2.3.4:
{noformat}
karaf@root> features:install -v camel-hdfs2
Installing feature camel-hdfs2 2.13-SNAPSHOT
Installing feature camel-core 2.13-SNAPSHOT
Installing feature xml-specs-api 2.2.0
Installing bundle
mvn:org.apache.servicemix.specs/org.apache.servicemix.specs.activation-api-1.1/2.2.0
Installing bundle
mvn:org.apache.servicemix.specs/org.apache.servicemix.specs.stax-api-1.0/2.2.0
Installing bundle
mvn:org.apache.servicemix.specs/org.apache.servicemix.specs.jaxb-api-2.2/2.2.0
Installing bundle mvn:org.codehaus.woodstox/stax2-api/3.1.1
Installing bundle mvn:org.codehaus.woodstox/woodstox-core-asl/4.2.0
Installing bundle
mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.jaxb-impl/2.2.1.1_2
Installing bundle mvn:org.apache.camel/camel-core/2.13-SNAPSHOT
Installing bundle mvn:org.apache.camel.karaf/camel-karaf-commands/2.13-SNAPSHOT
Installing bundle mvn:commons-lang/commons-lang/2.6
Installing bundle mvn:com.google.guava/guava/14.0.1
Installing bundle mvn:com.google.protobuf/protobuf-java/2.5.0
Installing bundle
mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.jsch/0.1.50_1
Installing bundle
mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.paranamer/2.4_1
Installing bundle
mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.avro/1.7.3_1
Installing bundle mvn:org.apache.commons/commons-compress/1.5
Installing bundle mvn:org.codehaus.jackson/jackson-core-asl/1.9.12
Installing bundle mvn:org.codehaus.jackson/jackson-mapper-asl/1.9.12
Installing bundle mvn:org.xerial.snappy/snappy-java/1.1.0.1
Installing bundle mvn:commons-codec/commons-codec/1.8
Installing bundle mvn:commons-collections/commons-collections/3.2.1
Installing bundle mvn:commons-io/commons-io/1.4
Installing bundle mvn:commons-net/commons-net/3.3
Installing bundle mvn:org.apache.zookeeper/zookeeper/3.4.5
Installing bundle
mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.xmlenc/0.52_1
Installing bundle mvn:org.apache.camel/camel-hdfs2/2.13-SNAPSHOT
Installing bundle
mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.hadoop-annotations/2.3.0_1-SNAPSHOT
Installing bundle
mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.hadoop-auth/2.3.0_1-SNAPSHOT
Installing bundle
mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.hadoop-common/2.3.0_1-SNAPSHOT
Installing bundle
mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.hadoop-hdfs/2.3.0_1-SNAPSHOT
{noformat}
The issues so far:
* I used
mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.avro/1.7.3_1.
But the default avro bundle version for features in current master of Camel is
*1.7.5_1* which has {{sun.misc}} in the list if {{Import-Package}} making
camel-hdfs2 unusable outside Oracle JDK (maybe it's not an issue after all)
* {{org.apache.camel:camel-itest-osgi}} can't be used to test both versions of
camel-hdfs becuse API's of Hadoop 1 and 2 are mostly the same. It'd be weird to
duplicate this itest project just for hdfs2.
> Upgrade Camel-HDFS component to be compatible with Apache Hadoop 2.x
> --------------------------------------------------------------------
>
> Key: CAMEL-7249
> URL: https://issues.apache.org/jira/browse/CAMEL-7249
> Project: Camel
> Issue Type: New Feature
> Reporter: Grzegorz Grzybek
>
> Currently camel-hdfs depends on hadoop-core:1.2.1. Make (working name)
> {{camel-hdfs2}} component using newest (hadoop 2.3.0) libraries.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
