[
https://issues.apache.org/jira/browse/SQOOP-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068554#comment-14068554
]
Venkat Ranganathan commented on SQOOP-1359:
-------------------------------------------
This is getting tricky from the build point of view. Some projects do not use
hadoop2/hadoop1 classifier (and are not dependent on avro-mapred.jar
hadoop1/hadoop2 specifiic classes). So it is more likely that you have in
your ivy and m2 cache, the avro-mapred.jar cached. I added the classifier
for hadoop2 builds, but the build fails when the cached artifact in the local
ivy and m2 caches do not have the right classifier. We have to clean the
avro-mapred cached jars in m2 and ivy caches to make the sqoop build successful
and this may affect building other components. I tried to update ivy to the
latest release version 2.3.0 but still ivy does not download the jar with
hadoop2 classifier if it finds something without the classifier in the m2 cache.
Looking at Sqoop, also we don't really depend on avro-mapred hadoop1/hadoop2
dependent classes (those that deal with TaskAttamptContexts).
I think we have two options:
Just update avro version alone without specifying classifiers (which avro
version would be question - 1.7.4 is used by hadoop, and 1.7.5 is used by hive
and they are some incompatibilities between the versions).
Or recommend users to nuke the avro-mapred cached jars before building sqoop
and any other avro-mapred dependent builds.
Thoughts?
> Fix avro versions in Sqoop to stop shipping hadoop1 jars with hadoop2
> ----------------------------------------------------------------------
>
> Key: SQOOP-1359
> URL: https://issues.apache.org/jira/browse/SQOOP-1359
> Project: Sqoop
> Issue Type: Bug
> Affects Versions: 1.4.4, 1.4.5
> Reporter: Venkat Ranganathan
> Assignee: Venkat Ranganathan
> Fix For: 1.4.5
>
>
> AVRO-1170 fixed avro-mapred for hadoop2 and we now have avro-mapred hadoop1
> and hadoop2 specific jars. Our dependency should be updated to use the
> right jars. I recently found similar issues with Hive shipping avro-mapred
> hadoop1 jars (HIVE-7240) causing issues in certain integration scenarios.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
