[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555916#comment-17555916
]
Ayush Saxena commented on HADOOP-18033:
---------------------------------------
Spent some time checking if we have any quick solution or not and see how
things are:
Both {{jsr311-api}} and {{javax.ws.rs-api}} have couple of similar classes and
different implementations, That is why this duplicate classes issue started
surfacing, I guess Jackson 2 requires implementation classes from
{{javax.ws.rs-api}} at runtime or so. In ideal situation we should either have
{{javax.ws.rs-api}} or {{js311-api}} in our code, when adding
{{javax.ws.rs-api}} if we could have got rid of {{js311-api}} then everything
would have been sorted for the shading part. But I guess we have some
dependencies on {{{}js311-api{}}}, and it is coming from some other thirdparty
libs as well, so may be we have to explore and upgrade them to a version, where
they ditch {{js311-api}} for {{{}javax.ws.rs-api{}}}. Then our shading jar
should get sorted. How tough is that we don't know, a normal exclude of
{{js311-api}} as a transitive dependency isn't a solution because
{{javax.ws.rs-api}} has different implementation of methods.
The duplicate class exception that we saw here was actually an alarm here that
these two dependencies can't stay in peace together, but we got away with that
by an exclude...
Now coming for Tez, Tez still has {{js311-api}} as a dependency, if we some how
ditch that and move to {{javax.ws.rs-api}} in hadoop, I am not very sure if Tez
too have to adapt to our Jackson version and do the same to get things working..
{quote}FWIW, although Hadoop 3.3 could revert this for 3.3.4 release but from
security viewpoint, staying up with latest Jackson2 is also in good favour of
3.3 release line
{quote}
Revert isn't an option now, HADOOP-18178 got its way clear only because of
this, else it would have been facing this same issue and would have crashed.
Now we have a CVE fixed in 3.3.2 & 3.3.3, we can't get it back in 3.3.4, We
won't fix a thirdparty CVE we could have said, but after fixing and claiming we
have fixed one, we can't get it back AFAIK, this issue only somehow we have to
fix.
BTW. I am not sure what Spark and Kyubi issues are exactly, that also seems
class conflicts may be.. [~pan3793] can you share some more information about
that here
> Upgrade fasterxml Jackson to 2.13.0
> -----------------------------------
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Reporter: Akira Ajisaka
> Assignee: Viraj Jasani
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
