[ https://issues.apache.org/jira/browse/SPARK-37102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17433914#comment-17433914 ]
Apache Spark commented on SPARK-37102: -------------------------------------- User 'vmalakhin' has created a pull request for this issue: https://github.com/apache/spark/pull/34383 > Missing dependencies for hadoop-azure > ------------------------------------- > > Key: SPARK-37102 > URL: https://issues.apache.org/jira/browse/SPARK-37102 > Project: Spark > Issue Type: Bug > Components: Build > Affects Versions: 3.2.0 > Reporter: Vasily Malakhin > Priority: Major > > If we build spark distribution (from 3.2.0 release tag) via > {code:java} > ./dev/make-distribution.sh --name custom-spark --tgz -Pkubernetes > -Phadoop-cloud{code} > and try to access Azure Data Lake storage like: > {code:java} > sqlContext.read.parquet("my_data"){code} > {{then read operation fails with error:}} > {code:java} > java.lang.NoClassDefFoundError: org/codehaus/jackson/map/ObjectMapper{code} > {{And indeed I can see exclusion being applied in multiple places for > org.codehaus.jackson:jackson-mapper-asl.}} > {{Surprisingly 3.1.2 is fine, but I believe it is because avro version was > changed for 3.2.0 and update stopped bringing in such dependency.}} > {{Shall the build allow to bring extra dependencies in?}} > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org