After upgrading from apache spark 2.1.1 to 2.2.0 our integration test fail with an exception:
java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.mapred.FileInputFormat at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:312) at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:194) Seems there is an incompatibility with hadoop artifacts and guava, more details are described here: https://stackoverflow.com/questions/36427291/illegalaccesserror-to-guavas-stopwatch-from-org-apache-hadoop-mapreduce-lib-inp Advising maven to use more recent versions of affected artifacts solved this issue. Questions: Is this a reasonable workaround for deployment into production? Or should we still stay on 2.1.1 until these dependencies are fixed in spark? Is there a ticket referencing this issue?