Re: Flink 1.11.2 test cases fail with Scala 2.12.12

[Chesnay Schepler]

Coud you check your dependency tree for the version of scala-library?

On 2/24/2021 7:28 AM, soumoks wrote:

Thank you for the response but this error continues to happen with Scala
2.12.7.
The app itself continues to compile without errors but the test cases fail
with the same error.

Seems to be related to
https://issues.apache.org/jira/browse/FLINK-12461

I have set the Scala version in pom.xml file and I have used this property
value for all dependencies present.

     <properties>
         <maven.compiler.source>1.8</maven.compiler.source>
         <maven.compiler.target>1.8</maven.compiler.target>
         <encoding>UTF-8</encoding>
         <scala.version>2.12.7</scala.version>
         <scala.compat.version>2.12</scala.compat.version>
         <aws.sdk.version>1.11.461</aws.sdk.version>
         <spec2.version>4.2.0</spec2.version>
     </properties>

However, I ran into the same error

org.apache.flink.shaded.guava18.com.google.common.util.concurrent.ExecutionError:
java.lang.NoClassDefFoundError: scala/math/Ordering$$anon$9
   at
org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
   at
org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache.get(LocalCache.java:3937)
   at
org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4739)
   at
org.apache.flink.api.scala.typeutils.TraversableSerializer$.compileCbf(TraversableSerializer.scala:184)
   at
org.apache.flink.api.scala.typeutils.TraversableSerializer.compileCbf(TraversableSerializer.scala:51)
   at
org.apache.flink.api.scala.typeutils.TraversableSerializer.<init>(TraversableSerializer.scala:41)




and it occurs if I use a Scala mutable/immutable map data-structure in my
code.

Sample test code:

val data: scala.collection.mutable.Map[String, String] =
       scala.collection.mutable.Map("key1" -> "v1", "key2" -> "v2")

     val kafkaMsg: String = write(
       SampleCaseClass(14L, "23FC", 10L, data)
     )

     val stream: DataStream[SampleCaseClass] =
       env.addSource(new MockKafkaSource(kafkaMsg)).
flatMap(new SampleCaseClassMapper)


I am including part of pom file related to test packages for reference.

         <dependency>
             <groupId>org.scalatest</groupId>
             <artifactId>scalatest_${scala.compat.version}</artifactId>
             <version>3.2.3</version>
             <scope>test</scope>
         </dependency>

     <build>
         <sourceDirectory>src/main/scala</sourceDirectory>
         <testSourceDirectory>src/test/scala</testSourceDirectory>
         <plugins>
             <plugin>

<groupId>net.alchim31.maven</groupId>

                 <artifactId>scala-maven-plugin</artifactId>
                 <version>4.4.0</version>
                 <executions>
                     <execution>
                         <goals>
                             <goal>compile</goal>
                             <goal>testCompile</goal>
                         </goals>
                         <configuration>
                             <args>
                                 <arg>-dependencyfile</arg>

<arg>${project.build.directory}/.scala_dependencies</arg>

                             </args>
                         </configuration>
                     </execution>
                 </executions>
             </plugin>
             <plugin>
                 <groupId>org.apache.maven.plugins</groupId>
                 <artifactId>maven-surefire-plugin</artifactId>
                 <version>2.22.2</version>
                 <configuration>

<skipTests>true</skipTests>

                 </configuration>
             </plugin>

             <plugin>
                 <groupId>org.scalatest</groupId>
                 <artifactId>scalatest-maven-plugin</artifactId>
                 <version>2.0.2</version>
                 <configuration>

<reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>

                     <junitxml>.</junitxml>
                     <filereports>TestSuiteReport.txt</filereports>

</configuration>

                 <executions>
                     <execution>
                         <id>test</id>
                         <goals>
                             <goal>test</goal>
                         </goals>
                     </execution>
                 </executions>
             </plugin>
         </plugins>
     </build>



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/