Re: Unable to run flink job after adding a new dependency (cannot find Main-class)

2017-09-25 Thread Federico D'Ambrosio
As a little update, the pattern for the exclusion of those files in sbt-assembly is the following: assemblyMergeStrategy in assembly := { case PathList(ps @ _*) if ps.last.endsWith(".DSA") || ps.last.endsWith(".SF") || ps.last.endsWith(".RSA") => MergeStrategy.discard //Other MergeStrategies

Re: Unable to run flink job after adding a new dependency (cannot find Main-class)

2017-09-25 Thread Federico D'Ambrosio
Hi Urs, Thank you very much for your advice, I will look into excluding those files directly during the assembly. 2017-09-25 10:58 GMT+02:00 Urs Schoenenberger < urs.schoenenber...@tngtech.com>: > Hi Federico, > > oh, I remember running into this problem some time ago. If I recall > correctly,

Re: Unable to run flink job after adding a new dependency (cannot find Main-class)

2017-09-25 Thread Urs Schoenenberger
Hi Federico, oh, I remember running into this problem some time ago. If I recall correctly, this is not a flink issue, but an issue with technically incorrect jars from dependencies which prevent the verification of the manifest. I was using the maven-shade plugin back then and configured an

Re: Unable to run flink job after adding a new dependency (cannot find Main-class)

2017-09-25 Thread Federico D'Ambrosio
Hi Urs, Yes the main class is set, just like you said. Still, I might have managed to get it working: during the assembly some .SF, .DSA and .RSA files are put inside the META-INF folder of the jar, possibly coming from some of the new dependencies in the deps tree. Apparently, this caused this

Re: Unable to run flink job after adding a new dependency (cannot find Main-class)

2017-09-25 Thread Urs Schoenenberger
Hi Federico, just guessing, but are you explicitly setting the Main-Class manifest attribute for the jar that you are building? Should be something like mainClass in (Compile, packageBin) := Some("org.yourorg.YourFlinkJobMainClass") Best, Urs On 23.09.2017 17:53, Federico D'Ambrosio wrote: >

Unable to run flink job after adding a new dependency (cannot find Main-class)

2017-09-23 Thread Federico D'Ambrosio
Hello everyone, I'd like to submit to you this weird issue I'm having, hoping you could help me. Premise: I'm using sbt 0.13.6 for building, scala 2.11.8 and flink 1.3.2 compiled from sources against hadoop 2.7.3.2.6.1.0-129 (HDP 2.6) So, I'm trying to implement an sink for Hive so I added the