RE: Spark 2.0.0-preview ... problem with jackson core version
Yes ! We got it :-) Btw it's not available on Maven yet. :-( Paolo PatiernoSenior Software Engineer (IoT) @ Red Hat Microsoft MVP on Windows Embedded & IoTMicrosoft Azure Advisor Twitter : @ppatierno Linkedin : paolopatierno Blog : DevExperience > From: so...@cloudera.com > Date: Sat, 2 Jul 2016 15:12:11 +0100 > Subject: Re: Spark 2.0.0-preview ... problem with jackson core version > To: ppatie...@live.com > CC: charles.al...@metamarkets.com; user@spark.apache.org > > Ah, it looks like it was 2.5.3 as of 2.0.0-preview: > > https://github.com/apache/spark/blob/2.0.0-preview/pom.xml#L164 > > but was updated to 2.6.5 soon after that, since it was 2.6.5 in 2.0.0-RC1: > > https://github.com/apache/spark/blob/v2.0.0-rc1/pom.xml#L163 > > On Sat, Jul 2, 2016 at 3:04 PM, Paolo Patierno wrote: > > This sounds strange to me because here : > > > > https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11/2.0.0-preview > > > > I see : > > > > com.fasterxml.jackson.module » jackson-module-scala_2.112.5.3 > > > > So it seems that 2.0.0-preview is bringing jackson module scala 2.5.3 that > > is what I see. > > > > > > > >> From: so...@cloudera.com > >> Date: Sat, 2 Jul 2016 14:32:58 +0100 > >> Subject: Re: Spark 2.0.0-preview ... problem with jackson core version > >> To: ppatie...@live.com > >> CC: charles.al...@metamarkets.com; user@spark.apache.org > > > >> > >> This is something to do with your app. The version is 2.6.5 in master > >> and branch-2.0, and jackson-module-scala is managed to this version > >> along with all the other jackson artifacts. > >> > >> On Sat, Jul 2, 2016 at 1:35 PM, Paolo Patierno wrote: > >> > What I see is the following ... > >> > > >> > - Working configuration > >> > > >> > Spark Version : "2.0.0-SNAPSHOT" > >> > > >> > The Vert.x library brings ... > >> > jackson-annotations:2.6.0 > >> > jackson-core:2.6.1 > >> > jackson-databind:2.6.1 > >> > > >> > Spark brings > >> > jackson-annotations:2.6.5 > >> > jackson-core:2.6.5 > >> > jackson-databind:2.6.5 > >> > jackson-module-scala_2.11: 2.6.5 > >> > > >> > The runtime will use the latest version 2.6.5 and all works fine. > >> > > >> > - NOT Working configuration > >> > > >> > Spark Version : "2.0.0-preview" > >> > > >> > The Vert.x library brings ... (same dependencies as before) > >> > jackson-annotations:2.6.0 > >> > jackson-core:2.6.1 > >> > jackson-databind:2.6.1 > >> > > >> > Spark brings > >> > jackson-module-scala_2.11: 2.5.3 > >> > > >> > The module scale related to jackson is 2.5.3 ... seems not work with > >> > above > >> > 2.6.0/2.6.1 version ... so the exception. > >> > > >> > > >> >> From: so...@cloudera.com > >> >> Date: Sat, 2 Jul 2016 08:34:45 +0100 > >> >> Subject: Re: Spark 2.0.0-preview ... problem with jackson core version > >> >> To: charles.al...@metamarkets.com > >> >> CC: ppatie...@live.com; user@spark.apache.org > >> > > >> >> > >> >> mvn dependency:tree? > >> >> > >> >> On Sat, Jul 2, 2016 at 12:46 AM, Charles Allen > >> >> wrote: > >> >> > I'm having the same difficulty porting > >> >> > https://github.com/metamx/druid-spark-batch/tree/spark2 over to > >> >> > spark2.x, > >> >> > where I have to go track down who is pulling in bad jackson versions. > >> >> > > >> >> > >> >> - > >> >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >> >> > >> > >> - > >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >> > > - > To unsubscribe e-mail: user-unsubscr...@spark.apache.org >
Re: Spark 2.0.0-preview ... problem with jackson core version
Ah, it looks like it was 2.5.3 as of 2.0.0-preview: https://github.com/apache/spark/blob/2.0.0-preview/pom.xml#L164 but was updated to 2.6.5 soon after that, since it was 2.6.5 in 2.0.0-RC1: https://github.com/apache/spark/blob/v2.0.0-rc1/pom.xml#L163 On Sat, Jul 2, 2016 at 3:04 PM, Paolo Patierno wrote: > This sounds strange to me because here : > > https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11/2.0.0-preview > > I see : > > com.fasterxml.jackson.module » jackson-module-scala_2.112.5.3 > > So it seems that 2.0.0-preview is bringing jackson module scala 2.5.3 that > is what I see. > > > >> From: so...@cloudera.com >> Date: Sat, 2 Jul 2016 14:32:58 +0100 >> Subject: Re: Spark 2.0.0-preview ... problem with jackson core version >> To: ppatie...@live.com >> CC: charles.al...@metamarkets.com; user@spark.apache.org > >> >> This is something to do with your app. The version is 2.6.5 in master >> and branch-2.0, and jackson-module-scala is managed to this version >> along with all the other jackson artifacts. >> >> On Sat, Jul 2, 2016 at 1:35 PM, Paolo Patierno wrote: >> > What I see is the following ... >> > >> > - Working configuration >> > >> > Spark Version : "2.0.0-SNAPSHOT" >> > >> > The Vert.x library brings ... >> > jackson-annotations:2.6.0 >> > jackson-core:2.6.1 >> > jackson-databind:2.6.1 >> > >> > Spark brings >> > jackson-annotations:2.6.5 >> > jackson-core:2.6.5 >> > jackson-databind:2.6.5 >> > jackson-module-scala_2.11: 2.6.5 >> > >> > The runtime will use the latest version 2.6.5 and all works fine. >> > >> > - NOT Working configuration >> > >> > Spark Version : "2.0.0-preview" >> > >> > The Vert.x library brings ... (same dependencies as before) >> > jackson-annotations:2.6.0 >> > jackson-core:2.6.1 >> > jackson-databind:2.6.1 >> > >> > Spark brings >> > jackson-module-scala_2.11: 2.5.3 >> > >> > The module scale related to jackson is 2.5.3 ... seems not work with >> > above >> > 2.6.0/2.6.1 version ... so the exception. >> > >> > >> >> From: so...@cloudera.com >> >> Date: Sat, 2 Jul 2016 08:34:45 +0100 >> >> Subject: Re: Spark 2.0.0-preview ... problem with jackson core version >> >> To: charles.al...@metamarkets.com >> >> CC: ppatie...@live.com; user@spark.apache.org >> > >> >> >> >> mvn dependency:tree? >> >> >> >> On Sat, Jul 2, 2016 at 12:46 AM, Charles Allen >> >> wrote: >> >> > I'm having the same difficulty porting >> >> > https://github.com/metamx/druid-spark-batch/tree/spark2 over to >> >> > spark2.x, >> >> > where I have to go track down who is pulling in bad jackson versions. >> >> > >> >> >> >> - >> >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> >> >> >> - >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> - To unsubscribe e-mail: user-unsubscr...@spark.apache.org
RE: Spark 2.0.0-preview ... problem with jackson core version
This sounds strange to me because here : https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11/2.0.0-preview I see : com.fasterxml.jackson.module » jackson-module-scala_2.112.5.3 So it seems that 2.0.0-preview is bringing jackson module scala 2.5.3 that is what I see. > From: so...@cloudera.com > Date: Sat, 2 Jul 2016 14:32:58 +0100 > Subject: Re: Spark 2.0.0-preview ... problem with jackson core version > To: ppatie...@live.com > CC: charles.al...@metamarkets.com; user@spark.apache.org > > This is something to do with your app. The version is 2.6.5 in master > and branch-2.0, and jackson-module-scala is managed to this version > along with all the other jackson artifacts. > > On Sat, Jul 2, 2016 at 1:35 PM, Paolo Patierno wrote: > > What I see is the following ... > > > > - Working configuration > > > > Spark Version : "2.0.0-SNAPSHOT" > > > > The Vert.x library brings ... > > jackson-annotations:2.6.0 > > jackson-core:2.6.1 > > jackson-databind:2.6.1 > > > > Spark brings > > jackson-annotations:2.6.5 > > jackson-core:2.6.5 > > jackson-databind:2.6.5 > > jackson-module-scala_2.11: 2.6.5 > > > > The runtime will use the latest version 2.6.5 and all works fine. > > > > - NOT Working configuration > > > > Spark Version : "2.0.0-preview" > > > > The Vert.x library brings ... (same dependencies as before) > > jackson-annotations:2.6.0 > > jackson-core:2.6.1 > > jackson-databind:2.6.1 > > > > Spark brings > > jackson-module-scala_2.11: 2.5.3 > > > > The module scale related to jackson is 2.5.3 ... seems not work with above > > 2.6.0/2.6.1 version ... so the exception. > > > > > >> From: so...@cloudera.com > >> Date: Sat, 2 Jul 2016 08:34:45 +0100 > >> Subject: Re: Spark 2.0.0-preview ... problem with jackson core version > >> To: charles.al...@metamarkets.com > >> CC: ppatie...@live.com; user@spark.apache.org > > > >> > >> mvn dependency:tree? > >> > >> On Sat, Jul 2, 2016 at 12:46 AM, Charles Allen > >> wrote: > >> > I'm having the same difficulty porting > >> > https://github.com/metamx/druid-spark-batch/tree/spark2 over to > >> > spark2.x, > >> > where I have to go track down who is pulling in bad jackson versions. > >> > > >> > >> - > >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >> > > - > To unsubscribe e-mail: user-unsubscr...@spark.apache.org >
Re: Spark 2.0.0-preview ... problem with jackson core version
This is something to do with your app. The version is 2.6.5 in master and branch-2.0, and jackson-module-scala is managed to this version along with all the other jackson artifacts. On Sat, Jul 2, 2016 at 1:35 PM, Paolo Patierno wrote: > What I see is the following ... > > - Working configuration > > Spark Version : "2.0.0-SNAPSHOT" > > The Vert.x library brings ... > jackson-annotations:2.6.0 > jackson-core:2.6.1 > jackson-databind:2.6.1 > > Spark brings > jackson-annotations:2.6.5 > jackson-core:2.6.5 > jackson-databind:2.6.5 > jackson-module-scala_2.11: 2.6.5 > > The runtime will use the latest version 2.6.5 and all works fine. > > - NOT Working configuration > > Spark Version : "2.0.0-preview" > > The Vert.x library brings ... (same dependencies as before) > jackson-annotations:2.6.0 > jackson-core:2.6.1 > jackson-databind:2.6.1 > > Spark brings > jackson-module-scala_2.11: 2.5.3 > > The module scale related to jackson is 2.5.3 ... seems not work with above > 2.6.0/2.6.1 version ... so the exception. > > >> From: so...@cloudera.com >> Date: Sat, 2 Jul 2016 08:34:45 +0100 >> Subject: Re: Spark 2.0.0-preview ... problem with jackson core version >> To: charles.al...@metamarkets.com >> CC: ppatie...@live.com; user@spark.apache.org > >> >> mvn dependency:tree? >> >> On Sat, Jul 2, 2016 at 12:46 AM, Charles Allen >> wrote: >> > I'm having the same difficulty porting >> > https://github.com/metamx/druid-spark-batch/tree/spark2 over to >> > spark2.x, >> > where I have to go track down who is pulling in bad jackson versions. >> > >> >> - >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> - To unsubscribe e-mail: user-unsubscr...@spark.apache.org
RE: Spark 2.0.0-preview ... problem with jackson core version
What I see is the following ... - Working configuration Spark Version : "2.0.0-SNAPSHOT" The Vert.x library brings ... jackson-annotations:2.6.0 jackson-core:2.6.1 jackson-databind:2.6.1 Spark brings jackson-annotations:2.6.5 jackson-core:2.6.5 jackson-databind:2.6.5 jackson-module-scala_2.11: 2.6.5 The runtime will use the latest version 2.6.5 and all works fine. - NOT Working configuration Spark Version : "2.0.0-preview" The Vert.x library brings ... (same dependencies as before) jackson-annotations:2.6.0 jackson-core:2.6.1 jackson-databind:2.6.1 Spark brings jackson-module-scala_2.11: 2.5.3 The module scale related to jackson is 2.5.3 ... seems not work with above 2.6.0/2.6.1 version ... so the exception. > From: so...@cloudera.com > Date: Sat, 2 Jul 2016 08:34:45 +0100 > Subject: Re: Spark 2.0.0-preview ... problem with jackson core version > To: charles.al...@metamarkets.com > CC: ppatie...@live.com; user@spark.apache.org > > mvn dependency:tree? > > On Sat, Jul 2, 2016 at 12:46 AM, Charles Allen > wrote: > > I'm having the same difficulty porting > > https://github.com/metamx/druid-spark-batch/tree/spark2 over to spark2.x, > > where I have to go track down who is pulling in bad jackson versions. > > > > - > To unsubscribe e-mail: user-unsubscr...@spark.apache.org >
Re: Spark 2.0.0-preview ... problem with jackson core version
mvn dependency:tree? On Sat, Jul 2, 2016 at 12:46 AM, Charles Allen wrote: > I'm having the same difficulty porting > https://github.com/metamx/druid-spark-batch/tree/spark2 over to spark2.x, > where I have to go track down who is pulling in bad jackson versions. > - To unsubscribe e-mail: user-unsubscr...@spark.apache.org
Re: Spark 2.0.0-preview ... problem with jackson core version
I'm having the same difficulty porting https://github.com/metamx/druid-spark-batch/tree/spark2 over to spark2.x, where I have to go track down who is pulling in bad jackson versions. On Fri, Jul 1, 2016 at 11:59 AM Sean Owen wrote: > Are you just asking why you can't use 2.5.3 in your app? because > Jackson isn't shaded, which is sort of the bad news. But just use > 2.6.5 too, ideally. I don't know where 2.6.1 is coming from, but Spark > doesn't use it. > > On Fri, Jul 1, 2016 at 5:48 PM, wrote: > > In my project I found the library which brings Jackson core 2.6.5 and it > is > > used in conjunction with the requested Jackson scala module 2.5.3 wanted > by > > spark 2.0.0 preview. At runtime it's the cause of exception. > > > > Now I have excluded 2.6.5 using sbt but it could be dangerous for the > other > > library. > > > > Why this regression to Jackson 2.5.3 switching from 2.0.0 snapshot to > > preview ? > > > > Thanks > > Paolo > > > > Get Outlook for Android > > > > > > > > > > On Fri, Jul 1, 2016 at 4:24 PM +0200, "Paolo Patierno" < > ppatie...@live.com> > > wrote: > > > > Hi, > > > > developing a custom receiver up today I used spark version > "2.0.0-SNAPSHOT" > > and scala version 2.11.7. > > With these version all tests work fine. > > > > I have just switching to "2.0.0-preview" as spark version but not I have > > following error : > > > > An exception or error caused a run to abort: class > > com.fasterxml.jackson.module.scala.ser.ScalaIteratorSerializer overrides > > final method > > > withResolved.(Lcom/fasterxml/jackson/databind/BeanProperty;Lcom/fasterxml/jackson/databind/jsontype/TypeSerializer;Lcom/fasterxml/jackson/databind/JsonSerializer;)Lcom/fasterxml/jackson/databind/ser/std/AsArraySerializerBase; > > java.lang.VerifyError: class > > com.fasterxml.jackson.module.scala.ser.ScalaIteratorSerializer overrides > > final method > > > withResolved.(Lcom/fasterxml/jackson/databind/BeanProperty;Lcom/fasterxml/jackson/databind/jsontype/TypeSerializer;Lcom/fasterxml/jackson/databind/JsonSerializer;)Lcom/fasterxml/jackson/databind/ser/std/AsArraySerializerBase; > > at java.lang.ClassLoader.defineClass1(Native Method) > > > > I see that using 2.0.0-SNAPSHOT the jackson core version is 2.6.5 ... now > > 2.6.1 with module scala 2.5.3. > > > > Thanks, > > Paolo. > > > > Paolo Patierno > > Senior Software Engineer (IoT) @ Red Hat > > Microsoft MVP on Windows Embedded & IoT > > Microsoft Azure Advisor > > > > Twitter : @ppatierno > > Linkedin : paolopatierno > > Blog : DevExperience > > - > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
Re: Spark 2.0.0-preview ... problem with jackson core version
Are you just asking why you can't use 2.5.3 in your app? because Jackson isn't shaded, which is sort of the bad news. But just use 2.6.5 too, ideally. I don't know where 2.6.1 is coming from, but Spark doesn't use it. On Fri, Jul 1, 2016 at 5:48 PM, wrote: > In my project I found the library which brings Jackson core 2.6.5 and it is > used in conjunction with the requested Jackson scala module 2.5.3 wanted by > spark 2.0.0 preview. At runtime it's the cause of exception. > > Now I have excluded 2.6.5 using sbt but it could be dangerous for the other > library. > > Why this regression to Jackson 2.5.3 switching from 2.0.0 snapshot to > preview ? > > Thanks > Paolo > > Get Outlook for Android > > > > > On Fri, Jul 1, 2016 at 4:24 PM +0200, "Paolo Patierno" > wrote: > > Hi, > > developing a custom receiver up today I used spark version "2.0.0-SNAPSHOT" > and scala version 2.11.7. > With these version all tests work fine. > > I have just switching to "2.0.0-preview" as spark version but not I have > following error : > > An exception or error caused a run to abort: class > com.fasterxml.jackson.module.scala.ser.ScalaIteratorSerializer overrides > final method > withResolved.(Lcom/fasterxml/jackson/databind/BeanProperty;Lcom/fasterxml/jackson/databind/jsontype/TypeSerializer;Lcom/fasterxml/jackson/databind/JsonSerializer;)Lcom/fasterxml/jackson/databind/ser/std/AsArraySerializerBase; > java.lang.VerifyError: class > com.fasterxml.jackson.module.scala.ser.ScalaIteratorSerializer overrides > final method > withResolved.(Lcom/fasterxml/jackson/databind/BeanProperty;Lcom/fasterxml/jackson/databind/jsontype/TypeSerializer;Lcom/fasterxml/jackson/databind/JsonSerializer;)Lcom/fasterxml/jackson/databind/ser/std/AsArraySerializerBase; > at java.lang.ClassLoader.defineClass1(Native Method) > > I see that using 2.0.0-SNAPSHOT the jackson core version is 2.6.5 ... now > 2.6.1 with module scala 2.5.3. > > Thanks, > Paolo. > > Paolo Patierno > Senior Software Engineer (IoT) @ Red Hat > Microsoft MVP on Windows Embedded & IoT > Microsoft Azure Advisor > > Twitter : @ppatierno > Linkedin : paolopatierno > Blog : DevExperience - To unsubscribe e-mail: user-unsubscr...@spark.apache.org
Re: Spark 2.0.0-preview ... problem with jackson core version
In my project I found the library which brings Jackson core 2.6.5 and it is used in conjunction with the requested Jackson scala module 2.5.3 wanted by spark 2.0.0 preview. At runtime it's the cause of exception. Now I have excluded 2.6.5 using sbt but it could be dangerous for the other library. Why this regression to Jackson 2.5.3 switching from 2.0.0 snapshot to preview ? Thanks Paolo Get Outlook for Android On Fri, Jul 1, 2016 at 4:24 PM +0200, "Paolo Patierno" wrote: Hi, developing a custom receiver up today I used spark version "2.0.0-SNAPSHOT" and scala version 2.11.7. With these version all tests work fine. I have just switching to "2.0.0-preview" as spark version but not I have following error : An exception or error caused a run to abort: class com.fasterxml.jackson.module.scala.ser.ScalaIteratorSerializer overrides final method withResolved.(Lcom/fasterxml/jackson/databind/BeanProperty;Lcom/fasterxml/jackson/databind/jsontype/TypeSerializer;Lcom/fasterxml/jackson/databind/JsonSerializer;)Lcom/fasterxml/jackson/databind/ser/std/AsArraySerializerBase; java.lang.VerifyError: class com.fasterxml.jackson.module.scala.ser.ScalaIteratorSerializer overrides final method withResolved.(Lcom/fasterxml/jackson/databind/BeanProperty;Lcom/fasterxml/jackson/databind/jsontype/TypeSerializer;Lcom/fasterxml/jackson/databind/JsonSerializer;)Lcom/fasterxml/jackson/databind/ser/std/AsArraySerializerBase; at java.lang.ClassLoader.defineClass1(Native Method) I see that using 2.0.0-SNAPSHOT the jackson core version is 2.6.5 ... now 2.6.1 with module scala 2.5.3. Thanks, Paolo. Paolo PatiernoSenior Software Engineer (IoT) @ Red Hat Microsoft MVP on Windows Embedded & IoTMicrosoft Azure Advisor Twitter : @ppatierno Linkedin : paolopatierno Blog : DevExperience