Well thats the point, I do have 2.2.3 installed as that's the version shipped with HDP 2.3 and that gets loaded first in the classpath for YARN resourcemanager.
I guess I have three alternatives. 1) build myriad using 2.2.3, thus matching the HDP installed jar's 2) replace the HDP version with 2.5.1, not sure what's that going to do for HDP compatibility 3) prepend a new classpath for yarn resourcemanager to pick up myriad versioned jars first. -- Jonathan (Bill) Sparks Software Architecture Cray Inc. On 8/19/15 12:36 PM, "Adam Bordelon" <a...@mesosphere.io> wrote: >Myriad should be using jackson 2.5.1 >https://github.com/mesos/myriad/blob/d6d765736ba1c8f59aa967457527331e1dab6 >743/myriad-scheduler/build.gradle#L13 >Double-check your build.gradle, and make sure you don't have a jackson >2.2.3 preinstalled somewhere else on your system > >On Wed, Aug 19, 2015 at 8:20 AM, Bill Sparks <jspa...@cray.com> wrote: > >> Odd the class path reported in the yarn log contains jackson-core-2.2.3 >> and not 2.5.1. Is there a way to build myriad to match the version >> supported by HDP - that being 2.2.3 ? >> >> >> -- >> Jonathan (Bill) Sparks >> Software Architecture >> Cray Inc. >> >> >> >> >> >> On 8/19/15 10:11 AM, "Bill Sparks" <jspa...@cray.com> wrote: >> >> >Thanks I'll check.. >> > >> >-- >> >Jonathan (Bill) Sparks >> >Software Architecture >> >Cray Inc. >> > >> > >> > >> > >> > >> >On 8/19/15 10:09 AM, "yuliya Feldman" <yufeld...@yahoo.com.INVALID> >> wrote: >> > >> >>This method is part of JsonFactory class which is part of jackson-core >> >>jar >> >>See if you have some other jars on the classpath (different versions) >> >>that precede jackson-core-2.5.1.jar >> >> From: Bill Sparks <jspa...@cray.com> >> >> To: "dev@myriad.incubator.apache.org" >><dev@myriad.incubator.apache.org> >> >> Sent: Wednesday, August 19, 2015 7:08 AM >> >> Subject: myriad scheduler startup with HDP2.7 >> >> >> >>I'm sure this is been resolved, but I've been triaging why I'm getting >> >>the following error on resourcemanager startup. Everything on the >> >>configuration side looks correct, but I must have missed something. >> >> >> >> >> >> >> >>2015-08-19 08:53:04,718 FATAL >> >>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error >> >>starting ResourceManager >> >> >> >>java.lang.NoSuchMethodError: >> >>>>com.fasterxml.jackson.dataformat.yaml.YAMLFactory._decorate(Ljava/io/In >>>>pu >> >>t >> >>Stream;Lcom/fasterxml/jackson/core/io/IOContext;)Ljava/io/InputStream; >> >> >> >>at >> >>>>com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFact >>>>or >> >>y >> >>.java:299) >> >> >> >>at >> >>>>com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFact >>>>or >> >>y >> >>.java:14) >> >> >> >>at >> >>>>com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java >>>>:2 >> >>0 >> >>11) >> >> >> >>at com.ebay.myriad.Main.initialize(Main.java:70) >> >> >> >>at >> >>>>com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationIntercep >>>>to >> >>r >> >>.init(MyriadInitializationInterceptor.java:32) >> >> >> >>at >> >>>>com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(Co >>>>mp >> >>o >> >>siteInterceptor.java:76) >> >> >> >>at >> >>>>com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFa >>>>ir >> >>S >> >>cheduler.java:50) >> >> >> >>at >> >>>>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163 >>>>) >> >> >> >>at >> >>>>org.apache.hadoop.service.CompositeService.serviceInit(CompositeService >>>>.j >> >>a >> >>va:107) >> >> >> >>at >> >>>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveS >>>>er >> >>v >> >>ices.serviceInit(ResourceManager.java:572) >> >> >> >>at >> >>>>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163 >>>>) >> >> >> >>at >> >>>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAnd >>>>In >> >>i >> >>tActiveServices(ResourceManager.java:972) >> >> >> >>at >> >>>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceIn >>>>it >> >>( >> >>ResourceManager.java:259) >> >> >> >>at >> >>>>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163 >>>>) >> >> >> >>at >> >>>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(Reso >>>>ur >> >>c >> >>eManager.java:1202) >> >> >> >> >> >>I have placed all the myriad jar in the hadoop-yarn/lib directory and >>the >> >>classpath reflect that. >> >> >> >> >> >>cp /tmp/myriad/myriad-scheduler/build/libs/* >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib >> >> >> >>cp >> >>>>/tmp/myriad/myriad-executor/build/libs/myriad-executor-runnable-0.0.1.j >>>>ar >> >>/usr/libexec/mesos/ >> >> >> >> >> >>[root@nid00037 myriad]# su - yarn >> >> >> >>-bash-4.1$ yarn classpath >> >> >> >>>>/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/conf:/us >>>>r/ >> >>h >> >>>>dp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/lib/*:/usr/hdp >>>>/2 >> >>. >> >>>>3.0.0-2557/hadoop/.//*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/./:/usr/hdp/2. >>>>3. >> >>0 >> >>>>.0-2557/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/.//*:/usr/h >>>>dp >> >>/ >> >>>>2.3.0.0-2557/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/.//*:/ >>>>us >> >>r >> >>>>/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-m >>>>ap >> >>r >> >>>>educe/.//*::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/ >>>>ja >> >>v >> >>>>a/mysql-connector-java.jar:/usr/hdp/current/hadoop-mapreduce-client/*:/ >>>>us >> >>r >> >>>>/hdp/2.3.0.0-2557/tez/*:/usr/hdp/2.3.0.0-2557/tez/lib/*:/usr/hdp/2.3.0. >>>>0- >> >>2 >> >>>>557/tez/conf:/usr/hdp/current/hadoop-yarn-client/.//*:/usr/hdp/current/ >>>>ha >> >>d >> >>oop-yarn-client/lib/* >> >> >> >> >> >> >> >>ls /usr/hdp/current/hadoop-yarn-client/lib/* has all the libraries >> >> >> >> >> >>-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad* >> >> >> >>-rw-r--r-- 1 root root 3456 Aug 19 08:50 >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-commons-0.0.1.jar >> >> >> >>-rw-r--r-- 1 root root 950687 Aug 19 08:50 >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-scheduler-0.0.1.jar >> >> >> >>and >> >> >> >> >> >>-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson* >> >> >> >>-rw-r--r-- 1 root root 39817 Aug 17 18:32 >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-annotations-2.5.1.jar >> >> >> >>-rw-r--r-- 1 root root 192699 Jul 14 08:22 >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.2.3.jar >> >> >> >>-rw-r--r-- 1 root root 229860 Aug 17 18:32 >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.5.1.jar >> >> >> >>-rw-r--r-- 1 root root 232248 Jul 14 08:22 >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar >> >> >> >>-rw-r--r-- 1 root root 1138921 Aug 17 18:32 >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-databind-2.5.1.jar >> >> >> >>-rw-r--r-- 1 root root 321751 Aug 17 18:32 >> >>>>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-dataformat-yaml-2.5.1.jar >> >> >> >>-rw-r--r-- 1 root root 18336 Jul 14 08:22 >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar >> >> >> >>-rw-r--r-- 1 root root 780664 Jul 14 08:22 >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar >> >> >> >>-rw-r--r-- 1 root root 27084 Jul 14 08:22 >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-xc-1.9.13.jar >> >> >> >> >> >> >> >>-bash-4.1$ jar tf >> >>>>/usr/hdp/current/hadoop-yarn-client/lib/jackson-dataformat-yaml-2.5.1.j >>>>ar >> >> | grep YAMLFactory >> >> >> >>com/fasterxml/jackson/dataformat/yaml/YAMLFactory.class >> >> >> >> >> >> >> > >> >>
