Hi all, Am 23.02.2016 um 12:55 schrieb Björn Hagemeier: > Hi Santosh, > > Am 23.02.2016 um 01:21 schrieb Santosh Marella: >> Hey guys.. the flow of things is as below: >> >> 1. Myriad registers itself as a Mesos framework. (Frameworks view on Mesos >> UI should show this) does Myriad register itself as a Mesos framework automatically, just by starting the RM as one would do it without Mesos? Or do I need to start the RM as a Mesos framework to make the magic happen, i.e. by "mesos execute"?
In the latter case, I think I found out why my Myriad does not seem to do anything. I did not start the resource manager as a Mesos framework. I somehow reckoned, the myriad integration would take care of that by itself. When I do this, I do see the framework, but with classpath issues (missing myriad for some reason). I'll keep digging in that direction. Cheers, Björn > I think this is where it is already failing for me. I do not see the > Myriad framework in the UI. >> 2. Myriad by default attempts to launch 1 medium profile NM as soon as RM >> process is up. >> This is queued up until Myriad receives an offer from Mesos that is big >> enough to launch a medium profle NM. > Yes, I do see the medium profile NM in the Myriad UI. >> (Mesos' UI should show a task such as "nm.medium.xxxx" if Myriad >> attempted to launch a NM). > Nope, unfortunately it does not. > > What makes me wonder is that I do have a running Myriad, i.e. I can > access the Myriad UI on port 8192, but it does not show up in Mesos. > > I guess my other problem, not seeing any offers, is a follow-on of this. > >> >> Mayank - in your cluster, does Mesos UI show Myriad as a registered >> framework? Does it show if Myriad launched NM tasks? Perhaps a good >> starting point is to check what are the sizes of the offers Myriad is >> receiving (Mesos' master log should show this). You can also enable DEBUG >> logging in RM for myriad by setting "org.apache.myriad" to DEBUG in RM's >> log4j.properties. > I will try and increase the log level to DEBUG. > > > Cheers, > Björn >> >> >> >> Santosh >> >> On Mon, Feb 22, 2016 at 10:39 AM, Björn Hagemeier <[email protected] >>> wrote: >> >>> Hi Mayank, >>> >>> it seems you and I are having similar issues. Let's see whether they can >>> be traced to the same causes. >>> >>> >>> Best regards, >>> Björn >>> >>> Am 22.02.2016 um 19:33 schrieb Mayank Bansal: >>>> Hi, >>>> >>>> I am trying to use Hadoop 2.6.3 with Myriad origin/0.1.x >>>> >>>> >>>> I have followed all the instructions in the wiki >>>> >>>> So the resource manager is up and i am able to launch the web UI of >>> myriad >>>> as well however whenever I try to use flex up it launches some tasks and >>>> put them in to pending queue and after that nothing launches. >>>> >>>> I am attaching below the resourcemanager.out and log snippets. >>>> >>>> Please let me know if I am missing something. >>>> >>>> Thanks, >>>> >>>> >>>> *I see some exceptions in resourcemanager.out files those are as follows* >>>> >>>> >>>> Feb 22, 2016 6:21:42 PM >>>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8 >>>> resolve >>>> >>>> SEVERE: null >>>> >>>> java.lang.IllegalAccessException: Class >>>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8 >>>> can not access a member of class javax.ws.rs.core.Response with modifiers >>>> "protected" >>>> >>>> at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:102) >>>> >>>> at java.lang.Class.newInstance(Class.java:436) >>>> >>>> at >>>> >>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8.resolve(WadlGeneratorJAXBGrammarGenerator.java:467) >>>> >>>> at >>>> >>> com.sun.jersey.server.wadl.WadlGenerator$ExternalGrammarDefinition.resolve(WadlGenerator.java:181) >>>> >>>> at >>>> >>> com.sun.jersey.server.wadl.ApplicationDescription.resolve(ApplicationDescription.java:81) >>>> >>>> at >>>> >>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator.attachTypes(WadlGeneratorJAXBGrammarGenerator.java:518) >>>> >>>> at >>>> com.sun.jersey.server.wadl.WadlBuilder.generate(WadlBuilder.java:124) >>>> >>>> at >>>> >>> com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl.getApplication(WadlApplicationContextImpl.java:104) >>>> >>>> at >>>> >>> com.sun.jersey.server.impl.wadl.WadlResource.getWadl(WadlResource.java:89) >>>> >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> >>>> at >>>> >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>> >>>> at >>>> >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>> >>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>> >>>> at >>>> >>> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) >>>> >>>> at >>>> >>> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) >>>> >>>> at >>>> >>> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) >>>> >>>> at >>>> >>> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288) >>>> >>>> at >>>> >>> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) >>>> >>>> at >>>> >>> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) >>>> >>>> at >>>> >>> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) >>>> >>>> at >>>> >>> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469) >>>> >>>> at >>>> >>> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400) >>>> >>>> at >>>> >>> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349) >>>> >>>> at >>>> >>> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339) >>>> >>>> at >>>> >>> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416) >>>> >>>> at >>>> >>> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537) >>>> >>>> at >>>> >>> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699) >>>> >>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) >>>> >>>> at >>>> >>> com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263) >>>> >>>> at >>>> >>> com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178) >>>> >>>> at >>>> >>> com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91) >>>> >>>> at >>>> >>> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62) >>>> >>>> at >>>> >>> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118) >>>> >>>> at >>>> com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113) >>>> >>>> at >>>> >>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) >>>> >>>> at >>>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) >>>> >>>> at >>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) >>>> >>>> at >>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) >>>> >>>> at org.mortbay.jetty.Server.handle(Server.java:326) >>>> >>>> at >>>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) >>>> >>>> at >>>> >>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) >>>> >>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) >>>> >>>> at >>> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) >>>> >>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) >>>> >>>> at >>>> >>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) >>>> >>>> at >>>> >>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) >>>> >>>> >>>> Feb 22, 2016 6:21:42 PM >>>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator >>>> attachTypes >>>> >>>> INFO: Couldn't find JAX-B element for class javax.ws.rs.core.Response >>>> >>>> Feb 22, 2016 6:21:42 PM >>>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator >>>> attachTypes >>>> >>>> INFO: Couldn't find JAX-B element for class >>>> org.apache.myriad.configuration.MyriadConfiguration >>>> >>>> Feb 22, 2016 6:21:42 PM >>>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator >>>> attachTypes >>>> >>>> INFO: Couldn't find JAX-B element for class >>>> org.apache.myriad.api.model.GetSchedulerStateResponse >>>> >>>> Feb 22, 2016 6:21:42 PM >>>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator >>>> attachTypes >>>> >>>> INFO: Couldn't find JAX-B element for class >>>> org.apache.myriad.api.model.FlexUpServiceRequest >>>> >>>> Feb 22, 2016 6:21:42 PM >>>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8 >>>> resolve >>>> >>>> SEVERE: null >>>> >>>> E0222 18:22:30.863994 47309 socket.hpp:174] Shutdown failed on fd=231: >>>> Transport endpoint is not connected [107] >>>> >>>> E0222 18:25:19.055994 47309 socket.hpp:174] Shutdown failed on fd=231: >>>> Transport endpoint is not connected [107] >>>> >>>> E0222 18:27:44.208027 47309 socket.hpp:174] Shutdown failed on fd=231: >>>> Transport endpoint is not connected [107] >>>> >>>> *This is the resource manager logs* >>>> >>>> 2016-02-22 17:37:27,747 INFO org.apache.hadoop.ipc.Server: IPC Server >>>> listener on 8033: starting >>>> >>>> 2016-02-22 17:39:31,266 INFO org.apache.myriad.api.ClustersResource: >>>> Received flexup request. Profile: medium, Instances: 1, Constraints: null >>>> >>>> 2016-02-22 17:39:31,266 INFO >>> org.apache.myriad.scheduler.MyriadOperations: >>>> Adding 1 NM instances to cluster >>>> >>>> 2016-02-22 17:39:31,267 INFO org.apache.myriad.state.SchedulerState: >>> Marked >>>> taskId nm.medium.efd5f3b4-fe5f-4eed-bd19-6906696a22bc pending, size of >>>> pending queue for nm is: 1 >>>> >>>> 2016-02-22 17:47:25,450 INFO >>>> >>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler: >>>> Release request cache is cleaned up >>>> >>>> 2016-02-22 18:22:22,093 INFO org.apache.myriad.api.ClustersResource: >>>> Received flexup request. Profile: medium, Instances: 1, Constraints: null >>>> >>>> 2016-02-22 18:22:22,094 INFO >>> org.apache.myriad.scheduler.MyriadOperations: >>>> Adding 1 NM instances to cluster >>>> >>>> 2016-02-22 18:22:22,094 INFO org.apache.myriad.state.SchedulerState: >>> Marked >>>> taskId nm.medium.607ebf49-045b-44bb-8940-8e3bd8c115a8 pending, size of >>>> pending queue for nm is: 2 >>>> >>>> >>>> >>>> Thanks, >>>> >>> >>> >>> -- >>> Dipl.-Inform. Björn Hagemeier >>> Federated Systems and Data >>> Juelich Supercomputing Centre >>> Institute for Advanced Simulation >>> >>> Phone: +49 2461 61 1584 >>> Fax : +49 2461 61 6656 >>> Email: [email protected] >>> Skype: bhagemeier >>> WWW : http://www.fz-juelich.de/jsc >>> >>> JSC is the coordinator of the >>> John von Neumann Institute for Computing >>> and member of the >>> Gauss Centre for Supercomputing >>> >>> >>> ------------------------------------------------------------------------------------- >>> >>> ------------------------------------------------------------------------------------- >>> Forschungszentrum Juelich GmbH >>> 52425 Juelich >>> Sitz der Gesellschaft: Juelich >>> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 >>> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher >>> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), >>> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, >>> Prof. Dr. Sebastian M. Schmidt >>> >>> ------------------------------------------------------------------------------------- >>> >>> ------------------------------------------------------------------------------------- >>> >>> >> > > -- Dipl.-Inform. Björn Hagemeier Federated Systems and Data Juelich Supercomputing Centre Institute for Advanced Simulation Phone: +49 2461 61 1584 Fax : +49 2461 61 6656 Email: [email protected] Skype: bhagemeier WWW : http://www.fz-juelich.de/jsc JSC is the coordinator of the John von Neumann Institute for Computing and member of the Gauss Centre for Supercomputing ------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------- Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------
<<attachment: b_hagemeier.vcf>>
smime.p7s
Description: S/MIME Cryptographic Signature
