Hey guys.. the flow of things is as below:
1. Myriad registers itself as a Mesos framework. (Frameworks view on Mesos
UI should show this)
2. Myriad by default attempts to launch 1 medium profile NM as soon as RM
process is up.
This is queued up until Myriad receives an offer from Mesos that is big
enough to launch a medium profle NM.
(Mesos' UI should show a task such as "nm.medium.xxxx" if Myriad
attempted to launch a NM).
Mayank - in your cluster, does Mesos UI show Myriad as a registered
framework? Does it show if Myriad launched NM tasks? Perhaps a good
starting point is to check what are the sizes of the offers Myriad is
receiving (Mesos' master log should show this). You can also enable DEBUG
logging in RM for myriad by setting "org.apache.myriad" to DEBUG in RM's
log4j.properties.
Santosh
On Mon, Feb 22, 2016 at 10:39 AM, Björn Hagemeier <[email protected]
> wrote:
> Hi Mayank,
>
> it seems you and I are having similar issues. Let's see whether they can
> be traced to the same causes.
>
>
> Best regards,
> Björn
>
> Am 22.02.2016 um 19:33 schrieb Mayank Bansal:
> > Hi,
> >
> > I am trying to use Hadoop 2.6.3 with Myriad origin/0.1.x
> >
> >
> > I have followed all the instructions in the wiki
> >
> > So the resource manager is up and i am able to launch the web UI of
> myriad
> > as well however whenever I try to use flex up it launches some tasks and
> > put them in to pending queue and after that nothing launches.
> >
> > I am attaching below the resourcemanager.out and log snippets.
> >
> > Please let me know if I am missing something.
> >
> > Thanks,
> >
> >
> > *I see some exceptions in resourcemanager.out files those are as follows*
> >
> >
> > Feb 22, 2016 6:21:42 PM
> > com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
> > resolve
> >
> > SEVERE: null
> >
> > java.lang.IllegalAccessException: Class
> > com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
> > can not access a member of class javax.ws.rs.core.Response with modifiers
> > "protected"
> >
> > at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:102)
> >
> > at java.lang.Class.newInstance(Class.java:436)
> >
> > at
> >
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8.resolve(WadlGeneratorJAXBGrammarGenerator.java:467)
> >
> > at
> >
> com.sun.jersey.server.wadl.WadlGenerator$ExternalGrammarDefinition.resolve(WadlGenerator.java:181)
> >
> > at
> >
> com.sun.jersey.server.wadl.ApplicationDescription.resolve(ApplicationDescription.java:81)
> >
> > at
> >
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator.attachTypes(WadlGeneratorJAXBGrammarGenerator.java:518)
> >
> > at
> > com.sun.jersey.server.wadl.WadlBuilder.generate(WadlBuilder.java:124)
> >
> > at
> >
> com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl.getApplication(WadlApplicationContextImpl.java:104)
> >
> > at
> >
> com.sun.jersey.server.impl.wadl.WadlResource.getWadl(WadlResource.java:89)
> >
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >
> > at java.lang.reflect.Method.invoke(Method.java:498)
> >
> > at
> >
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> >
> > at
> >
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
> >
> > at
> >
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> >
> > at
> >
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> >
> > at
> >
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> >
> > at
> >
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> >
> > at
> >
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> >
> > at
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> >
> > at
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
> >
> > at
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
> >
> > at
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
> >
> > at
> >
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> >
> > at
> >
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
> >
> > at
> >
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
> >
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> >
> > at
> >
> com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
> >
> > at
> >
> com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
> >
> > at
> >
> com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
> >
> > at
> >
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
> >
> > at
> >
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
> >
> > at
> > com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
> >
> > at
> >
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> >
> > at
> > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> >
> > at
> > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> >
> > at
> > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> >
> > at org.mortbay.jetty.Server.handle(Server.java:326)
> >
> > at
> > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> >
> > at
> >
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> >
> > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> >
> > at
> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> >
> > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> >
> > at
> >
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> >
> > at
> >
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> >
> >
> > Feb 22, 2016 6:21:42 PM
> > com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator
> > attachTypes
> >
> > INFO: Couldn't find JAX-B element for class javax.ws.rs.core.Response
> >
> > Feb 22, 2016 6:21:42 PM
> > com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator
> > attachTypes
> >
> > INFO: Couldn't find JAX-B element for class
> > org.apache.myriad.configuration.MyriadConfiguration
> >
> > Feb 22, 2016 6:21:42 PM
> > com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator
> > attachTypes
> >
> > INFO: Couldn't find JAX-B element for class
> > org.apache.myriad.api.model.GetSchedulerStateResponse
> >
> > Feb 22, 2016 6:21:42 PM
> > com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator
> > attachTypes
> >
> > INFO: Couldn't find JAX-B element for class
> > org.apache.myriad.api.model.FlexUpServiceRequest
> >
> > Feb 22, 2016 6:21:42 PM
> > com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
> > resolve
> >
> > SEVERE: null
> >
> > E0222 18:22:30.863994 47309 socket.hpp:174] Shutdown failed on fd=231:
> > Transport endpoint is not connected [107]
> >
> > E0222 18:25:19.055994 47309 socket.hpp:174] Shutdown failed on fd=231:
> > Transport endpoint is not connected [107]
> >
> > E0222 18:27:44.208027 47309 socket.hpp:174] Shutdown failed on fd=231:
> > Transport endpoint is not connected [107]
> >
> > *This is the resource manager logs*
> >
> > 2016-02-22 17:37:27,747 INFO org.apache.hadoop.ipc.Server: IPC Server
> > listener on 8033: starting
> >
> > 2016-02-22 17:39:31,266 INFO org.apache.myriad.api.ClustersResource:
> > Received flexup request. Profile: medium, Instances: 1, Constraints: null
> >
> > 2016-02-22 17:39:31,266 INFO
> org.apache.myriad.scheduler.MyriadOperations:
> > Adding 1 NM instances to cluster
> >
> > 2016-02-22 17:39:31,267 INFO org.apache.myriad.state.SchedulerState:
> Marked
> > taskId nm.medium.efd5f3b4-fe5f-4eed-bd19-6906696a22bc pending, size of
> > pending queue for nm is: 1
> >
> > 2016-02-22 17:47:25,450 INFO
> >
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler:
> > Release request cache is cleaned up
> >
> > 2016-02-22 18:22:22,093 INFO org.apache.myriad.api.ClustersResource:
> > Received flexup request. Profile: medium, Instances: 1, Constraints: null
> >
> > 2016-02-22 18:22:22,094 INFO
> org.apache.myriad.scheduler.MyriadOperations:
> > Adding 1 NM instances to cluster
> >
> > 2016-02-22 18:22:22,094 INFO org.apache.myriad.state.SchedulerState:
> Marked
> > taskId nm.medium.607ebf49-045b-44bb-8940-8e3bd8c115a8 pending, size of
> > pending queue for nm is: 2
> >
> >
> >
> > Thanks,
> >
>
>
> --
> Dipl.-Inform. Björn Hagemeier
> Federated Systems and Data
> Juelich Supercomputing Centre
> Institute for Advanced Simulation
>
> Phone: +49 2461 61 1584
> Fax : +49 2461 61 6656
> Email: [email protected]
> Skype: bhagemeier
> WWW : http://www.fz-juelich.de/jsc
>
> JSC is the coordinator of the
> John von Neumann Institute for Computing
> and member of the
> Gauss Centre for Supercomputing
>
>
> -------------------------------------------------------------------------------------
>
> -------------------------------------------------------------------------------------
> Forschungszentrum Juelich GmbH
> 52425 Juelich
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
>
> -------------------------------------------------------------------------------------
>
> -------------------------------------------------------------------------------------
>
>
