Zameer Manji created AURORA-1654:
------------------------------------
Summary: `getJobSummary` can result in exception
Key: AURORA-1654
URL: https://issues.apache.org/jira/browse/AURORA-1654
Project: Aurora
Issue Type: Story
Reporter: Zameer Manji
The following exception can occur from the `getJobSummary` API call from master:
{noformat}
W0329 23:21:31.865 [qtp1149120917-394167, LoggingInterceptor:78] Uncaught
exception while handling getJobSummary(mesos)
com.google.common.collect.Ordering$IncomparableValueException: Cannot compare
value: INIT
at
com.google.common.collect.ExplicitOrdering.rank(ExplicitOrdering.java:47)
~[guava-19.0.jar:na]
at
com.google.common.collect.ExplicitOrdering.compare(ExplicitOrdering.java:41)
~[guava-19.0.jar:na]
at
com.google.common.collect.ByFunctionOrdering.compare(ByFunctionOrdering.java:45)
~[guava-19.0.jar:na]
at
com.google.common.collect.CompoundOrdering.compare(CompoundOrdering.java:42)
~[guava-19.0.jar:na]
at com.google.common.collect.Ordering.max(Ordering.java:620)
~[guava-19.0.jar:na]
at com.google.common.collect.Ordering.max(Ordering.java:587)
~[guava-19.0.jar:na]
at com.google.common.collect.Ordering.max(Ordering.java:603)
~[guava-19.0.jar:na]
at
org.apache.aurora.scheduler.base.Tasks.getLatestActiveTask(Tasks.java:133)
~[aurora-155.jar:na]
at
org.apache.aurora.scheduler.thrift.ReadOnlySchedulerImpl.lambda$getJobs$11(ReadOnlySchedulerImpl.java:417)
~[aurora-155.jar:na]
at com.google.common.collect.Maps$10.getValue(Maps.java:1913)
~[guava-19.0.jar:na]
at java.util.HashMap.putMapEntries(HashMap.java:513)
~[na:1.8.0_72-Tw8r10b0]
at java.util.HashMap.putAll(HashMap.java:784) ~[na:1.8.0_72-Tw8r10b0]
at
org.apache.aurora.scheduler.thrift.ReadOnlySchedulerImpl.getJobs(ReadOnlySchedulerImpl.java:411)
~[aurora-155.jar:na]
at
org.apache.aurora.scheduler.thrift.ReadOnlySchedulerImpl.getJobSummary(ReadOnlySchedulerImpl.java:248)
~[aurora-155.jar:na]
at
org.apache.aurora.scheduler.thrift.SchedulerThriftInterface.getJobSummary(SchedulerThriftInterface.java:399)
~[aurora-155.jar:na]
at
org.apache.aurora.scheduler.thrift.aop.ThriftStatsExporterInterceptor.invoke(ThriftStatsExporterInterceptor.java:47)
~[aurora-155.jar:na]
at
org.apache.aurora.scheduler.thrift.aop.LoggingInterceptor.invoke(LoggingInterceptor.java:70)
~[aurora-155.jar:na]
at
org.apache.aurora.scheduler.thrift.aop.ServerInfoInterceptor.invoke(ServerInfoInterceptor.java:30)
~[aurora-155.jar:na]
at
org.apache.aurora.gen.ReadOnlyScheduler$Processor$getJobSummary.getResult(ReadOnlyScheduler.java:971)
[aurora-api-155.jar:na]
at
org.apache.aurora.gen.ReadOnlyScheduler$Processor$getJobSummary.getResult(ReadOnlyScheduler.java:956)
[aurora-api-155.jar:na]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
[libthrift-0.9.1.jar:0.9.1]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
[libthrift-0.9.1.jar:0.9.1]
at org.apache.thrift.server.TServlet.doPost(TServlet.java:83)
[libthrift-0.9.1.jar:0.9.1]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
[javax.servlet-api-3.1.0.jar:3.1.0]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
[javax.servlet-api-3.1.0.jar:3.1.0]
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
[guice-servlet-3.0.jar:na]
at
org.apache.aurora.scheduler.http.LeaderRedirectFilter.doFilter(LeaderRedirectFilter.java:72)
[aurora-155.jar:na]
at
org.apache.aurora.scheduler.http.AbstractFilter.doFilter(AbstractFilter.java:44)
[aurora-155.jar:na]
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
[guice-servlet-3.0.jar:na]
at
org.apache.aurora.scheduler.http.HttpStatsFilter.doFilter(HttpStatsFilter.java:71)
[aurora-155.jar:na]
at
org.apache.aurora.scheduler.http.AbstractFilter.doFilter(AbstractFilter.java:44)
[aurora-155.jar:na]
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
[guice-servlet-3.0.jar:na]
at
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
[guice-servlet-3.0.jar:na]
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
[guice-servlet-3.0.jar:na]
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
[jetty-servlet-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
[jetty-servlet-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1158)
[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
[jetty-servlet-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1090)
[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:318)
[jetty-rewrite-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:437)
[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106]
at org.eclipse.jetty.server.Server.handle(Server.java:517)
[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242)
[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
[jetty-io-9.3.6.v20151106.jar:9.3.6.v20151106]
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[jetty-io-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75)
[jetty-io-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213)
[jetty-util-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147)
[jetty-util-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
[jetty-util-9.3.6.v20151106.jar:9.3.6.v20151106]
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
[jetty-util-9.3.6.v20151106.jar:9.3.6.v20151106]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72-Tw8r10b0]
{noformat}
A brief investigation shows that the {{Tasks}} utility class has a bug where
the ordering for task status is not exhaustive because it doesn't cover the
{{INIT}} state:
{noformat}
static final List<ScheduleStatus> ORDERED_TASK_STATUSES =
ImmutableList.<ScheduleStatus>builder()
.addAll(apiConstants.TERMINAL_STATES)
.addAll(apiConstants.ACTIVE_STATES)
.build();
{noformat}
This means fetching the summary for a role that has tasks in the {{INIT}} state
(just submitted by a user) will cause an exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
