[
https://issues.apache.org/jira/browse/FALCON-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14716521#comment-14716521
]
Jaydeep Vishwakarma commented on FALCON-1411:
---------------------------------------------
Yes, dryrun does not wait for input data, But it does synchronous call.
Everytime it generates the new path by jobs frequency and it checks the same
path in hdfs till it find the path or it reaches to initial instance.
When you have initial instance very old and path does not exist or belongs to
very old instance , you might experience the slowness.
> El expressions not getting evaluated correctly with falcon-0.7 jar
> ------------------------------------------------------------------
>
> Key: FALCON-1411
> URL: https://issues.apache.org/jira/browse/FALCON-1411
> Project: Falcon
> Issue Type: Bug
> Components: oozie
> Affects Versions: 0.7
> Environment: QA
> Reporter: Pragya Mittal
> Assignee: Pallavi Rao
>
> Following El-expressions are broken :
> 1. currentWeek and lastWeek
> While scheduling a process containing currentWeek/lastWeek el expression it
> gets scheduled but instance keeps on waiting (although there are no missing
> dependency)
> Process -definition is :
> {code}
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <process name="Adda4dfb0-5eb92594" xmlns="uri:falcon:process:0.1">
> <clusters>
> <cluster name="Adda4dfb0-a611389d">
> <validity start="2015-02-17T10:30Z" end="2015-02-17T10:50Z"/>
> </cluster>
> </clusters>
> <parallel>1</parallel>
> <order>FIFO</order>
> <frequency>minutes(5)</frequency>
> <timezone>UTC</timezone>
> <inputs>
> <input name="inputData" feed="Adda4dfb0-04fe993c"
> start="currentWeek('WED',2,15)" end="currentWeek('WED',2,25)"/>
> </inputs>
> <outputs>
> <output name="outputData" feed="Adda4dfb0-68dd6cce"
> instance="now(0,0)"/>
> </outputs>
> <properties>
> <property name="queueName" value="default"/>
> <property name="fileTime"
> value="${formatTime(dateOffset(instanceTime(), 1, 'DAY'), 'yyyy-MMM-dd')}"/>
> <property name="user" value="${user()}"/>
> </properties>
> <workflow
> path="/tmp/falcon-regression/ELExpCurrentAndLastWeekTest/aggregator"/>
> <retry policy="periodic" delay="minutes(3)" attempts="3"/>
> <ACL owner="pragya" group="dataqa" permission="*"/>
> </process>
> {code}
> Oozie stack-trace is as follows :
> {code}
> 2015-08-19 12:34:57,199 WARN pool-6-thread-2
> CallableQueueService$CompositeCallable - SERVER[ip-192-168-138-200] USER[-]
> GROUP[-] TOKEN[-] APP[-] JOB[0000002-150819121241197-oozie-oozi-C]
> ACTION[0000002-150819121241197-oozie-oozi-C@4] exception callable
> [coord_action_input], E1021: Coord Action Input Check Error: Unable to
> evaluate :${dataIn('inputData', 'null')}:
> org.apache.oozie.command.CommandException: E1021: Coord Action Input Check
> Error: Unable to evaluate :${dataIn('inputData', 'null')}:
> at
> org.apache.oozie.command.coord.CoordActionInputCheckXCommand.execute(CoordActionInputCheckXCommand.java:279)
> at
> org.apache.oozie.command.coord.CoordActionInputCheckXCommand.execute(CoordActionInputCheckXCommand.java:71)
> at org.apache.oozie.command.XCommand.call(XCommand.java:286)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:321)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:250)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Unable to evaluate :${dataIn('inputData',
> 'null')}:
> at
> org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:723)
> at
> org.apache.oozie.command.coord.CoordActionInputCheckXCommand.resolveTagContents(CoordActionInputCheckXCommand.java:398)
> at
> org.apache.oozie.command.coord.CoordActionInputCheckXCommand.materializeDataProperties(CoordActionInputCheckXCommand.java:379)
> at
> org.apache.oozie.command.coord.CoordActionInputCheckXCommand.resolveCoordConfiguration(CoordActionInputCheckXCommand.java:288)
> at
> org.apache.oozie.command.coord.CoordActionInputCheckXCommand.execute(CoordActionInputCheckXCommand.java:230)
> ... 8 more
> Caused by: javax.servlet.jsp.el.ELException: An error occurred while
> evaluating function "dataIn"
> at org.apache.commons.el.Logger.logError(Logger.java:484)
> at org.apache.commons.el.Logger.logError(Logger.java:545)
> at
> org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java:174)
> at
> org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:263)
> at
> org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190)
> at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:204)
> at
> org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:714)
> ... 12 more
> {code}
> Oozie coordinator.xml contains the following property :
> {code}
> <property>
> <name>falconInPaths</name>
> <value>${dataIn('inputData', 'null')}</value>
> </property>
> {code}
> 2. future and latest
> While scheduling a process containing future/latest el expression it gets
> submitted but fails on scheduling.
> Falcon stack trace shows falcon is not able to communicate with oozie and
> gives socket timeout exception .
> {code}
> 2015-08-19 12:31:02,527 ERROR - [1768191773@qtp-351417028-14 -
> 25e8c020-17ed-4d56-85e7-f5ded1f6d442:pragya:POST//entities/schedule/process/Ac21dad17-c7ef19c7]
> ~ Unable to schedule workflow (AbstractSchedulableEntityManager:75)
> org.apache.falcon.FalconException: Entity schedule failed for process:
> Ac21dad17-c7ef19c7
> at
> org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:95)
> at
> org.apache.falcon.resource.AbstractSchedulableEntityManager.schedule(AbstractSchedulableEntityManager.java:72)
> at
> org.apache.falcon.resource.SchedulableEntityManager.schedule(SchedulableEntityManager.java:130)
> at sun.reflect.GeneratedMethodAccessor47.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> at
> org.apache.falcon.security.FalconAuthorizationFilter.doFilter(FalconAuthorizationFilter.java:106)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.falcon.security.FalconAuthenticationFilter$2.doFilter(FalconAuthenticationFilter.java:184)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:396)
> at
> org.apache.falcon.security.FalconAuthenticationFilter.doFilter(FalconAuthenticationFilter.java:193)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.falcon.security.FalconAuditFilter.doFilter(FalconAuditFilter.java:64)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> Caused by: org.apache.falcon.FalconException: IO_ERROR :
> java.net.SocketTimeoutException: Read timed out
> at
> org.apache.falcon.workflow.engine.OozieWorkflowEngine.dryRunInternal(OozieWorkflowEngine.java:223)
> at
> org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:171)
> at
> org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:93)
> ... 45 more
> Caused by: IO_ERROR : java.net.SocketTimeoutException: Read timed out
> at
> org.apache.oozie.client.OozieClient$ClientCallable.call(OozieClient.java:523)
> at org.apache.oozie.client.OozieClient.dryrun(OozieClient.java:735)
> at
> org.apache.oozie.client.ProxyOozieClient.access$401(ProxyOozieClient.java:48)
> at
> org.apache.oozie.client.ProxyOozieClient$4.call(ProxyOozieClient.java:177)
> at
> org.apache.oozie.client.ProxyOozieClient$4.call(ProxyOozieClient.java:174)
> at org.apache.oozie.client.OozieClient.doAs(OozieClient.java:198)
> at
> org.apache.oozie.client.ProxyOozieClient.dryrun(ProxyOozieClient.java:174)
> at
> org.apache.falcon.workflow.engine.OozieWorkflowEngine.dryRunInternal(OozieWorkflowEngine.java:221)
> ... 47 more
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
