Can you please check the "echo" method that I mentioned?
Please check the code at: http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java and search for "ph1_coord_formatTime_echo". Also check how it is defined in ooze-default.xml. That means you need to follow the same pattern that we used for formatTime(...) ________________________________ From: Eduardo Afonso Ferreira <[email protected]> To: "[email protected]" <[email protected]> Sent: Thursday, November 15, 2012 2:46 PM Subject: Re: Creating/implementing custom EL Functions. Hey, I added those three but the result appears to be the same. Here's the stack trace I see in oozie.log. It looks like it does not resolve coord:nominalTime() but just passes that as a string to the function call. Notice the "Caused by" towards the end of the stack trace: ..... Caused by: java.text.ParseException: Unparseable date: "coord:nominalTime()" ..... Here's the stack trace: 2012-11-15 17:40:12,213 WARN V1JobsServlet:542 - USER[aspen] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST http://localhost:11000/oozie/v1/jobs] error[E1004], E1004: Expression language evaluation error [U nable to evaluate :${coord:formatTimeTZ(coord:nominalTime(), -FREQUENCY-OFFSET, "'min-'yyyyMMddHHmm")}: ], java.lang.Exception: Unable to evaluate :${coord:formatTimeTZ(coord:nominalTime(), -FREQUENCY-OFFSET, "'min-'yyyyMMddHHmm")}: org.apache.oozie.servlet.XServletException: E1004: Expression language evaluation error [Unable to evaluate :${coord:formatTimeTZ(coord:nominalTime(), -FREQUENCY-OFFSET, "'min-'yyyyMMddHHmm")}: ], java.lang.Exception: Unable to evaluate :${coord:formatTimeTZ(coord:nominalTime(), -FREQUENCY-OFFSET, "'min-'yyyyMMddHHmm")}: at org.apache.oozie.servlet.V1JobsServlet.submitCoordinatorJob(V1JobsServlet.java:205) at org.apache.oozie.servlet.V1JobsServlet.submitJob(V1JobsServlet.java:80) at org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:95) at javax.servlet.http.HttpServlet.service(HttpServlet.java:637) at org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:285) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:126) at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:372) at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:131) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:67) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:662) Caused by: org.apache.oozie.CoordinatorEngineException: E1004: Expression language evaluation error [Unable to evaluate :${coord:formatTimeTZ(coord:nominalTime(), -FREQUENCY-OFFSET, "'min-'yyyyMMddHHmm")}: ], java.lang.Exception: Unable to evaluate :${coord:formatTimeTZ(coord:nominalTime(), -FREQUENCY-OFFSET, "'min-'yyyyMMddHHmm")}: at org.apache.oozie.CoordinatorEngine.submitJob(CoordinatorEngine.java:374) at org.apache.oozie.servlet.V1JobsServlet.submitCoordinatorJob(V1JobsServlet.java:200) ... 25 more Caused by: org.apache.oozie.command.CommandException: E1004: Expression language evaluation error [Unable to evaluate :${coord:formatTimeTZ(coord:nominalTime(), -FREQUENCY-OFFSET, "'min-'yyyyMMddHHmm")}: ], java.lang.Exception: Unable to evaluate :${coord:formatTimeTZ(coord:nominalTime(), -FREQUENCY-OFFSET, "'min-'yyyyMMddHHmm")}: at org.apache.oozie.command.coord.CoordSubmitXCommand.submit(CoordSubmitXCommand.java:272) at org.apache.oozie.command.SubmitTransitionXCommand.execute(SubmitTransitionXCommand.java:81) at org.apache.oozie.command.SubmitTransitionXCommand.execute(SubmitTransitionXCommand.java:29) at org.apache.oozie.command.XCommand.call(XCommand.java:277) at org.apache.oozie.CoordinatorEngine.submitJob(CoordinatorEngine.java:371) ... 26 more Caused by: org.apache.oozie.coord.CoordinatorJobException: E1004: Expression language evaluation error [Unable to evaluate :${coord:formatTimeTZ(coord:nominalTime(), -FREQUENCY-OFFSET, "'min-'yyyyMMddHHmm")}: ], java.lang.Exception: Unable to evaluate :${coord:formatTimeTZ(coord:nominalTime(), -FREQUENCY-OFFSET, "'min-'yyyyMMddHHmm")}: at org.apache.oozie.command.coord.CoordSubmitXCommand.resolveTagContents(CoordSubmitXCommand.java:823) at org.apache.oozie.command.coord.CoordSubmitXCommand.resolveInitial(CoordSubmitXCommand.java:663) at org.apache.oozie.command.coord.CoordSubmitXCommand.basicResolveAndIncludeDS(CoordSubmitXCommand.java:484) at org.apache.oozie.command.coord.CoordSubmitXCommand.submit(CoordSubmitXCommand.java:222) ... 30 more Caused by: java.lang.Exception: Unable to evaluate :${coord:formatTimeTZ(coord:nominalTime(), -FREQUENCY-OFFSET, "'min-'yyyyMMddHHmm")}: at org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:551) at org.apache.oozie.command.coord.CoordSubmitXCommand.resolveTagContents(CoordSubmitXCommand.java:819) ... 33 more Caused by: java.text.ParseException: Unparseable date: "coord:nominalTime()" at java.text.DateFormat.parse(DateFormat.java:337) at com.turner.util.Utils.formatTimeTZ(Unknown Source) at com.turner.util.Utils.formatTimeTZ(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java:172) at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:263) at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190) at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:203) at org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:542) ... 34 more ________________________________ From: Mohammad Islam <[email protected]> To: "[email protected]" <[email protected]>; Eduardo Afonso Ferreira <[email protected]> Sent: Thursday, November 15, 2012 4:45 PM Subject: Re: Creating/implementing custom EL Functions. Hi eduardo, You have to add the similar thing in three other properties. Please search for "formatTime" in ooze-default.xml. Coordinator EL functions are evaluated in different contexts (multiple phases). Other cases, it called "*echo" method such as "coord_formatTime_echo". For example, during job submission value nominalTime is not defined. <name>oozie.service.ELService.ext.functions.coord-action-create</name> and oozie.service.ELService.ext.functions.coord-action-create-inst oozie.service.ELService.ext.functions.coord-action-start Regards, Mohammad ________________________________ From: Eduardo Afonso Ferreira <[email protected]> To: "[email protected]" <[email protected]> Sent: Thursday, November 15, 2012 1:10 PM Subject: Re: Creating/implementing custom EL Functions. Hey, I'm trying to do some date manipulation via custom EL functions. I added a function coord:formatTimeTZ() very similar to the original coord:formatTime() but the new one will take an offset and optionally a timezone. Here's an example of usage (below). The problem I see now is when I use only the original coord functions, all functions are called and processed fine, but when I use my new function (formatTimeTZ), the attribute coord:nominalTime() is not called, but it simply passes the string "coord:nominalTime()" to the function. Did I miss anything? <property> <name>START_ROW_DT</name> <value>${coord:dateOffset(coord:nominalTime(), -FREQUENCY-OFFSET, 'MINUTE')}</value> </property> <property> <name>START_ROW_FORM_1</name> <value>${coord:formatTime(coord:dateOffset(coord:nominalTime(), -FREQUENCY-OFFSET, 'MINUTE'), "'min-'yyyyMMddHHmm")}</value> </property> <property> <name>START_ROW_FORM_2</name> <value>${coord:formatTimeTZ(coord:nominalTime(), -FREQUENCY-OFFSET, "'min-'yyyyMMddHHmm")}</value> </property> NOTE: Here's what I added to oozie-site.xml to define the new function. I also added my new jar to oozie.war with oozie setup. <property> <name>oozie.service.ELService.ext.functions.coord-job-submit-data</name> <value> coord:formatTimeTZ=com.turner.util.Utils#formatTimeTZ </value> <description> EL functions declarations, separated by commas, format is [PREFIX:]NAME=CLASS#METHOD. </description> </property> ________________________________ From: Harsh J <[email protected]> To: [email protected]; Eduardo Afonso Ferreira <[email protected]> Sent: Wednesday, November 14, 2012 3:10 PM Subject: Re: Creating/implementing custom EL Functions. Hey Eduardo, There is lack of docs in this area but you can certainly plug in custom EL functions and give them aliases. Look at all the configs prefixed "oozie.service.ELService.ext.functions" in http://oozie.apache.org/docs/3.2.0-incubating/oozie-default.xml. You can create your own function that way by mapping a name to a class plus method call. The class needs to be in the Oozie deployed WAR though, so this isn't a dynamic thing. See the examples around these config entries in the link for the inbuilt definitions we carry. I'd also strongly recommend contributing back nifty EL extensions, so that its not a maintenance pain in the long run for you guys - and others can use it as well :) On Thu, Nov 15, 2012 at 12:58 AM, Eduardo Afonso Ferreira <[email protected]> wrote: > Hey, > > Is it possible to implement custom EL Functions without having to change > Oozie source code? > I did not find documentation about how to implement custom EL Functions. > Would someone throw some links my way about where I can find information on > this? > > Thank you. > Eduardo. -- Harsh J
