[
https://issues.apache.org/jira/browse/OOZIE-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473564#comment-13473564
]
Virag Kothari commented on OOZIE-1012:
--------------------------------------
looks good... If you want the stats to be exposed also through CLI, webService
or Java Api's, you can add logic similar to that in MapReduceActionExecutor
{code}
// If action stats write property is set to false by user or
// size of stats is greater than the maximum allowed size,
// do not store the action stats
if
(Boolean.parseBoolean(evaluateConfigurationProperty(actionXml,
OOZIE_ACTION_EXTERNAL_STATS_WRITE, "false"))
&& (statsJsonString.getBytes().length <=
getMaxExternalStatsSize())) {
context.setExecutionStats(statsJsonString);
log.debug(
"Printing stats for Map-Reduce action as a JSON
string : [{0}]" + statsJsonString);
}
{code}
> Sqoop jobs are unable to utilize Hadoop Counters
> ------------------------------------------------
>
> Key: OOZIE-1012
> URL: https://issues.apache.org/jira/browse/OOZIE-1012
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 3.1.3
> Reporter: Kevin Odell
> Assignee: Jarek Jarcec Cecho
> Attachments: OOZIE-1012-sqoop-counters.patch
>
>
> Sqoop actions are unable to use the Hadoop Counters, I believe it is because
> they point to the Oozie Launcher, where MR jobs point to the actual action so
> Hadoop Counters is able to run.
> {code}
> 2012-09-24 15:18:31,998 WARN org.apache.oozie.command.wf.ActionStartXCommand:
> USER[nbadmin] GROUP[-] TOKEN[] APP[1-user-wf] JOB[0000500-120911100707873-o
> ozie-oozi-W]
> ACTION[0000500-120911100707873-oozie-oozi-W@writeJobInputCountersToRDBMS]
> Exception in ActionStartXCommand
> java.lang.IllegalArgumentException: Hadoop counters not available for action
> [importACSUserData]
> at
> org.apache.oozie.action.hadoop.HadoopELFunctions.getCounters(HadoopELFunctions.java:65)
>
> at
> org.apache.oozie.action.hadoop.HadoopELFunctions.hadoop_counters(HadoopELFunctions.java:50)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java:172)
>
> at org.apache.commons.el.ComplexValue.evaluate(ComplexValue.java:140)
> at org.apache.commons.el.ExpressionString.evaluate(ExpressionString.java:114)
> at
> org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:274)
>
> at
> org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190)
>
> at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:203)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:157)
>
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:55)
>
> at org.apache.oozie.command.XCommand.call(XCommand.java:260)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:300)
>
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:247)
>
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:166)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:662)
> 2012-09-24 15:18:31,998 WARN org.apache.oozie.command.wf.ActionStartXCommand:
> USER[nbadmin] GROUP[-] TOKEN[] APP[1-user-wf]
> JOB[0000500-120911100707873-oozie-oozi-W]
> ACTION[0000500-120911100707873-oozie-oozi-W@writeJobInputCountersToRDBMS]
> Failing Job due to failed action [writeJobInputCountersToRDBMS]
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
