Hi Steve, I'm not sure how to process log output, but oozie do have the ability to work with counters. There is EL function hadoop:counters(action name) that will return all counters for given action. You can find examples in the documentation:
http://incubator.apache.org/oozie/docs/3.2.0-incubating/docs/WorkflowFunctionalSpec.html Those counters are available even for the sqoop action if I remember correctly. Jarcec On Tue, Jul 03, 2012 at 04:29:38PM -0400, [email protected] wrote: > Does anyone know how to make use of log output from one workflow action in a > following action? > Specifically, I have a sqoop action to import data into the HDFS. Following > that action (if ok) I want to make use of information like rows/bytes > imported in the sqoop action. I can, via the oozie console, look at the log > file for the sqoop action and scrolling down, I see something like: > > Job complete: job_201205212157_421694 > Counters: 12 > Job Counters > SLOTS_MILLIS_MAPS=4794 > Total time spent by all reduces waiting after reserving slots (ms)=0 > Total time spent by all maps waiting after reserving slots (ms)=0 > Launched map tasks=1 > SLOTS_MILLIS_REDUCES=0 > FileSystemCounters > HDFS_BYTES_READ=87 > FILE_BYTES_WRITTEN=79795 > HDFS_BYTES_WRITTEN=95560 > Map-Reduce Framework > Map input records=1000 > Spilled Records=0 > Map output records=1000 > SPLIT_RAW_BYTES=87 > > So my thought is, to have a follow up java action, pass it the path to the > log file, and have the java class extract the data I want, and update a > results table. > I also considered whether I could direct the sqoop action to output > information to a properties file. Oozie does NOT like the <capture-output> > tag in the sqoop action. > Any help on this is appreciated. > Thanks. > > > Steve Charles > Senior Consultant > > Sogeti USA > 3636 N. Central Ave. > #810 > Phoenix, AZ 85012 > www.us.sogeti.com <http://www.us.sogeti.com/> >
signature.asc
Description: Digital signature
