[ https://issues.apache.org/jira/browse/CRUNCH-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006389#comment-14006389 ]
Micah Whitacre commented on CRUNCH-272: --------------------------------------- Logged CRUNCH-400 to track that change regarding tracking the material stages. I'll roll with vetting this prototype + tests assuming the missing stages will be handled on the other issue. > Unable to correlate crunch jobs within Oozie > -------------------------------------------- > > Key: CRUNCH-272 > URL: https://issues.apache.org/jira/browse/CRUNCH-272 > Project: Crunch > Issue Type: Improvement > Reporter: Mike Zimmerman > Assignee: Micah Whitacre > Attachments: CRUNCH-272_prototype.patch > > > I'm not really sure if this should be logged to Oozie or to Crunch, so please > feel free to move as needed. > I would like to request a way to decorate map/reduce jobs that are spawned by > a Crunch pipeline so that I can programmatically determine their origin. The > primary use case for this is integration with Oozie. Oozie launches a single > map job to run a java action (in our case this java action runs a crunch > job). Traceability from this original "launcher" job to the jobs created by > the crunch job is impossible without trolling logs. This leaves a big black > hole for the system operator to assess the performance/impact of these jobs. > My initial thought was to provide a simple way to indicate a correlationId or > similar on a map/reduce job and then make it accessible within Oozie to query > for. Obviously, that request would have to come after the correlation > feature was available within map/reduce. -- This message was sent by Atlassian JIRA (v6.2#6252)