Few comments
# For topology output, you can change 'output' to 'write'
# '-recursive' can be changed to
<tr>
<td><code>-recursive</code></td>
<td>Recursively traverse input paths for job history logs.</td>
<td>This options should be used to inform the TraceBuilder to
recursively scan the input paths and process all the files under under it. Note
that, by default, only the history logs that are directly under the input
folder are considered for generating the trace.
</td>
</tr>
# Testcase
/**
* Test {...@link TraceBuilder#processInputArgument()}.
* @throws IOException
*/
@Test
public void testTraceBuilderInputArgumentProcessing() throws IOException {
List<String> inputPaths = createInput();
// test with recursive turned OFF
testTraceBuilderDeaultInputArgumentProcessing(inputPaths); // these can be
private unit test cases
// test with recursive turned ON
testTraceBuilderRecursiveInputArgumentProcessing(inputPaths);
}
Also "testRecursive()" should be more detailed.
W.r.t to the discussion we had, the data/directory creation step can be
# TraceBuilder: "List<Path> inputPaths = new LinkedList<Path>();" can be done
only after "if (inStatuses == null || inStatuses.length == 0) {" succeeds.
I will review the main code changes to TraceBuilder at your desk
Amar
On 9/23/10 11:21 AM, "Ravi Gummadi (JIRA)" <[email protected]> wrote:
[
https://issues.apache.org/jira/browse/MAPREDUCE-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Gummadi updated MAPREDUCE-1978:
------------------------------------
Attachment: 1978.patch
Attaching patch that adds the option "-recursive" to TraceBuilder.
With -recursive option, TraceBuilder generates trace by scanning all the job
history logs recursively under the given path.
Please review the patch and provide your comments.
> [Rumen] TraceBuilder should provide recursive input folder scanning
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-1978
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1978
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tools/rumen
> Reporter: Amar Kamat
> Assignee: Ravi Gummadi
> Attachments: 1978.patch
>
>
> Currently, {{TraceBuilder}} assumes that the input is either jobhistory files
> or a folders containing jobhistory files directly underneath the specified
> folder. There could be a use cases where the input folder could contain
> sub-folders containing jobhistory files. Rumen should support such input
> folders.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
