[
https://issues.apache.org/jira/browse/HADOOP-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated HADOOP-8303:
----------------------------
Attachment: 0008-MapReduce-Streaming-Cleanup.patch
A few things:
# Looks like even Hadoop-Common QA does not pick up hadoop-tools changes.
# The eclipse's auto unused import fixer removed RecRecord imports which caused
TestIO not to compile. I fixed those manually in this new patch.
# TestStreaming had a change in original patch that caused it to lose info
about the running jobs and such. I tweaked the change to use a non ToolRunner
approach, and TestStreamingCounters/Combiner now work nicely again - minus
deprecation warnings.
Since Hadoop QA bot can't run the tests, here is the test result manually run,
with patch applied:
{{cd hadoop-tools/hadoop-streaming; mvn clean test}}
{code}
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.hadoop.streaming.TestAutoInputFormat
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.663 sec
Running org.apache.hadoop.streaming.TestClassWithNoPackage
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.261 sec
Running org.apache.hadoop.streaming.TestDumpTypedBytes
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.21 sec
Running org.apache.hadoop.streaming.TestFileArgs
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.49 sec
Running org.apache.hadoop.streaming.TestGzipInput
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.188 sec
Running org.apache.hadoop.streaming.TestLoadTypedBytes
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.486 sec
Running org.apache.hadoop.streaming.TestMRFramework
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.265 sec
Running org.apache.hadoop.streaming.TestMultipleArchiveFiles
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.727 sec
Running org.apache.hadoop.streaming.TestMultipleCachefiles
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.324 sec
Running org.apache.hadoop.streaming.TestRawBytesStreaming
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.07 sec
Running org.apache.hadoop.streaming.TestStreamAggregate
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.062 sec
Running org.apache.hadoop.streaming.TestStreamDataProtocol
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.058 sec
Running org.apache.hadoop.streaming.TestStreaming
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.165 sec
Running org.apache.hadoop.streaming.TestStreamingBackground
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.09 sec
Running org.apache.hadoop.streaming.TestStreamingBadRecords
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.769 sec
Running org.apache.hadoop.streaming.TestStreamingCombiner
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.21 sec
Running org.apache.hadoop.streaming.TestStreamingCounters
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.191 sec
Running org.apache.hadoop.streaming.TestStreamingExitStatus
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.031 sec
Running org.apache.hadoop.streaming.TestStreamingFailure
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.788 sec
Running org.apache.hadoop.streaming.TestStreamingKeyValue
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.324 sec
Running org.apache.hadoop.streaming.TestStreamingOutputKeyValueTypes
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.626 sec
Running org.apache.hadoop.streaming.TestStreamingSeparator
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.068 sec
Running org.apache.hadoop.streaming.TestStreamingStderr
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.579 sec
Running org.apache.hadoop.streaming.TestStreamingTaskLog
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.659 sec
Running org.apache.hadoop.streaming.TestStreamJob
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.761 sec
Running org.apache.hadoop.streaming.TestStreamReduceNone
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.146 sec
Running org.apache.hadoop.streaming.TestStreamXmlMultipleRecords
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.238 sec
Running org.apache.hadoop.streaming.TestStreamXmlRecordReader
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.152 sec
Running org.apache.hadoop.streaming.TestSymLink
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.418 sec
Running org.apache.hadoop.streaming.TestTypedBytesStreaming
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.146 sec
Running org.apache.hadoop.streaming.TestUnconsumedInput
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.139 sec
Running org.apache.hadoop.typedbytes.TestIO
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.166 sec
Running org.apache.hadoop.typedbytes.TestTypedBytesWritable
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.032 sec
Results :
Tests run: 59, Failures: 0, Errors: 0, Skipped: 0
{code}
> Clean up hadoop-streaming
> -------------------------
>
> Key: HADOOP-8303
> URL: https://issues.apache.org/jira/browse/HADOOP-8303
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build
> Affects Versions: 3.0.0
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Minor
> Attachments: 0008-MapReduce-Streaming-Cleanup.patch,
> 0008-MapReduce-Streaming-Cleanup.patch
>
>
> Clean up a bunch of existing javac warnings in hadoop-streaming module.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
