[
https://issues.apache.org/jira/browse/MAPREDUCE-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13800761#comment-13800761
]
Ravi Prakash commented on MAPREDUCE-3860:
-----------------------------------------
Thanks for the update Andrey! I was able to get clean runs. I had inadvertently
left my local dev single node cluster up and running while running the test.
*facepalm*
So far most of the changes seem fine. However I am having doubts about whether
these unit tests are the right way to go about unit testing. Most of them
depend on these resource files which will be checked in. I am wondering how
many of these files will become stale when the state transitions are changed.
And then even if these unit tests pass, rumen functionality would be broken
because it would not be able to parse the JH files. Maybe the correct way of
testing would be to generate these resource files using fake jobs? What do you
think? I can't help but think that these tests were removed in the first place
because of their poor design.
Anyway, here are some other things I had concerns about:
h6. TestRumenAnonymization
• Line 822 catch (Exception e) I shouldn't have to wait 100 seconds if there
was an exception. Same in TestRumenJobTraces:testCurrentJHParser()
• You've made Conf common for all tests. Changes to conf in 1 test will affect
the others. I'm not sure that is a good change. Should we mark rootInputDir
final?
• Line 868: Is null a valid anonymization of the job's username?
• Why did you remove the "lfs.delete(tempDir, true);" lines?
TestRumenJobTraces:
• Line 923: int[][] splits = new int[4][12];
I've stopped reviewing in the middle hoping to get your thoughts about what we
should do. I am a lot more interested in the long term solution rather than the
short term solution.
> [Rumen] Bring back the removed Rumen unit tests
> -----------------------------------------------
>
> Key: MAPREDUCE-3860
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3860
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tools/rumen
> Reporter: Ravi Gummadi
> Assignee: Andrey Klochkov
> Attachments: MAPREDUCE-3860--n2.patch, MAPREDUCE-3860.patch,
> rumen-test-data.tar.gz
>
>
> MAPREDUCE-3582 did not move some of the Rumen unit tests to the new folder
> and then MAPREDUCE-3705 deleted those unit tests. These Rumen unit tests need
> to be brought back:
> TestZombieJob.java
> TestRumenJobTraces.java
> TestRumenFolder.java
> TestRumenAnonymization.java
> TestParsedLine.java
> TestConcurrentRead.java
--
This message was sent by Atlassian JIRA
(v6.1#6144)
