[
https://issues.apache.org/jira/browse/HADOOP-5069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12667946#action_12667946
]
Nigel Daley commented on HADOOP-5069:
-------------------------------------
Steve, I like option 1 best as it seems simplest and achieves the goal of not
buffering. But will option 1 work correctly? Aren't stdout and stderr separate
elements in the XML? How can they be streamed to the file?
> add a Hadoop-centric junit test result listener
> -----------------------------------------------
>
> Key: HADOOP-5069
> URL: https://issues.apache.org/jira/browse/HADOOP-5069
> Project: Hadoop Core
> Issue Type: New Feature
> Components: test
> Affects Versions: 0.21.0
> Reporter: Steve Loughran
> Priority: Minor
>
> People are encountering different problems with hadoop's unit tests, defects
> currently being WONTFIX'd
> # HADOOP-5001 : Junit tests that time out don't write any test progress
> related logs
> # HADOOP-4721 : OOM in .TestSetupAndCleanupFailure
> There is a root cause here, the XmlResultFormatter of Ant buffers everything
> before writing out a DOM. Too much logged: OOM and no output. Timeout: kill
> and no output.
> We could add a new logger class to hadoop and then push it back into Ant once
> we were happy, or keep it separate if we had specific dependencies (like on
> hadoop-dfs API) that they lacked.
> Some ideas
> # stream XML to disk. We would have to put the test summary at the end; could
> use XSL to generate HTML and the classic XML content
> # stream XHTML to disk. Makes it readable as you go along; makes the XSL work
> afterwards harder.
> # push out results as records to a DFS. There's a problem here in that this
> needs to be a different DFS from that you are testing, yet it needs to be
> compatible with the client.
> Item #3 would be interesting but doing it inside JUnit is too dangerous
> classpath and config wise. Better to have Ant do the copy afterwards. What is
> needed then is a way to easily append different tests to the same DFS file in
> a way that tools can analyse them all afterwards. The copy is easy -add a new
> Ant resource for that- but the choice of format is trickier.
> Here's some work I did on this a couple of years back; I've not done much
> since then:
> http://people.apache.org/~stevel/slides/distributed_testing_with_smartfrog_slides.pdf
> Is anyone else interested in exploring this?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
