[
https://issues.apache.org/jira/browse/BIGTOP-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13916625#comment-13916625
]
Konstantin Boudnik commented on BIGTOP-1065:
--------------------------------------------
Yes, I agree. Perhaps this JIRA can stay open and add a different angle to the
cluster load. BIGTOP-1198 actually stresses out the underlying hardware
basically limiting the amount of RAM, CPU, disk bandwidth available to the
Hadoop software. Hence, it isn't exactly Hadoop stress test.
> Stress Tests
> -------------
>
> Key: BIGTOP-1065
> URL: https://issues.apache.org/jira/browse/BIGTOP-1065
> Project: Bigtop
> Issue Type: Test
> Reporter: jay vyas
> Priority: Minor
>
> There are issues in the past where the JobTracker has failures because of so
> called "memory-leaks" , or infinitely growing objects, unclean tmp/ folders
> from jobs, etc.
> Thus, the proposal here is a lighteweight job which can be run rapidly,
> several 1000 times, which confirms that jobtracker state does not grow out of
> bounds or infinitely with respect to number of tasks/jobs run/submitted.
> IMPLEMENTATION PROPOSAL:
> Some simple starts would be to :
> - run word count, or a the sleep job 100 or 1000 times or 10,000 times.
> - create and delete the same file over and over again several thousand times
> to see if filesystem consistency is maintained
> To start, I'd like to add all these tests in a single module , under
> test-executions/stress/. Then later we could shard it out in another way.
> UPDATE:
> As per comments below, just noting that although phrased in terms of
> "JobTracker", the spirit of this ticket is to be applicable in both mr1 and
> mr2, since in either case, the purpose is to test the impact that several
> 100/1000 mapreduce job runs has over time and confirm that tmp dirs, objects
> in memory, etc are all managed and lifecycled properly .
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
