On Tue, Aug 27, 2013 at 10:18PM, Jay Vyas wrote: > hmm ok. now. Thinking about teragen makes me think of benchmarking.. > > In the longer term we could add benchmarking jobs to all the submodules not > just mapreduce. For example there are hi bench and ycsb workloads which > might be usable or pulled in as bigtop components ... Iff of course > benchmarking is in the cards for bigtop?
It indeed is! I think doing tera-gen/sort so it can be parameterized will provide a good basis for future benchmarking (as a bit of reflection: I have did simplistic yet efficient way of benchmarking HDFS and MR a couple years ago, but my employer back then has never let it go into the open. Go figure...) And I have a way of building YCSB against a particular version of Hadoop, so I guess I will have it packaged as a benchmarking test pretty soon. Cos > On Aug 27, 2013, at 7:39 PM, Roman Shaposhnik <[email protected]> wrote: > > > On Tue, Aug 27, 2013 at 4:26 PM, Jay Vyas <[email protected]> wrote: > >> Hi guys: > >> > >> I run TeraSort/TeraGen as additions to bigtop in some shell scripts. > >> > >> Any interest in these as an update to TestHadoopExamples in the MapReduce > >> smokes? > >> > >> If so I could patch them in :) > > > > Sure! Sounds like a useful addition. > > > > Thanks, > > Roman.
signature.asc
Description: Digital signature
