Re: LinkedIn Dynamometer Tool (was About 2.7.4 Release)

2017-07-21 Thread Erik Krogen
Hi Sanjay, Actually I was not aware of that work… This seems to be a better way of achieving some of the same things we do externally to the DN process. I will look into reimplementing some parts on top of this; seems it should just require some very small extensions to DataNodeCluster. Thank

Re: LinkedIn Dynamometer Tool (was About 2.7.4 Release)

2017-07-21 Thread sanjay Radia
Erik Great stuff. BTW did you build on top of the “simulated data nodes” in HDFS which has a way to storing only the length of data (but not real data)? That work allowed supplementing with a matching editsLog for the NN. Your approach of using a real image has the advantage of being able

Re: LinkedIn Dynamometer Tool (was About 2.7.4 Release)

2017-07-20 Thread Anu Engineer
Hi Erik, Looking forward to the release of this tool. Thank you very much for the contribution. Had a couple of questions about how the tool works. 1. Would you be able to provide the traces along with this tool? In other words, would I be able to use this out of the box, or do I have to