Hi, Gentle ping. :)
It seems my current implementation is buggy when sending events from the fake processor (e.g. to test how Tez handles load), is there documentation on Tez hearbeat requests/responses anywhere? I don't know what the `preRoutedStartIndex` and `startIndex` values in `TezHeartbeatRequest` are for, nor whether I should be doing anything more than one heartbeat with a "progress is 100%" and a "task is finished" event. Cheers, Solal Pirelli -----Original Message----- From: Solal Pirelli [mailto:[email protected]] Sent: Monday, September 25, 2017 4:41 PM To: [email protected] Subject: Proposal: Simulator mode Hi, I opened a JIRA issue and was redirected to the mailing list, so here I am. :) https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTEZ-3841&data=02%7C01%7Ct-sopire%40microsoft.com%7Cd5c5a48fa5a540e4533408d5046f0085%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636419797147565479&sdata=JMCTd8DzQSvWietTfE5fr2NZ7L8lRpdnLTN5YjZH1uc%3D&reserved=0 is early work on a new feature proposal: a Tez "simulator" in which vertices are not actually executed, but instead use a simplified "fake" processor (which by default does nothing) to let a developer see how Tez will handle certain workloads. The goal is to be relatively close to an actual Tez run (including support for e.g. blacklisting nodes, to see what happens when simulating an operation with a high failure rate) , without requiring an actual Hadoop cluster; the whole thing runs inside a single JVM. The JIRA issue describes the current implementation, and some possible questions a simulator could help answer. What do you think about this proposal? I'd appreciate any pointers regarding the implementation. Cheers, Solal Pirelli
