tqchen commented on issue #4304: [RFC] Benchmark Performance Log Format WIP URL: https://github.com/apache/incubator-tvm/issues/4304#issuecomment-557744542 re @u99127 's comment about raw statistics, given that some executions have to be run over multiple iterations to get a meaningful measurement, individual run stats may not be as meaningful and accurate. We can still provide a runtime_raw array which might be useful for future statistics calculation, but I do think providing mean and std would be what most folks will look for. the term "engine". the schema was also meant to store baseline benchmarks(e.g. TFLite) and allow cross comparisons between tvm results and baselines, that is why we adopt the term engine.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
