areusch opened a new issue #10037: URL: https://github.com/apache/tvm/issues/10037
AutoTVM debugging is fairly difficult today (it generally involves modifying core AutoTVM code), but adding in the complexities of microTVM makes it _much_ more difficult. AutoTVM errors with traditional backends tend to involve two classes of errors: 1. Every program fails -> indicates an underlying problem with the device or measurement setup 2. A few programs fail -> likely these were invalid programs for the device and should be discarded anyway from AutoTVM Autotuning with microTVM introdues many new classes of error: - Flaky device communication - Flash wear - Compilation errors - Memory access problems Debugging these problems requires more information than the current AutoTVM infrastructure provides. However, the output provided by AutoTVM's builder/runner infrastructure is either too unstructured or too terse to be useful here. Some concrete problems: - Due to use of PopenScheduler and/or multiprocessing, some log output is suppressed (e.g. stdout can be suppressed) - Log levels aren't controlled hierarchically (e.g. "autotvm" is the name of the logger), and even so, there isn't a reasonable progression of log levels that logically increases verbosity. - Some information isn't logged at all and could require recompiling the C++ source code - There isn't any sort of machine-parseable output from AutoTVM, meaning it's not easy to mechanically analyze or separate information from tuning runs after-the-fact. A challenge to fixing these problems is that the builder/runner logic has been duplicated and overhauled for auto_scheduler, and so should probably be consolidated before making user-facing fixes like these. On top of this, specific to microTVM: right now, Project API provides some basic building blocks which can be used to break apart the compile/flash/measurement step, but it's not made clear how to rerun these steps in the logging. - [ ] discuss these problems in pre-RFC - [ ] Write up an RFC - [ ] (tbd) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
