jinhuang415 commented on issue #10433: [MXNET-290] MKLDNN support for model quantization URL: https://github.com/apache/incubator-mxnet/pull/10433#issuecomment-395478506 @marcoabreu I tried to use below "try .. catch ..." statement to test native CPU, MKLDNN, GPU ``` for dtype in ['int8','uint8']: try: run_test(..., dtype) catch: ``` But for those OPs not supported by native CPU, the test would "segmentation fault" directly and the exception handler could not catch it so the test will abort directly and could not execute any further case, I think the MXNet framework didn't handle the unsupported OP very well (when OP didn't register FCompute/FComputeEx like for native CPU it didn't register FCompute/FComputeEx for quantized conv/pooling/fc), another case is if we test unsupported dtype for one backend (like test int8 for MKLDNN or uint8 for GPU), the C++ code will do parameter check failure and throw runtime error message, of course we can catch this exception and ignore it but it is not the specific OperatorMissingError exception. I think it's not trivial to change MXNet C++ code to throw the specific exception for various unsupported cases (need to add proper exception thrower for all these cases), how about use below 2 approaches which makes life a little easier: ``` run_test(..., dtype) // add checking at the top of run_test() so we can see what kind of test is expected to be skipped and the rest should be tested and passed if context == cpu_mkldnn: if dtype == 'int8' logging.info(skipping due to unsupported dtype {}, dtype) return elif context == cpu_native: ... <normal-test-path for dtype> for dtype in ['int8','uint8']: run_test(..., dtype) ``` This approach tested all cases and skipped the unsupported case explicitly so users can see, for not skipped case they are supposed to test and pass through. ``` run_test(..., dtype) // test case implementation <normal-test-code-path for dtype> if context == cpu_native: // list test cases for each context implicitly dtype_list = [] // empty means doesn't support this quantized OP elif if context == cpu_mkldnn: dtype_list = [uint8] elif if context == gpu: dtype_list = [int8] for dtype in dtype_list: run_test(..., dtype) ``` This approach explicitly listed the dtype_list each context support and only test the supported scenario, we don't have backend info in the test case implementation( `run_test()` ) so the test case implementation will be clean. Please let us know your comments.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
