ryujaehun opened a new issue #8703: URL: https://github.com/apache/tvm/issues/8703
When testing DensetNet-121 using AutoTvm, I saw unrealistic kernels like input channel 1 output channel 512. there are some problems with the internal implementation of [densenet](https://github.com/apache/tvm/blob/b893774f38c648536645c3ac5775428b3e9d25b0/python/tvm/relay/testing/densenet.py) ## Problem1 The call of the arguments was incorrect. https://github.com/apache/tvm/blob/b893774f38c648536645c3ac5775428b3e9d25b0/python/tvm/relay/testing/densenet.py#L82 The order of growth_rate,bn_size should be changed. https://github.com/apache/tvm/blob/b893774f38c648536645c3ac5775428b3e9d25b0/python/tvm/relay/testing/densenet.py#L44 ## Solution1 ``` for i, num_layers in enumerate(block_config): layer_out = _make_dense_block(layer_out, num_layers, bn_size,growth_rate, i) num_features = num_features + num_layers * growth_rate ``` ## Problem2 The implementation of [mxnet](https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/gluon/model_zoo/vision/densenet.py) describes bn_size to "Multiplicative factor for number of bottle neck layers".So it is not batch_size. So it must be changed `get_workload ` function. May be addition variable(bn_size) is a simple solution. https://github.com/apache/tvm/blob/b893774f38c648536645c3ac5775428b3e9d25b0/python/tvm/relay/testing/densenet.py#L137-L139 https://github.com/apache/tvm/blob/b893774f38c648536645c3ac5775428b3e9d25b0/python/tvm/relay/testing/densenet.py#L62-L64 ## Solution2 ``` bn_size = 4 net = _make_dense_net( num_init_features, growth_rate, block_config, data_shape, dtype, bn_size, classes ) ``` ## Problem 3 It hasn't been implemented about concatenate operation:( https://github.com/apache/tvm/blob/b893774f38c648536645c3ac5775428b3e9d25b0/python/tvm/relay/testing/densenet.py#L44-L49 ## Solution3 ``` def _make_dense_block(data, num_layers, bn_size, growth_rate, index): """Makes a block of dense layers of the specified size.""" layer_out = data block = [] for i in range(num_layers): layer_out = _make_dense_layer(layer_out, growth_rate, bn_size, "%s_%s" % (index, i)) block.append(layer_out) out = relay.concatenate(block, 1) return out ``` ## Environment TVM: commit 2d1847c9d3ce70daed518d8b3d9dbf750ae34672 CUDA version: 10.2 System: Ubuntu 20.04 GCC 7.5 Build options: -DUSE_LLVM=ON -DUSE_CUDA=ON -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
