eric-haibin-lin commented on issue #9921: [DISCUSSION] module.contrib.SparseModule API URL: https://github.com/apache/incubator-mxnet/issues/9921#issuecomment-371033223 @anirudh2290 How much would a function call cost? I don't think static dictionary would be a common case.. Usually you pull differently for each iteration. @saswatac Good point. I need to further evaluate how much code I need to change to enable this in the existing Module API. Another thing I didn't think of is that modules like `BucketingModule` depends on the `Module` class, which also has to be updated. @zheng-da two benefits `update_on_kvstore=False` provides: One is that users don't have to provide rowids explicitly, which makes it easier to use. This also helps the case where the model generate random samples as part of the training process and users don't know what rowids to pull before calling forward(). The other is that, for operators such as SparseEmbedding with row_sparse weights, it usually comes with some overhead of constructing a lookup table, or the effort of searching a particular row when weights.indices don't contain all rows. ====== Note that `update_on_kvstore=False` is not supported by distributed kvstore, I'll try to implement a prototype for `update_on_kvstore=True` first.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
