eric-haibin-lin commented on issue #9921: [DISCUSSION] 
module.contrib.SparseModule API
   @anirudh2290 Yes. It takes any function. So you could have sth like:
   class A:
       def __init__(self):
           self.dict = {'a': NDArray([1,2,3])}
       def rowids(self):
           return self.dict
   and pass `A().rowids` as the function, which returns a cached dict object. 
   @saswatac Good point. I need to further evaluate how much code I need to 
change to enable this in the existing Module API. Another thing I didn't think 
of is that modules like `BucketingModule` depends on the `Module` class, which 
also has to be updated. 
   @zheng-da two benefits `update_on_kvstore=False` provides:
   One is that users don't have to provide rowids explicitly, which makes it 
easier to use. This also helps the case where the model generate random samples 
as part of the training process and users don't know what rowids to pull before 
calling forward().
   The other is that, for operators such as SparseEmbedding with row_sparse 
weights, it usually comes with some overhead of constructing a lookup table, or 
the effort of searching a particular row when weights.indices don't contain all 
   Note that `update_on_kvstore=False` is not supported by distributed kvstore, 
I'll try to implement a prototype for `update_on_kvstore=True` first. 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

With regards,
Apache Git Services

Reply via email to