[GitHub] ShootingSpace commented on issue #10068: rnn.encode_sentences deals with unknown token
ShootingSpace commented on issue #10068: rnn.encode_sentences deals with unknown token URL: https://github.com/apache/incubator-mxnet/issues/10068#issuecomment-372504273 This is a simple but necessary solution. As people sometimes are only interested in k most frequent tokens, that is the provided dictionary, other tokens are considered as unknown tokens, and could be replaced with special mark (e.g. 'UNK') and added into the dictionary. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ShootingSpace commented on issue #10068: rnn.encode_sentences deals with unknown token
ShootingSpace commented on issue #10068: rnn.encode_sentences deals with unknown token URL: https://github.com/apache/incubator-mxnet/issues/10068#issuecomment-372504273 This is a simple but necessary solution. As people sometimes are only interested in k most frequent token, that is the provided dictionary, other tokens are considered as unknown tokens, and could be replaced with special mark (e.g. 'UNK') and added into the dictionary. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
