lfengad opened a new pull request #4882: [Relay][Topi] Hashtable support URL: https://github.com/apache/incubator-tvm/pull/4882 We observe a great amount of tensorflow NLP models used in our production environment, and a lot of them use Hashtable related operators for word/character hashing, convert DT_STRING to integer type, and do further embedding/feature extraction in a later stage. This PR implements 4 hashtable related tensorflow operators in tvm: - HashTableV2 - InitializeTableFromTextFileV2 - LookupTableImportV2 - LookupTableFindV2 Implementation uses the recommended way of adding external operations. The hashtable related operations are first built as an external library, which is invoked by the TOPI layer to form operators. The tensorflow frontend parses hashtable related operators from tensorflow graph as registered relay operators. One special thing about hashtable is that it requires an extra initialization phase of graph runtime, that populates hashtable with constant params or from an external file. Just like what tensorflow does in `tf.table_initializer()`, we need to do initialization before execution. And that involves minor change on graph runtime as well. Finally, as hashtable is mostly used for string->numeric value lookup, so we tentatively add support for string as custom datatype. As the two features are tightly coupled, so this pr contains code for both features. Test case is provided as well to help verify the functionality of hashtable implementation. Any suggestions are welcome! @tqchen @FrozenGene
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
