kenjewu opened a new pull request #12535: Implementation of Structured-Self-Attentive-Sentence-Embedding URL: https://github.com/apache/incubator-mxnet/pull/12535 Attention mechanism is becoming more and more important in natural language processing. In order to further study the principle of attention mechanism, I read the paper [A Structured Self-Attentive Sentence Embedding](https://arxiv.org/abs/1703.03130). This is an implementation of the paper [A Structured Self-Attentive Sentence Embedding](https://arxiv.org/abs/1703.03130). This program implements most of the details in the paper. Finally, the user reviews the emotional star ratings in the three experiments mentioned in the original paper, and used the same data set: [The reviews of Yelp Data](https://www.kaggle.com/yelp-dataset/yelp-dataset#yelp_academic_dataset_review.json). **Implemented** 1. **Attention mechanism proposed in the original paper.** 2. **Punishment constraints to ensure diversity of attention.** 3. **Parameter pruning proposed in the appendix of the paper.** 4. **Gradient clip and learning rate decay.** 5. **SoftmaxCrossEntropy with category weights**
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
