MaximilianSchreff opened a new pull request, #2213:
URL: https://github.com/apache/systemds/pull/2213
This PR adds the backward pass for previously introduced full BERT layer
from the BERT transformer architecture to SystemDS as a built-in operation.
This PR is part of a series of PRs to support the BERT architecture in
SystemDS. The BERT layer is the component in the BERT architecture.
**Includes**
- Backward pass
### Testing:
Added 2 test cases comparing the backward pass results against HuggingFace
Transformer Library's implementation, which uses PyTorch Autograd module for
backward pass computation, for correctness.
- The tests validate:
- Backward pass against HuggingFace Transformer Library's
transformers.models.bert.modeling_bert.BertLayer
- For every single gradient from the whole layer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@systemds.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
