e-strauss opened a new pull request, #1994: URL: https://github.com/apache/systemds/pull/1994
- Bug fix for memory estimates of dedup block - optimised recalc of nnz for dedup block - add stats for spark broadcast for transformapply encoder - added accurate mem estimate for transform apply's wordembedding in ParameterizedBuiltinOp.java Built-in LSTM Operator for CP - the LSTM operators interface is similar to the equivalent lstm dml script - forward pass and backward pass, generic for all underlying matrix block representations - forward pass with specific optimization for mini batches on FP64 by operator fusing and loop tiling - the generic backward pass differs from the lstm dml script instructions that it replaced the expensive left-indexing operations (which really boosts the performance for use cases witch large sequences [T>100]) and more efficient reusing of already computed subresults - test cases for checking functionality with comparison lstm dml script I left TODOs for future optimizations: - for the generic backward: fusion of operators for single threaded execution -> parallelize over input batch rows - implementation for an LSTM operation that exploits DedupBlocks (forward and backward) I conducted performance measurements on a single node with 48 vcores and 210gb heap with different batch sizes and input dimensions and random dense inputs, which showed general performance improvements of 2x. For a small input dimensions (32) and a small number LSTM Cells (32), the optimized forward pass even showed improvements up to 10x. Additionally the experiments showed the better performance for long sequences (T=1000) of 4-6x, through the fusion of operations over the "time axis" for a subsets of rows the input batch. Two tables with the measurements are attached as PDFs. [ForwardPassLSTM.pdf](https://github.com/apache/systemds/files/14138852/ForwardPassLSTM.pdf) [BackwardPassLSTM .pdf](https://github.com/apache/systemds/files/14138853/BackwardPassLSTM.pdf) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@systemds.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
