louislepage commented on PR #1849: URL: https://github.com/apache/systemds/pull/1849#issuecomment-1619985178
> Have you tried pop count on the bit set for the sum operation? This should speed up the bit set significantly for sum. I did look into how some operators would probably have a speedup when using bitset because they can be performed on the entire bitset and not value by value. `OR`, `AND`, and `XOR` are also candidates for this (e.g. [`BitSet.or(BitSet bitset)`](https://docs.oracle.com/javase/8/docs/api/java/util/BitSet.html#or-java.util.BitSet-). But, it would mean a reimplementation of all of these operators. And a mechanism to decide when to use value by value and when Bitset on Bitset. However, the GraterThan and other Comparison Functions were already slower for bitset , than when run on FP64, and these have to run value by value. This would mean that one would need a dynamic typing mechanism to decide during runtime which kind of boolean denseblock would be most beneficial for the upcoming operations. @phaniarnab and I discussed this briefly in a meeting but came to the result, that in the end, there should be only one kind of boolean array implementation for now. Therefore I proposed the boolean array since it seems to have some speedup over all operators while using the same value-by-value operations as the FP64 implementation did. It would provide a speedup while being _backward-compatible_ with the current operator implementation, so to say. But I agree that there most certainly are situations where bitset could be significantly faster. I think that having both boolean implementations and deciding at runtime what should be used would be very interesting, but for now, I do not understand systemds well enough to confidently say if this would be feasible and how complex this would be. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@systemds.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
