[
https://issues.apache.org/jira/browse/PIRK-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15420419#comment-15420419
]
Suneel Marthi commented on PIRK-40:
-----------------------------------
Thinking loud here (am a Pirk novice so pardon my rants)
We are using BigInteger to represent vectors and most of the bits are zero.
Have we considered using Math packages like Breeze, Commons Math instead for
Sparse Vector representations. They also provide the relevant Vector addition,
multiply etc operations.
How about Distributed Vector multiplication on Spark ? We do that on Mahout
with Mahout distributed Math backend that can presently run on Flink, Spark and
H2O and has engine agnostic semantics.
If we could represent all the queries vectors as a Matrix, then Mahout has the
concept of Distributed Row Matrix (DRM) which basically splits a Matrix as
either blocks or blocks of Rows across the cluster and multiplication
operations like Matrix-Matrix, Matrix-Vector etc would all be distributed.
Happy to discuss more.
> Explore Options to Increase Efficiency of Modular Multiplication
> ----------------------------------------------------------------
>
> Key: PIRK-40
> URL: https://issues.apache.org/jira/browse/PIRK-40
> Project: PIRK
> Issue Type: Improvement
> Reporter: Ellison Anne Williams
> Assignee: Jacob WIlder
>
> We are using JNAGMP (which wraps some operations in GNU's Multiple Precision
> Arithmetic Library) to improve the performance of modular exponentiation; we
> are benchmarking these improvements with the provide benchmark package.
> JNAGMP does not currently provide wrappings for modular multiplication.
> As modular multiplications are one of our most expensive operations, we need
> to explore methods to increase the efficiency.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
