[
https://issues.apache.org/jira/browse/MAHOUT-1346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13906710#comment-13906710
]
Dmitriy Lyubimov commented on MAHOUT-1346:
------------------------------------------
a few obvious optimizer rules
A.t %*% A is obviously detected as a family of unary algorithsm rather than a
binary multiplication alborithm
Geometry and non-zero element estimate plays role in selection of type of
algorithm.
Biggest multiplication via group-by will have to deal, obviously, with
cartesian operator and will apply to (A * B')
Obvious rewrites:
A'*B' = (B * A )' (transposition push-up, including elementwise operators too)
(A')' = A (transposition merge)
cost based grouping (A*B)*C versus A*(B*C)
special distributed algorithm versions for in-core operands and diagonal
matrices
> Spark Bindings (DRM)
> --------------------
>
> Key: MAHOUT-1346
> URL: https://issues.apache.org/jira/browse/MAHOUT-1346
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.8
> Reporter: Dmitriy Lyubimov
> Assignee: Dmitriy Lyubimov
> Fix For: 1.0
>
>
> Spark bindings for Mahout DRM.
> DRM DSL.
> Disclaimer. This will all be experimental at this point.
> The idea is to wrap DRM by Spark RDD with support of some basic
> functionality, perhaps some humble beginning of Cost-based optimizer
> (0) Spark serialization support for Vector, Matrix
> (1) Bagel transposition
> (2) slim X'X
> (2a) not-so-slim X'X
> (3) blockify() (compose RDD containing vertical blocks of original input)
> (4) read/write Mahout DRM off HDFS
> (5) A'B
> ...
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
