[
https://issues.apache.org/jira/browse/MAHOUT-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12779591#action_12779591
]
Jake Mannix commented on MAHOUT-165:
------------------------------------
bq. I think this deserves a bit more thought. I was actually surprised to hear
iterateNonZero() doesn't iterate in order?? that is surprising. Can that be
right?
I'm pretty sure this is in all of the hash-based implementations I've seen.
Nobody goes to the trouble to make it a linked hash or whatnot, because that
slows down insertion (to ad a different complexity class!) and takes up more
memory.
In my view, there's really two different sparse vector implementations: one
primarily read-only and optimized for speed of iteration (and can guarantee
ordering), and another which is optimized for random-access read-write actions.
The client needs to know which action they're going to be using most and make
the choice of impl based on that (and if necessary, make a switch partway
through: I often build up vectors using a map-based approach, then "seal" them
into a faster read-only form if I don't need random-access reads done on them
[which is the slow read-only action on int[] / double[] vectors])
> Using better primitives hash for sparse vector for performance gains
> --------------------------------------------------------------------
>
> Key: MAHOUT-165
> URL: https://issues.apache.org/jira/browse/MAHOUT-165
> Project: Mahout
> Issue Type: Improvement
> Components: Matrix
> Affects Versions: 0.2
> Reporter: Shashikant Kore
> Assignee: Grant Ingersoll
> Fix For: 0.3
>
> Attachments: colt.jar, mahout-165-18nov-updated.patch,
> mahout-165-18nov.patch, mahout-165-trove.patch, MAHOUT-165-updated.patch,
> MAHOUT-165-with-colt-module.patch, MAHOUT-165-with-colt.patch,
> mahout-165.patch, MAHOUT-165.patch, mahout-165.patch
>
>
> In SparseVector, we need primitives hash map for index and values. The
> present implementation of this hash map is not as efficient as some of the
> other implementations in non-Apache projects.
> In an experiment, I found that, for get/set operations, the primitive hash of
> Colt performance an order of magnitude better than OrderedIntDoubleMapping.
> For iteration it is 2x slower, though.
> Using Colt in Sparsevector improved performance of canopy generation. For an
> experimental dataset, the current implementation takes 50 minutes. Using
> Colt, reduces this duration to 19-20 minutes. That's 60% reduction in the
> delay.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
