[
https://issues.apache.org/jira/browse/MAHOUT-165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Kore updated MAHOUT-165:
-----------------------------------
Attachment: mahout-165-18nov.patch
Here is the updated patch.
The dependency on colt-1.2.jar needs to be added to core/pom.xml
This patch reported following 3 failures.
----------
testAssignUnaryFunction(org.apache.mahout.matrix.TestSparseVector) Time
elapsed: 0.006 sec <<< FAILURE!
junit.framework.AssertionFailedError: get [0] expected:<0.0> but was:<-0.0>
----------
testDot(org.apache.mahout.matrix.TestSparseVector) Time elapsed: 0.001 sec
<<< FAILURE!
junit.framework.AssertionFailedError: dot expected:<16.939999999999998> but
was:<16.94>
----------
testHashCodeEquivalence(org.apache.mahout.matrix.VectorTest) Time elapsed:
0.051 sec <<< FAILURE!
junit.framework.AssertionFailedError: expected:<-1302772127> but
was:<2062521697>
----------
First two look innocent.
The hashcode mismatch occurs because while iterating on the non-zero elements,
sparse vector gives elements in different order than dense vector. We could
sort the elements before generating hashcode. Not sure if this is the optimal
solution.
> Using better primitives hash for sparse vector for performance gains
> --------------------------------------------------------------------
>
> Key: MAHOUT-165
> URL: https://issues.apache.org/jira/browse/MAHOUT-165
> Project: Mahout
> Issue Type: Improvement
> Components: Matrix
> Affects Versions: 0.2
> Reporter: Shashikant Kore
> Assignee: Grant Ingersoll
> Fix For: 0.3
>
> Attachments: colt.jar, mahout-165-18nov.patch,
> mahout-165-trove.patch, MAHOUT-165-updated.patch,
> MAHOUT-165-with-colt-module.patch, MAHOUT-165-with-colt.patch,
> mahout-165.patch, MAHOUT-165.patch, mahout-165.patch
>
>
> In SparseVector, we need primitives hash map for index and values. The
> present implementation of this hash map is not as efficient as some of the
> other implementations in non-Apache projects.
> In an experiment, I found that, for get/set operations, the primitive hash of
> Colt performance an order of magnitude better than OrderedIntDoubleMapping.
> For iteration it is 2x slower, though.
> Using Colt in Sparsevector improved performance of canopy generation. For an
> experimental dataset, the current implementation takes 50 minutes. Using
> Colt, reduces this duration to 19-20 minutes. That's 60% reduction in the
> delay.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
