[
https://issues.apache.org/jira/browse/SPARK-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14717704#comment-14717704
]
Joseph K. Bradley commented on SPARK-10307:
-------------------------------------------
I tested this a number of times to try to reproduce the issue on branch-1.5.
Weirdly, I reproduced it once, with running times:
{code}
"results":[{"time":79.313},{"time":82.344},{"time":77.169},{"time":63.269},{"time":86.671},{"time":79.732},{"time":76.208},{"time":91.78},{"time":73.738},{"time":56.931},{"time":75.267},{"time":75.316},{"time":63.639},{"time":66.429},{"time":67.172}]
{code}
But when I tried re-running on branch-1.5 a few times (on both RC1 and the most
recent branch with updates post-RC21), I got times like this:
{code}
"results":[{"time":49.95},{"time":49.081},{"time":50.712},{"time":49.272},{"time":49.81},{"time":47.067},{"time":52.498},{"time":48.093},{"time":48.468},{"time":49.142},{"time":47.212},{"time":47.21},{"time":48.007},{"time":55.267},{"time":48.121}]
{code}
Note these were all on the same EC2 cluster.
So...I'd say there is no obvious regression. If something is wrong, then it's
pretty subtle. I'll close this for now.
> Fix regression in block matrix multiply (1.4->1.5 regression)
> -------------------------------------------------------------
>
> Key: SPARK-10307
> URL: https://issues.apache.org/jira/browse/SPARK-10307
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Affects Versions: 1.5.0
> Reporter: Joseph K. Bradley
> Assignee: Joseph K. Bradley
> Priority: Critical
>
> Running spark-perf on the block-matrix-mult test (BlockMatrix.multiply), I
> found the running time increased from 50 sec to 80 sec. This was on the
> default test settings, on 16 r3.2xlarge workers on EC2, and with 15 trials,
> dropping the first 2.
> The only relevant changes I found are:
> *
> [https://github.com/apache/spark/commit/520ad44b17f72e6465bf990f64b4e289f8a83447]
> *
> [https://github.com/apache/spark/commit/99c40cd0d8465525cac34dfa373b81532ef3d719]
> I'm testing reverting each of those now.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
