[ https://issues.apache.org/jira/browse/SPARK-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14717704#comment-14717704 ]
Joseph K. Bradley commented on SPARK-10307: ------------------------------------------- I tested this a number of times to try to reproduce the issue on branch-1.5. Weirdly, I reproduced it once, with running times: {code} "results":[{"time":79.313},{"time":82.344},{"time":77.169},{"time":63.269},{"time":86.671},{"time":79.732},{"time":76.208},{"time":91.78},{"time":73.738},{"time":56.931},{"time":75.267},{"time":75.316},{"time":63.639},{"time":66.429},{"time":67.172}] {code} But when I tried re-running on branch-1.5 a few times (on both RC1 and the most recent branch with updates post-RC21), I got times like this: {code} "results":[{"time":49.95},{"time":49.081},{"time":50.712},{"time":49.272},{"time":49.81},{"time":47.067},{"time":52.498},{"time":48.093},{"time":48.468},{"time":49.142},{"time":47.212},{"time":47.21},{"time":48.007},{"time":55.267},{"time":48.121}] {code} Note these were all on the same EC2 cluster. So...I'd say there is no obvious regression. If something is wrong, then it's pretty subtle. I'll close this for now. > Fix regression in block matrix multiply (1.4->1.5 regression) > ------------------------------------------------------------- > > Key: SPARK-10307 > URL: https://issues.apache.org/jira/browse/SPARK-10307 > Project: Spark > Issue Type: Improvement > Components: MLlib > Affects Versions: 1.5.0 > Reporter: Joseph K. Bradley > Assignee: Joseph K. Bradley > Priority: Critical > > Running spark-perf on the block-matrix-mult test (BlockMatrix.multiply), I > found the running time increased from 50 sec to 80 sec. This was on the > default test settings, on 16 r3.2xlarge workers on EC2, and with 15 trials, > dropping the first 2. > The only relevant changes I found are: > * > [https://github.com/apache/spark/commit/520ad44b17f72e6465bf990f64b4e289f8a83447] > * > [https://github.com/apache/spark/commit/99c40cd0d8465525cac34dfa373b81532ef3d719] > I'm testing reverting each of those now. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org