[
https://issues.apache.org/jira/browse/SPARK-21491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095587#comment-16095587
]
Iurii Antykhovych edited comment on SPARK-21491 at 7/21/17 12:30 AM:
---------------------------------------------------------------------
I searched for all such places in the whole GraphX module.
These are only three occurrences of such tuples-to-map conversion I could boost
without major refactoring.
Let GraphX be the pilot module of such an optimization)
The changes in `LabelPropagation` and `ShortestPaths` are on a hot execution
path.
The fix in `PageRank` class is executed once per run, not 'hot' enough.
Shall I revert it?
was (Author: sereneant):
I searched for all such places in the whole GraphX module.
These are only three occurrences of such tuples-to-map conversion I could boost
without major refactoring.
Let GraphX be the pilot module of such an optimization)
The only change on a hot path of execution is the code is the one on
`graphx.lib.ShortestPaths` class.
The rest is executed once per run, not 'hot' enough.
Shall I revert it (LabelPropagation.scala, PageRank.scala)?
> Performance enhancement: eliminate creation of intermediate collections
> -----------------------------------------------------------------------
>
> Key: SPARK-21491
> URL: https://issues.apache.org/jira/browse/SPARK-21491
> Project: Spark
> Issue Type: Improvement
> Components: GraphX
> Affects Versions: 2.2.0
> Reporter: Iurii Antykhovych
> Priority: Trivial
>
> Simple performance optimization in a few places of GraphX:
> {{Traversable.toMap}} can be replaced with {{collection.breakout}}.
> This would eliminate creation of an intermediate collection of tuples, see
> [Stack Overflow
> article|https://stackoverflow.com/questions/1715681/scala-2-8-breakout]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
