[
https://issues.apache.org/jira/browse/FLINK-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324228#comment-16324228
]
flora karniav commented on FLINK-8414:
--------------------------------------
Thank you for your reply,
I am running the ConnectedComponents and PageRank algorithms from Gelly
examples on two SNAP datasets:
1) https://snap.stanford.edu/data/egonets-Twitter.html - 81,306 vertices and
2,420,766 edges.
2) https://snap.stanford.edu/data/com-Youtube.html - 1,134,890 vertices and
2,987,624 edges.
I also want to point out that I looked into CPU utilization when changing the
parallelism level and it seems to grow as expected, however performance is
still reduced.
(I am sorry if I posted in an inappropriate section but thought of the issue
bizarre enough to be configuration or bug-related.)
> Gelly performance seriously decreases when using the suggested parallelism
> configuration
> ----------------------------------------------------------------------------------------
>
> Key: FLINK-8414
> URL: https://issues.apache.org/jira/browse/FLINK-8414
> Project: Flink
> Issue Type: Bug
> Components: Configuration, Documentation, Gelly
> Reporter: flora karniav
> Priority: Minor
>
> I am running Gelly examples with different datasets in a cluster of 5
> machines (1 Jobmanager and 4 Taskmanagers) of 32 cores each.
> The number of Slots parameter is set to 32 (as suggested) and the parallelism
> to 128 (32 cores*4 taskmanagers).
> I observe a vast performance degradation using these suggested settings than
> setting parallelism.default to 16 for example were the same job completes at
> ~60 seconds vs ~140 in the 128 parallelism case.
> Is there something wrong in my configuration? Should I decrease parallelism
> and -if so- will this inevitably decrease CPU utilization?
> Another matter that may be related to this is the number of partitions of the
> data. Is this somehow related to parallelism? How many partitions are created
> in the case of parallelism.default=128?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
