[ 
https://issues.apache.org/jira/browse/FLINK-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324228#comment-16324228
 ] 

flora karniav commented on FLINK-8414:
--------------------------------------

Thank you for your reply, 

I am running the ConnectedComponents and PageRank algorithms from Gelly 
examples on two SNAP datasets:

1) https://snap.stanford.edu/data/egonets-Twitter.html - 81,306 vertices and 
2,420,766 edges.
2) https://snap.stanford.edu/data/com-Youtube.html - 1,134,890 vertices and 
2,987,624 edges.

I also want to point out that I looked into CPU utilization when changing the 
parallelism level and it seems to grow as expected, however performance is 
still reduced. 

(I am sorry if I posted in an inappropriate section but thought of the issue 
bizarre enough to be configuration or bug-related.)

> Gelly performance seriously decreases when using the suggested parallelism 
> configuration
> ----------------------------------------------------------------------------------------
>
>                 Key: FLINK-8414
>                 URL: https://issues.apache.org/jira/browse/FLINK-8414
>             Project: Flink
>          Issue Type: Bug
>          Components: Configuration, Documentation, Gelly
>            Reporter: flora karniav
>            Priority: Minor
>
> I am running Gelly examples with different datasets in a cluster of 5 
> machines (1 Jobmanager and 4 Taskmanagers) of 32 cores each.
> The number of Slots parameter is set to 32 (as suggested) and the parallelism 
> to 128 (32 cores*4 taskmanagers).
> I observe a vast performance degradation using these suggested settings than 
> setting parallelism.default to 16 for example were the same job completes at 
> ~60 seconds vs ~140 in the 128 parallelism case.
> Is there something wrong in my configuration? Should I decrease parallelism 
> and -if so- will this inevitably decrease CPU utilization?
> Another matter that may be related to this is the number of partitions of the 
> data. Is this somehow related to parallelism? How many partitions are created 
> in the case of parallelism.default=128? 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to