[
https://issues.apache.org/jira/browse/FLINK-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
flora karniav updated FLINK-8414:
---------------------------------
Description:
I am running Gelly examples with different datasets in a cluster of 5 machines
(1 Jobmanager and 4 Taskmanagers) of 32 cores each.
The number of Slots parameter is set to 32 (as suggested) and the parallelism
to 128 (32 cores*4 taskmanagers).
I observe a vast performance degradation using these suggested settings than
setting parallelism.default to 16 for example were the same job completes at
~60 seconds vs ~140 in the 128 parallelism case.
Is there something wrong in my configuration? Should I decrease parallelism and
-if so- will this inevitably decrease CPU utilization?
Another matter that may be related to this is the number of partitions of the
data. Is this somehow related to parallelism? How many partitions are created
in the case of parallelism.default=128?
was:
I am running Gelly examples with different datasets in a cluster of 5 machines
(1 Jobmanager and 4 Taskmanagers) of 32 cores each.
The number of Slots parameter is set to 32 (as suggested) and the parallelism
to 128 (32 cores*4 taskmanagers).
I observe a vast performance degradation using these suggested settings than
setting parallelism.default to 16 for example were the same job completes at 37
seconds vs 140 in the 128 parallelism case.
Is there something wrong in my configuration? Should I decrease parallelism and
-if so- will this inevitably decrease CPU utilization?
Another matter that may be related to this is the number of partitions of the
data. Is this somehow related to parallelism? How many partitions are created
in the case of parallelism.default=128?
> Gelly performance seriously decreases when using the suggested parallelism
> configuration
> ----------------------------------------------------------------------------------------
>
> Key: FLINK-8414
> URL: https://issues.apache.org/jira/browse/FLINK-8414
> Project: Flink
> Issue Type: Bug
> Components: Configuration, Documentation, Gelly
> Reporter: flora karniav
> Priority: Minor
>
> I am running Gelly examples with different datasets in a cluster of 5
> machines (1 Jobmanager and 4 Taskmanagers) of 32 cores each.
> The number of Slots parameter is set to 32 (as suggested) and the parallelism
> to 128 (32 cores*4 taskmanagers).
> I observe a vast performance degradation using these suggested settings than
> setting parallelism.default to 16 for example were the same job completes at
> ~60 seconds vs ~140 in the 128 parallelism case.
> Is there something wrong in my configuration? Should I decrease parallelism
> and -if so- will this inevitably decrease CPU utilization?
> Another matter that may be related to this is the number of partitions of the
> data. Is this somehow related to parallelism? How many partitions are created
> in the case of parallelism.default=128?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
