[
https://issues.apache.org/jira/browse/FLINK-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-15564:
---------------------------------
Description:
As discovered in [PR#10834|https://github.com/apache/flink/pull/10834], the
following test cases of {{YarnClusterDescriptorTest}} have failed to validate
the original intended behavior and are temporally skipped by PR#10834.
- {{testFailIfTaskSlotsHigherThanMaxVcores}}
- {{testConfigOverwrite}}
The original purpose of these two test cases was to verify the validation logic
against yarn max allocation vcores (in
{{5836f7eddb4849b95d4860cf20045bc61d061918}}).
These two cases should have failed when we change the validation logic to get
yarn max allocation vcores from yarnClient instead of configuration (in
{{e959e6d0cd42f0c5b21c0f03ce547f2025ac58d5}}), because there are no yarn
cluster (neither {{MiniYARNCluster}}) started in these cases, thus
{{yarnClient#getNodeReports}} will never return.
The cases have not failed because another {{IllegalConfigurationException}} was
thrown in {{validateClusterSpecification}}, because of memory validation
failure. The memory validation failure was by design, and in order to verify
the original purpose these two test cases should have been updated with
reasonable memory sizes, which is unfortunately overlooked.
The problem could be fixed with the following changes:
- Update the memory setups for the test cases, to pass the memory validation
and thus validate the original intended behavior.
- Extract the logic of getting yarn max allocation vcores into a separate
method, and override it in the test cases to provide a constant max vcores.
was:
As discovered in [PR#10834|https://github.com/apache/flink/pull/10834], the
following test cases of {{YarnClusterDescriptorTest}} have failed to validate
the original intended behavior and are temporally skipped by PR#10834.
- {{testFailIfTaskSlotsHigherThanMaxVcores}}
- {{testConfigOverwrite}}
The original purpose of these two test cases was to verify the validation logic
against yarn max allocation vcores (in
{{5836f7eddb4849b95d4860cf20045bc61d061918}}).
These two cases should have failed when we change the validation logic to get
yarn max allocation vcores from yarnClient instead of configuration (in
{{e959e6d0cd42f0c5b21c0f03ce547f2025ac58d5}}), because there are no yarn
cluster (neither {{MiniYARNCluster}}) started in these cases, thus
{{yarnClient#getNodeReports}} will never return.
The cases have not failed because another {{IllegalConfigurationException}} was
thrown in {{validateClusterSpecification}}, because of memory validation
failure. The memory validation failure was by design, and in order to verify
the original purpose these two test cases should have been updated with
reasonable memory sizes, which is unfortunately overlooked.
> YarnClusterDescriptorTest failed to validate the original intended behavior
> ---------------------------------------------------------------------------
>
> Key: FLINK-15564
> URL: https://issues.apache.org/jira/browse/FLINK-15564
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
> Reporter: Xintong Song
> Assignee: Xintong Song
> Priority: Major
> Labels: pull-request-available, testability
> Time Spent: 10m
> Remaining Estimate: 0h
>
> As discovered in [PR#10834|https://github.com/apache/flink/pull/10834], the
> following test cases of {{YarnClusterDescriptorTest}} have failed to validate
> the original intended behavior and are temporally skipped by PR#10834.
> - {{testFailIfTaskSlotsHigherThanMaxVcores}}
> - {{testConfigOverwrite}}
> The original purpose of these two test cases was to verify the validation
> logic against yarn max allocation vcores (in
> {{5836f7eddb4849b95d4860cf20045bc61d061918}}).
> These two cases should have failed when we change the validation logic to get
> yarn max allocation vcores from yarnClient instead of configuration (in
> {{e959e6d0cd42f0c5b21c0f03ce547f2025ac58d5}}), because there are no yarn
> cluster (neither {{MiniYARNCluster}}) started in these cases, thus
> {{yarnClient#getNodeReports}} will never return.
> The cases have not failed because another {{IllegalConfigurationException}}
> was thrown in {{validateClusterSpecification}}, because of memory validation
> failure. The memory validation failure was by design, and in order to verify
> the original purpose these two test cases should have been updated with
> reasonable memory sizes, which is unfortunately overlooked.
> The problem could be fixed with the following changes:
> - Update the memory setups for the test cases, to pass the memory validation
> and thus validate the original intended behavior.
> - Extract the logic of getting yarn max allocation vcores into a separate
> method, and override it in the test cases to provide a constant max vcores.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
