[
https://issues.apache.org/jira/browse/YARN-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619580#comment-13619580
]
Hitesh Shah commented on YARN-193:
----------------------------------
{code}
+ and will get capped to this value. When it is set to -1, checking against
the
+ maximum allocation should be disable.</description>
{code}
I am not sure if we should allow disabling of the max memory and max vcores
setting. Was it supported earlier or does the patch introduce this support?
Spelling mistake: alloacated
{code}
+ LOG.info("Resource request was not able to be alloacated for" +
+ " application attempt " + appAttemptId + " because it" +
+ " failed to pass the validation. " + e.getMessage());
{code}
The above could be made more simple and brief. For example, "LOG.warn("Invalid
resource ask by application " + appAttemptId, e);" . Also, please use
LOG.level(message, throwable) when trying to log an exception.
{code}
+ RPCUtil.getRemoteException(e);
{code}
Above is missing a throw.
Likewise, in handling of submitApplication, please change log level to warn and
also use the correct log function instead of using e.getMessage().
{code}
if (globalMaxAppAttempts <= 0) {
throw new YarnException(
"The global max attempts should be a positive integer.");
}
{code}
Unrelated to this patch but when throwing/logging errors related to configs, we
should always point to the configuration property to let the user know which
property needs to be changed. Please file a separate jira for the above. With
respect to this, it may be useful to point to the property when throwing
exceptions for invalid min/max memory/vcores.
Unnecessary import in RMAppAttemptImpl:
{code}
+import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils;
{code}
For InvalidResourceRequestException, missing javadocs for class description.
Question - should normalization of resource requests be done inside the
scheduler or in the ApplicationMasterService itself which handles the allocate
call?
If maxMemory or maxVcores is set to -1, what will happen when normalize() is
called? I think there are missing tests related to use of
DISABLE_RESOURCELIMIT_CHECK in both validate and normalize functions that
should have caught this error. In any case, the main question is whether
DISABLE_RESOURCELIMIT_CHECK should actually be allowed.
> Scheduler.normalizeRequest does not account for allocation requests that
> exceed maximumAllocation limits
> ---------------------------------------------------------------------------------------------------------
>
> Key: YARN-193
> URL: https://issues.apache.org/jira/browse/YARN-193
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
> Affects Versions: 2.0.2-alpha, 3.0.0
> Reporter: Hitesh Shah
> Assignee: Zhijie Shen
> Attachments: MR-3796.1.patch, MR-3796.2.patch, MR-3796.3.patch,
> MR-3796.wip.patch, YARN-193.10.patch, YARN-193.11.patch, YARN-193.4.patch,
> YARN-193.5.patch, YARN-193.6.patch, YARN-193.7.patch, YARN-193.8.patch,
> YARN-193.9.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
