GitHub user sujith71955 opened a pull request:
https://github.com/apache/spark/pull/22199
[SPARK-25073][SQL]When wild card is been used in load command system
## What changes were proposed in this pull request?
When the yarn.nodemanager.resource.memory-mb or
yarn.scheduler.maximum-allocation-mb memory assignment is insufficient, Spark
always reports an error request to adjust yarn.scheduler.maximum-allocation-mb
even though in message it shows the memory value of
yarn.nodemanager.resource.memory-mb,As the error Message is bit misleading to
the user we can modify the same, We can keep the error message same as
executor memory validation message.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sujith71955/spark maste_am_log
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22199.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22199
----
commit 9fe1a6232c4ed61ef67c9baf6e5aaa751a55f3fe
Author: s71955 <sujithchacko.2010@...>
Date: 2018-08-23T22:24:10Z
[SPARK-25073][SQL]When wild card is been used in load command system is
throwing analysis exception
## What changes were proposed in this pull request?
When the yarn.nodemanager.resource.memory-mb or
yarn.scheduler.maximum-allocation-mb memory assignment is insufficient, Spark
always reports an error request to adjust yarn.scheduler.maximum-allocation-mb
even though in message it shows the memory value of
yarn.nodemanager.resource.memory-mb.As the error Message is bit misleading to
the user we can modify the same, We can keep the error message same as
executor memory validation message.
## How was this patch tested?
Manually tested in hdfs-Yarn clustaer
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
