That just means G = GB mem, C = cores, but yeah the driver and executors
are very small, possibly related.
On Wed, Oct 26, 2022 at 12:34 PM Artemis User
wrote:
> Are these Cloudera specific acronyms? Not sure how Cloudera configures
> Spark differently, but obviously the number of nodes is too
Are these Cloudera specific acronyms? Not sure how Cloudera configures
Spark differently, but obviously the number of nodes is too small,
considering each app only uses a small number of cores and RAM. So you
may consider increase the number of nodes. When all these apps jam on
a few
Resource contention. Now all the CPU and I/O is competing and probably
slows down
On Wed, Oct 26, 2022, 5:37 AM eab...@163.com wrote:
> Hi All,
>
> I have a CDH5.16.2 hadoop cluster with 1+3 nodes(64C/128G, 1NN/RM +
> 3DN/NM), and yarn with 192C/240G. I used the following test scenario:
>
>
Hi All,
I have a CDH5.16.2 hadoop cluster with 1+3 nodes(64C/128G, 1NN/RM + 3DN/NM),
and yarn with 192C/240G. I used the following test scenario:
1.spark app resource with 2G driver memory/2C driver vcore/1 executor nums/2G
executor memory/2C executor vcore.
2.one spark app will use 5G4C on
