unsubscribe
On Thu, Mar 3, 2022 at 5:44 AM Basavaraj wrote:
> unsubscribe
unsubscribe
smime.p7s
Description: S/MIME cryptographic signature
2nd attempt..
Any suggestions to troubleshoot and fix the problem ? thanks in advance.
Regards,
Anil
From: Anil Dasari
Date: Wednesday, March 2, 2022 at 7:00 AM
To: Gourav Sengupta , Yang,Jie(INF)
Cc: user@spark.apache.org
Subject: Re: {EXT} Re: Spark Parquet write OOM
Hi Gourav and Yang
Tha
Hi Gourav and Yang
Thanks for the response.
Please find the answers below.
1. What is the version of SPARK you are using?
[AD] : Spark 2.4.7 (EMR 5.33.1)
2. Are you doing a lot of in-memory transformations like adding columns, or
running joins, or UDFs thus increasing the size of the data befor
Hi Anil,
before jumping to the quick symptomatic fix, can we try to understand the
issues?
1. What is the version of SPARK you are using?
2. Are you doing a lot of in-memory transformations like adding columns, or
running joins, or UDFs thus increasing the size of the data before writing
out?
3.