gt;
> Thanks
> Amit
>
> On Fri, May 8, 2020 at 12:59 PM Hrishikesh Mishra
> wrote:
>
>> We submit spark job through spark-submit command, Like below one.
>>
>>
>> sudo /var/lib/pf-spark/bin/spark-submit \
>> --total-executor-cores 30 \
>> --d
e Books <https://books.japila.pl/>
> Follow me on https://twitter.com/jaceklaskowski
>
> <https://twitter.com/jaceklaskowski>
>
>
> On Fri, May 8, 2020 at 2:32 PM Hrishikesh Mishra
> wrote:
>
>> Thanks Jacek for quick response.
>> Due to our system c
y be Spark's.
>
> Pozdrawiam,
> Jacek Laskowski
>
> https://about.me/JacekLaskowski
> "The Internals Of" Online Books <https://books.japila.pl/>
> Follow me on https://twitter.com/jaceklaskowski
>
> <https://twitter.com/jaceklaskowski>
>
-e126-4cff-9512-d36ad30ee922
On Thu, May 7, 2020 at 10:16 PM Hrishikesh Mishra
wrote:
> It's only happening for Hadoop config. The exceptions trace are different
> for each time it gets died. And Jobs run for couple hours then worker dies.
>
> Another Reason:
>
> *20/05
o load
> configuration (XML files). Make sure they're well formed.
>
> On Thu, May 7, 2020 at 6:12 AM Hrishikesh Mishra
> wrote:
>
>> Hi
>>
>> I am getting out of memory error in worker log in streaming jobs in every
>> couple of hours. After this worke
Hi
I am getting out of memory error in worker log in streaming jobs in every
couple of hours. After this worker dies. There is no shuffle, no
aggression, no. caching in job, its just a transformation.
I'm not able to identify where is the problem, driver or executor. And why
worker getting dead
here a
> reason you chose to start reading again from the beginning by using a new
> consumer group rather then sticking to the same consumer group?
>
> In your application, are you manually committing offsets to Kafka?
>
> Regards,
>
> Waleed
>
> On Wed, Apr 1, 202
Hi
Our Spark streaming job was working fine as expected (the number of events
to process in a batch). But due to some reasons, we added compaction on
Kafka topic and restarted the job. But after restart it was failing for
below reason:
org.apache.spark.SparkException: Job aborted due to stage
