[
https://issues.apache.org/jira/browse/SPARK-39815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marzieh updated SPARK-39815:
----------------------------
Description:
h1. I have a Spark SQL Program which run in Spark Cluster. Even though the
program is finished without any Error, after finishing State of the Application
becomes Killed. It shows this error on the log:
22/07/19 13:14:50 INFO CoarseGrainedExecutorBackend: Got assigned task 10686
22/07/19 13:14:50 INFO Executor: Running task 232.0 in stage 137.0 (TID 10686)
22/07/19 13:14:50 INFO CoarseGrainedExecutorBackend: Got assigned task 10687
22/07/19 13:14:50 INFO Executor: Running task 233.0 in stage 137.0 (TID 10687)
22/07/19 13:14:50 INFO PythonRunner: Times: total = 6, boot = 2, init = 4,
finish = 0 22/07/19 13:14:50 INFO Executor: Finished task 232.0 in stage 137.0
(TID 10686). 1785 bytes result sent to driver 22/07/19 13:14:50 INFO
PythonRunner: Times: total = 10, boot = 9, init = 1, finish = 0 22/07/19
13:14:50 INFO Executor: Finished task 233.0 in stage 137.0 (TID 10687). 1785
bytes result sent to driver 22/07/19 13:14:50 INFO
CoarseGrainedExecutorBackend: Got assigned task 10688 22/07/19 13:14:50 INFO
Executor: Running task 234.0 in stage 137.0 (TID 10688) 22/07/19 13:14:50 INFO
CoarseGrainedExecutorBackend: Got assigned task 10689 22/07/19 13:14:50 INFO
Executor: Running task 235.0 in stage 137.0 (TID 10689) 22/07/19 13:14:50 INFO
PythonRunner: Times: total = 1, boot = 1, init = 0, finish = 0 22/07/19
13:14:50 INFO Executor: Finished task 235.0 in stage 137.0 (TID 10689). 1785
bytes result sent to driver 22/07/19 13:14:50 INFO
CoarseGrainedExecutorBackend: Got assigned task 10690 22/07/19 13:14:50 INFO
Executor: Running task 236.0 in stage 137.0 (TID 10690) 22/07/19 13:14:50 WARN
JdbcUtils: Requested isolation level 1 is not supported; falling back to
default isolation level 2 22/07/19 13:14:50 INFO PythonRunner: Times: total =
42, boot = -13, init = 55, finish = 0 22/07/19 13:14:50 INFO Executor: Finished
task 231.0 in stage 137.0 (TID 10685). 1785 bytes result sent to driver
22/07/19 13:14:50 INFO CoarseGrainedExecutorBackend: Got assigned task 10691
22/07/19 13:14:50 INFO Executor: Running task 237.0 in stage 137.0 (TID 10691)
22/07/19 13:14:50 INFO PythonRunner: Times: total = 43, boot = -4, init = 47,
finish = 0 22/07/19 13:14:50 INFO Executor: Finished task 234.0 in stage 137.0
(TID 10688). 1785 bytes result sent to driver 22/07/19 13:14:50 INFO
CoarseGrainedExecutorBackend: Got assigned task 10692 22/07/19 13:14:50 INFO
Executor: Running task 238.0 in stage 137.0 (TID 10692) 22/07/19 13:14:50 INFO
PythonRunner: Times: total = 43, boot = 2, init = 41, finish = 0 22/07/19
13:14:50 INFO Executor: Finished task 236.0 in stage 137.0 (TID 10690). 1785
bytes result sent to driver 22/07/19 13:14:50 INFO
CoarseGrainedExecutorBackend: Got assigned task 10693 22/07/19 13:14:50 INFO
Executor: Running task 239.0 in stage 137.0 (TID 10693) 22/07/19 13:14:50 INFO
JDBCRDD: closed connection 22/07/19 13:14:50 INFO PythonRunner: Times: total =
44, boot = 3, init = 41, finish = 0 22/07/19 13:14:50 INFO Executor: Finished
task 237.0 in stage 137.0 (TID 10691). 1785 bytes result sent to driver
22/07/19 13:14:50 INFO PythonRunner: Times: total = 44, boot = 2, init = 42,
finish = 0 22/07/19 13:14:50 INFO Executor: Finished task 238.0 in stage 137.0
(TID 10692). 1785 bytes result sent to driver 22/07/19 13:14:50 INFO Executor:
Finished task 219.0 in stage 137.0 (TID 10673). 1785 bytes result sent to
driver 22/07/19 13:14:50 INFO PythonRunner: Times: total = 42, boot = 2, init =
40, finish = 0 22/07/19 13:14:50 INFO Executor: Finished task 239.0 in stage
137.0 (TID 10693). 1785 bytes result sent to driver 22/07/19 13:14:50 INFO
CoarseGrainedExecutorBackend: Driver commanded a shutdown 22/07/19 13:14:50
ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
was:
I have a Spark SQL Program which run in Spark Cluster. Even though the program
is finished without any Error, after finishing State of the Application becomes
Killed. It shows this error on the log:
22/07/19 13:14:50 INFO CoarseGrainedExecutorBackend: Got assigned task 10686
22/07/19 13:14:50 INFO Executor: Running task 232.0 in stage 137.0 (TID 10686)
22/07/19 13:14:50 INFO CoarseGrainedExecutorBackend: Got assigned task 10687
22/07/19 13:14:50 INFO Executor: Running task 233.0 in stage 137.0 (TID 10687)
22/07/19 13:14:50 INFO PythonRunner: Times: total = 6, boot = 2, init = 4,
finish = 0 22/07/19 13:14:50 INFO Executor: Finished task 232.0 in stage 137.0
(TID 10686). 1785 bytes result sent to driver 22/07/19 13:14:50 INFO
PythonRunner: Times: total = 10, boot = 9, init = 1, finish = 0 22/07/19
13:14:50 INFO Executor: Finished task 233.0 in stage 137.0 (TID 10687). 1785
bytes result sent to driver 22/07/19 13:14:50 INFO
CoarseGrainedExecutorBackend: Got assigned task 10688 22/07/19 13:14:50 INFO
Executor: Running task 234.0 in stage 137.0 (TID 10688) 22/07/19 13:14:50 INFO
CoarseGrainedExecutorBackend: Got assigned task 10689 22/07/19 13:14:50 INFO
Executor: Running task 235.0 in stage 137.0 (TID 10689) 22/07/19 13:14:50 INFO
PythonRunner: Times: total = 1, boot = 1, init = 0, finish = 0 22/07/19
13:14:50 INFO Executor: Finished task 235.0 in stage 137.0 (TID 10689). 1785
bytes result sent to driver 22/07/19 13:14:50 INFO
CoarseGrainedExecutorBackend: Got assigned task 10690 22/07/19 13:14:50 INFO
Executor: Running task 236.0 in stage 137.0 (TID 10690) 22/07/19 13:14:50 WARN
JdbcUtils: Requested isolation level 1 is not supported; falling back to
default isolation level 2 22/07/19 13:14:50 INFO PythonRunner: Times: total =
42, boot = -13, init = 55, finish = 0 22/07/19 13:14:50 INFO Executor: Finished
task 231.0 in stage 137.0 (TID 10685). 1785 bytes result sent to driver
22/07/19 13:14:50 INFO CoarseGrainedExecutorBackend: Got assigned task 10691
22/07/19 13:14:50 INFO Executor: Running task 237.0 in stage 137.0 (TID 10691)
22/07/19 13:14:50 INFO PythonRunner: Times: total = 43, boot = -4, init = 47,
finish = 0 22/07/19 13:14:50 INFO Executor: Finished task 234.0 in stage 137.0
(TID 10688). 1785 bytes result sent to driver 22/07/19 13:14:50 INFO
CoarseGrainedExecutorBackend: Got assigned task 10692 22/07/19 13:14:50 INFO
Executor: Running task 238.0 in stage 137.0 (TID 10692) 22/07/19 13:14:50 INFO
PythonRunner: Times: total = 43, boot = 2, init = 41, finish = 0 22/07/19
13:14:50 INFO Executor: Finished task 236.0 in stage 137.0 (TID 10690). 1785
bytes result sent to driver 22/07/19 13:14:50 INFO
CoarseGrainedExecutorBackend: Got assigned task 10693 22/07/19 13:14:50 INFO
Executor: Running task 239.0 in stage 137.0 (TID 10693) 22/07/19 13:14:50 INFO
JDBCRDD: closed connection 22/07/19 13:14:50 INFO PythonRunner: Times: total =
44, boot = 3, init = 41, finish = 0 22/07/19 13:14:50 INFO Executor: Finished
task 237.0 in stage 137.0 (TID 10691). 1785 bytes result sent to driver
22/07/19 13:14:50 INFO PythonRunner: Times: total = 44, boot = 2, init = 42,
finish = 0 22/07/19 13:14:50 INFO Executor: Finished task 238.0 in stage 137.0
(TID 10692). 1785 bytes result sent to driver 22/07/19 13:14:50 INFO Executor:
Finished task 219.0 in stage 137.0 (TID 10673). 1785 bytes result sent to
driver 22/07/19 13:14:50 INFO PythonRunner: Times: total = 42, boot = 2, init =
40, finish = 0 22/07/19 13:14:50 INFO Executor: Finished task 239.0 in stage
137.0 (TID 10693). 1785 bytes result sent to driver 22/07/19 13:14:50 INFO
CoarseGrainedExecutorBackend: Driver commanded a shutdown 22/07/19 13:14:50
ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> --------------------------------------------------------
>
> Key: SPARK-39815
> URL: https://issues.apache.org/jira/browse/SPARK-39815
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 3.3.0
> Environment: Ubuntu 20.04
> Python 3.8.10
> Java 8
>
> Reporter: Marzieh
> Priority: Major
>
> h1. I have a Spark SQL Program which run in Spark Cluster. Even though the
> program is finished without any Error, after finishing State of the
> Application becomes Killed. It shows this error on the log:
>
> 22/07/19 13:14:50 INFO CoarseGrainedExecutorBackend: Got assigned task 10686
> 22/07/19 13:14:50 INFO Executor: Running task 232.0 in stage 137.0 (TID
> 10686) 22/07/19 13:14:50 INFO CoarseGrainedExecutorBackend: Got assigned task
> 10687 22/07/19 13:14:50 INFO Executor: Running task 233.0 in stage 137.0 (TID
> 10687) 22/07/19 13:14:50 INFO PythonRunner: Times: total = 6, boot = 2, init
> = 4, finish = 0 22/07/19 13:14:50 INFO Executor: Finished task 232.0 in stage
> 137.0 (TID 10686). 1785 bytes result sent to driver 22/07/19 13:14:50 INFO
> PythonRunner: Times: total = 10, boot = 9, init = 1, finish = 0 22/07/19
> 13:14:50 INFO Executor: Finished task 233.0 in stage 137.0 (TID 10687). 1785
> bytes result sent to driver 22/07/19 13:14:50 INFO
> CoarseGrainedExecutorBackend: Got assigned task 10688 22/07/19 13:14:50 INFO
> Executor: Running task 234.0 in stage 137.0 (TID 10688) 22/07/19 13:14:50
> INFO CoarseGrainedExecutorBackend: Got assigned task 10689 22/07/19 13:14:50
> INFO Executor: Running task 235.0 in stage 137.0 (TID 10689) 22/07/19
> 13:14:50 INFO PythonRunner: Times: total = 1, boot = 1, init = 0, finish = 0
> 22/07/19 13:14:50 INFO Executor: Finished task 235.0 in stage 137.0 (TID
> 10689). 1785 bytes result sent to driver 22/07/19 13:14:50 INFO
> CoarseGrainedExecutorBackend: Got assigned task 10690 22/07/19 13:14:50 INFO
> Executor: Running task 236.0 in stage 137.0 (TID 10690) 22/07/19 13:14:50
> WARN JdbcUtils: Requested isolation level 1 is not supported; falling back to
> default isolation level 2 22/07/19 13:14:50 INFO PythonRunner: Times: total =
> 42, boot = -13, init = 55, finish = 0 22/07/19 13:14:50 INFO Executor:
> Finished task 231.0 in stage 137.0 (TID 10685). 1785 bytes result sent to
> driver 22/07/19 13:14:50 INFO CoarseGrainedExecutorBackend: Got assigned task
> 10691 22/07/19 13:14:50 INFO Executor: Running task 237.0 in stage 137.0 (TID
> 10691) 22/07/19 13:14:50 INFO PythonRunner: Times: total = 43, boot = -4,
> init = 47, finish = 0 22/07/19 13:14:50 INFO Executor: Finished task 234.0 in
> stage 137.0 (TID 10688). 1785 bytes result sent to driver 22/07/19 13:14:50
> INFO CoarseGrainedExecutorBackend: Got assigned task 10692 22/07/19 13:14:50
> INFO Executor: Running task 238.0 in stage 137.0 (TID 10692) 22/07/19
> 13:14:50 INFO PythonRunner: Times: total = 43, boot = 2, init = 41, finish =
> 0 22/07/19 13:14:50 INFO Executor: Finished task 236.0 in stage 137.0 (TID
> 10690). 1785 bytes result sent to driver 22/07/19 13:14:50 INFO
> CoarseGrainedExecutorBackend: Got assigned task 10693 22/07/19 13:14:50 INFO
> Executor: Running task 239.0 in stage 137.0 (TID 10693) 22/07/19 13:14:50
> INFO JDBCRDD: closed connection 22/07/19 13:14:50 INFO PythonRunner: Times:
> total = 44, boot = 3, init = 41, finish = 0 22/07/19 13:14:50 INFO Executor:
> Finished task 237.0 in stage 137.0 (TID 10691). 1785 bytes result sent to
> driver 22/07/19 13:14:50 INFO PythonRunner: Times: total = 44, boot = 2, init
> = 42, finish = 0 22/07/19 13:14:50 INFO Executor: Finished task 238.0 in
> stage 137.0 (TID 10692). 1785 bytes result sent to driver 22/07/19 13:14:50
> INFO Executor: Finished task 219.0 in stage 137.0 (TID 10673). 1785 bytes
> result sent to driver 22/07/19 13:14:50 INFO PythonRunner: Times: total = 42,
> boot = 2, init = 40, finish = 0 22/07/19 13:14:50 INFO Executor: Finished
> task 239.0 in stage 137.0 (TID 10693). 1785 bytes result sent to driver
> 22/07/19 13:14:50 INFO CoarseGrainedExecutorBackend: Driver commanded a
> shutdown 22/07/19 13:14:50 ERROR CoarseGrainedExecutorBackend: RECEIVED
> SIGNAL TERM
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
