jobit mathew created SPARK-29152:
------------------------------------
Summary: Spark Executor Plugin API shutdown is not proper when
dynamic allocation enabled[SPARK-24918]
Key: SPARK-29152
URL: https://issues.apache.org/jira/browse/SPARK-29152
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.4.4
Reporter: jobit mathew
*Issue Description*
Spark Executor Plugin API *shutdown handling is not proper*, when dynamic
allocation enabled .Plugin shutdown method is not processed-while dynamic
allocation is enabled and *executors become dead* after inactive time.
*Test Precondition*
1.Prepared 4 spark applications with executor plugin interface.
First application-SparkExecutorplugin.jar
import org.apache.spark.ExecutorPlugin;
public class ExecutoTest1 implements ExecutorPlugin{
public void init(){
System.out.println("Executor Plugin Initialised.");
}
public void shutdown(){
System.out.println("Executor plugin closed successfully.");
}
}
2. Create the jars with the same and put it in folder /spark/examples/jars
*Test Steps*
1. launch bin/spark-sql with dynamic allocation enabled
./spark-sql --master yarn --conf spark.executor.plugins=ExecutoTest1
--conf="spark.executor.extraClassPath=/opt/HA/C10/install/spark/spark/examples/jars/*"
--conf spark.dynamicAllocation.enabled=true --conf
spark.dynamicAllocation.initialExecutors=2 --conf
spark.dynamicAllocation.minExecutors=0
2 create a table , insert the data and select * from tablename
3.Check the spark UI Jobs tab/SQL tab
4. Check all Executors(executor tab will give all executors details)
application log file for Executor plugin Initialization and Shutdown messages
or operations.
Example
/yarn/logdir/application_1567156749079_0025/container_e02_1567156749079_0025_01_000005/
stdout
5. Wait for the executor to be dead after the inactive time and check the same
container log
6. Kill the spark sql and check the container log for executor plugin shutdown.
*Expect Output*
1. Job should be success. Create table ,insert and select query should be
success.
2.While running query All Executors log should contain the executor plugin
Init and shutdown messages or operations.
"Executor Plugin Initialised.
3.Once the executors are dead ,executor shutdown should call shutdown message
should be there in log file.
“ Executor plugin closed successfully.
4.Once the sql application closed ,executor shutdown should call shutdown
message should be there in log.
“ Executor plugin closed successfully".
*Actual Output*
Shutdown message is not called when executor is dead or after closing the
application
*Observation*
Without dynamic allocation Executor plugin is working fine.But after enabling
dynamic allocation,Executor shutdown is not processed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
