Hi ,

I have primary key on sql table iam trying to insert Dataframe into table
using insertIntoJDBC.

I could see failure instances in logs but still spark job is getting
successful. Do you know  how can we handle in code to make it fail?



16/09/19 18:52:51 INFO TaskSetManager: Starting task 0.99 in stage 82.0 (TID
5032, 10.0.0.24, partition 0,PROCESS_LOCAL, 11300 bytes)
16/09/19 18:52:52 INFO TaskSetManager: Lost task 0.99 in stage 82.0 (TID
5032) on executor 10.0.0.24: java.sql.BatchUpdateException (Violation of
PRIMARY KEY constraint 'pk_unique'. Cannot insert duplicate key in object
'table_name'. The duplicate key value is (2016-09-13 04:00, 2016-09-13
04:15, 5816324).) [duplicate 99]
16/09/19 18:52:52 ERROR TaskSetManager: Task 0 in stage 82.0 failed 100
times; aborting job
16/09/19 18:52:52 INFO YarnClusterScheduler: Removed TaskSet 82.0, whose
tasks have all completed, from pool 
16/09/19 18:52:52 INFO YarnClusterScheduler: Cancelling stage 82
16/09/19 18:52:52 INFO DAGScheduler: ResultStage 82 (insertIntoJDBC at
sparkjob.scala:143) failed in 9.440 s
16/09/19 18:52:52 INFO DAGScheduler: Job 19 failed: insertIntoJDBC at
sparkjob.scala:143, took 9.449118 s
16/09/19 18:52:52 INFO ApplicationMaster: Final app status: SUCCEEDED,
exitCode: 0
16/09/19 18:52:52 INFO SparkContext: Invoking stop() from shutdown hook


Regards,
Sai



-----
Sai Ganesh
--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Job-not-failing-tp27756.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Reply via email to