Dears, I needs to commit DB Transaction for each partition,Not for each row. below didn't work for me.
rdd.mapPartitions(partitionOfRecords => { DBConnectionInit() val results = partitionOfRecords.map(......) DBConnection.commit() results }) Best regards, Ahmed Atef Nawwar Data Management & Big Data Consultant