Thanks for the input everyone . What I am trying to understand is if I use oracle to store my data after Spark job processing. And if any spark job fails half the way. What happens then.. Does rollback happens or we have to programatically handle this kind of situation in spark job itself? How transaction are being handled n spark to oracle storage ? My apologies for such a naive question . Thanks, Divya
agreed Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* http://talebzadehmich.wordpress.com NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Technology Ltd, its subsidiaries nor their employees accept any responsibility. *From:* Russell Jurney [mailto:russell.jur...@gmail.com] *Sent:* 19 February 2016 16:49 *To:* Jörn Franke <jornfra...@gmail.com> *Cc:* Divya Gehlot <divya.htco...@gmail.com>; user @spark < user@spark.apache.org> *Subject:* Re: Spark JDBC connection - data writing success or failure cases Oracle is a perfectly reasonable endpoint for publishing data processed in Spark. I've got to assume he's using it that way and not as a stand in for HDFS? On Friday, February 19, 2016, Jörn Franke <jornfra...@gmail.com> wrote: Generally oracle db should not be used as a storage layer for spark due to performance reasons. You should consider HDFS. This will help you also with fault - tolerance. > On 19 Feb 2016, at 03:35, Divya Gehlot <divya.htco...@gmail.com> wrote: > > Hi, > I am a Spark job which connects to RDBMS (in mycase its Oracle). > How can we check that complete data writing is successful? > Can I use commit in case of success or rollback in case of failure ? > > > > Thanks, > Divya --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org -- Russell Jurney twitter.com/rjurney russell.jur...@gmail.com relato.io
