Re: call a mysql stored procedure from spark

Well that is not the best way as you have to wait for RDBMS to process and populate the temp table. A more sound way would be to write a shell script to talk to RDBMS first and creates and populates that table. Once ready the same shell script can kick off Spark job to read the temp table which

Re: call a mysql stored procedure from spark

Thanks Michael, Michael, Ayan rightly said, yes this stored procedure is invoked from driver, this creates the temporary table is DB, the reason being i want to load some specific data after processing it, i do not wish to bring it in spark, instead want to keep the processing at DB level, later

Re: call a mysql stored procedure from spark

More than technical feasibility, I would ask why to invoke a stored procedure for every row? If not, jdbcRdd is moot point. In case stored procedure should be invoked from driver, it can be easily done. Or at most for each partition, at each executor. On 15 Aug 2016 03:06, "Mich Talebzadeh"

Re: call a mysql stored procedure from spark

Hi, The link deals with JDBC and states: [image: Inline images 1] So it is only SQL. It lacks functionality on Stored procedures with returning result set. This is on an Oracle table scala> var _ORACLEserver = "jdbc:oracle:thin:@rhes564:1521:mydb12" _ORACLEserver: String =

Re: call a mysql stored procedure from spark

As described here , you can use the DataSource API to connect to an external database using JDBC. While the dbtable option is usually just a table name, it can also be any valid SQL command that returns a

Re: call a mysql stored procedure from spark

to be executed in MySQL and results sent back to Spark? No I don't think so. On the other hand a stored procedure is nothing but a compiled code so can you use the raw SQL behind the stored proc? You can certainly send the SQL via JDBC and the RS back. HTH Dr Mich Talebzadeh LinkedIn *

call a mysql stored procedure from spark

Hi, Is there a way to call a stored procedure using spark ? thanks, Sujeet