Hi Patrick,
This object must be serializable right? I wonder if I will access to this
object in my driver(since it is getting created on the executor side) so I
can close when I am done with my batch?
Thanks!
On Mon, Jul 30, 2018 at 7:37 AM, Patrick McGloin <mcgloin.patr...@gmail.com>
wrote:
> You could use an object in Scala, of which only one instance will be
> created on each JVM / Executor. E.g.
>
> object MyDatabseSingleton {
> var dbConn = ???
> }
>
> On Sat, 28 Jul 2018, 08:34 kant kodali, <kanth...@gmail.com> wrote:
>
>> Hi All,
>>
>> I understand creating a connection forEachPartition but I am wondering
>> can I create one DB connection per executor and close it after the job is
>> done? any sample code would help. you can imagine I am running a simple
>> batch processing application.
>>
>> Thanks!
>>
>
