object MyDatabseSingleton {
@transient
lazy val dbConn = DB.connect(…)
`transient` marks the variable to be excluded from serialization
and `lazy` would open connection only when it's needed and also makes
sure that the val is thread-safe
Hi Patrick,
This object must be serializable right? I wonder if I will access to this
object in my driver(since it is getting created on the executor side) so I
can close when I am done with my batch?
Thanks!
On Mon, Jul 30, 2018 at 7:37 AM, Patrick McGloin
wrote:
> You could use an object in
You could use an object in Scala, of which only one instance will be
created on each JVM / Executor. E.g.
object MyDatabseSingleton {
var dbConn = ???
}
On Sat, 28 Jul 2018, 08:34 kant kodali, wrote:
> Hi All,
>
> I understand creating a connection forEachPartition but I am wondering can
>
Hi All,
I understand creating a connection forEachPartition but I am wondering can
I create one DB connection per executor and close it after the job is done?
any sample code would help. you can imagine I am running a simple batch
processing application.
Thanks!
