Have you tried the following ?
val result = rdd.map(line => { val table = Util.Connection.getTable("user")
...
Util.Connection.close() }
On Wed, Oct 15, 2014 at 6:09 AM, Fengyun RAO <[email protected]> wrote:
> In order to share an HBase connection pool, we create an object
>
> Object Util {
> val HBaseConf = HBaseConfiguration.create
> val Connection= HConnectionManager.createConnection(HBaseConf)
> }
>
> which would be shared among tasks on the same executor. e.g.
>
> val result = rdd.map(line => {
> val table = Util.Connection.getTable("user")
> ...
> }
>
> However, we don’t how to close the Util.Connection.
> If we write Util.Connection.close() in the main function,
> it’ll only run on the driver, not the executor.
>
> So, How to make sure every Connection closed before exist?
>
>
