[
https://issues.apache.org/jira/browse/SPARK-16717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Colombo updated SPARK-16717:
----------------------------------
Description:
In JdbcRRD it was possible to use a function to get a JDBC connection. This
allow an external handling of the connections while now this is no more
possible with dataframes.
Please consider an addition to Dataframes for using an externally provided
connectionFactory (such as a connection pool) in order to make data loading
more efficient, avoiding connection close/recreation. Connections should be
taken from provided function and returned to a second function whenever no more
used by the RRD. So this will make jdbc handling more efficient.
I.e. extending DataFrame class with a method like
jdbc(Function0<java.sql.Connection> getConnection, Function0
releaseConnection(java.sql.Connection))
was:
In JdbcRRD it was possible to use a function to get a JDBC connection. This
allow an external handling of the connections while now this is no more
possible with dataframes.
Please consider an addition to Dataframes for using an externally provided
connectionFactory (such as a connection pool) in order to make data loading
more efficient, avoiding connection close/recreation.
> Dataframe (jdbc) is missing a way to link and external function to get a
> connection
> -----------------------------------------------------------------------------------
>
> Key: SPARK-16717
> URL: https://issues.apache.org/jira/browse/SPARK-16717
> Project: Spark
> Issue Type: Improvement
> Affects Versions: 1.6.2
> Reporter: Marco Colombo
>
> In JdbcRRD it was possible to use a function to get a JDBC connection. This
> allow an external handling of the connections while now this is no more
> possible with dataframes.
> Please consider an addition to Dataframes for using an externally provided
> connectionFactory (such as a connection pool) in order to make data loading
> more efficient, avoiding connection close/recreation. Connections should be
> taken from provided function and returned to a second function whenever no
> more used by the RRD. So this will make jdbc handling more efficient.
> I.e. extending DataFrame class with a method like
> jdbc(Function0<java.sql.Connection> getConnection, Function0
> releaseConnection(java.sql.Connection))
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
