[jira] [Assigned] (SPARK-17955) Use the same read path in DataFrameReader.jdbc and DataFrameReader.format("jdbc")
[ https://issues.apache.org/jira/browse/SPARK-17955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17955: Assignee: Apache Spark > Use the same read path in DataFrameReader.jdbc and > DataFrameReader.format("jdbc") > -- > > Key: SPARK-17955 > URL: https://issues.apache.org/jira/browse/SPARK-17955 > Project: Spark > Issue Type: Improvement > Components: SQL >Reporter: Hyukjin Kwon >Assignee: Apache Spark >Priority: Trivial > > It seems APIs in {{DataFrameReader}}/{{DataFrameWriter}} share > {{format("...").load()}} or {{format("...").save()}} APIs for > {{json(...)}}/{{csv(...)}} and etc. > We can share this within {{DataFrameReader.jdbc(...)}} too consistently with > other APIs. > {code} > -// connectionProperties should override settings in extraOptions. > -val params = extraOptions.toMap ++ connectionProperties.asScala.toMap > -val options = new JDBCOptions(url, table, params) > -val relation = JDBCRelation(parts, options)(sparkSession) > -sparkSession.baseRelationToDataFrame(relation) > +// connectionProperties should override settings in extraOptions > +this.extraOptions = this.extraOptions ++ (connectionProperties.asScala) > +// explicit url and dbtable should override all > +this.extraOptions += ("url" -> url, "dbtable" -> table) > +format("jdbc").load() > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-17955) Use the same read path in DataFrameReader.jdbc and DataFrameReader.format("jdbc")
[ https://issues.apache.org/jira/browse/SPARK-17955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17955: Assignee: (was: Apache Spark) > Use the same read path in DataFrameReader.jdbc and > DataFrameReader.format("jdbc") > -- > > Key: SPARK-17955 > URL: https://issues.apache.org/jira/browse/SPARK-17955 > Project: Spark > Issue Type: Improvement > Components: SQL >Reporter: Hyukjin Kwon >Priority: Trivial > > It seems APIs in {{DataFrameReader}}/{{DataFrameWriter}} share > {{format("...").load()}} or {{format("...").save()}} APIs for > {{json(...)}}/{{csv(...)}} and etc. > We can share this within {{DataFrameReader.jdbc(...)}} too consistently with > other APIs. > {code} > -// connectionProperties should override settings in extraOptions. > -val params = extraOptions.toMap ++ connectionProperties.asScala.toMap > -val options = new JDBCOptions(url, table, params) > -val relation = JDBCRelation(parts, options)(sparkSession) > -sparkSession.baseRelationToDataFrame(relation) > +// connectionProperties should override settings in extraOptions > +this.extraOptions = this.extraOptions ++ (connectionProperties.asScala) > +// explicit url and dbtable should override all > +this.extraOptions += ("url" -> url, "dbtable" -> table) > +format("jdbc").load() > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org