Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/21590#discussion_r196489749
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala
---
@@ -109,6 +134,20 @@ class JDBCOptions(
s"When reading JDBC data sources, users need to specify all or none
for the following " +
s"options: '$JDBC_PARTITION_COLUMN', '$JDBC_LOWER_BOUND',
'$JDBC_UPPER_BOUND', " +
s"and '$JDBC_NUM_PARTITIONS'")
+
+ require(!(query.isDefined && partitionColumn.isDefined),
+ s"""
+ |Options '$JDBC_QUERY_STRING' and '$JDBC_PARTITION_COLUMN' can not
be specified together.
+ |Please define the query using `$JDBC_TABLE_NAME` option instead
and make sure to qualify
+ |the partition columns using the supplied subquery alias to resolve
any ambiguity.
+ |Example :
+ |spark.read.format("jdbc")
+ | .option("dbtable", "(select c1, c2 from t1) as subq")
+ | .option("partitionColumn", "subq.c1"
+ | .load()
+ """.stripMargin
+ )
--- End diff --
@maropu So since the we auto generate a subquery alias here for easy of
use, we r disallowing the query option together with partition columns. As
users wouldn't know how to qualify the partition columns given the suquery
alias is generated implicitly. In this case, we ask them to use the existing
dbtable to specify the query where they are in control to specify the alias
themselves. Another option i considered is to introduce "queryAlias" as another
option. But thought to avoid it for simplicity.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
