Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5488#discussion_r28394137
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JDBCRelation.scala ---
@@ -50,9 +50,11 @@ private[sql] object JDBCRelation {
* Given a partitioning schematic (a column of integral type, a number of
* partitions, and upper and lower bounds on the column's value),
generate
* WHERE clauses for each partition so that each row in the table appears
- * exactly once. The parameters minValue and maxValue are advisory in
that
+ * exactly once. The parameters minValue and maxValue are advisory in
that
* incorrect values may cause the partitioning to be poor, but no data
- * will fail to be represented.
+ * will fail to be represented. Note: the upper and lower bounds are just
+ * used to decide partition stride, not for filtering. So all the rows in
+ * table will be partitioned.
--- End diff --
> The parameters minValue and maxValue are advisory in that incorrect
values may cause the partitioning to be poor, but no data will fail to be
represented.
The sentence above already explains that the filters are only used for
partitioning and that all data will always be returned. I think the best place
to update would be in the [SQL programming
guide](https://github.com/apache/spark/blob/master/docs/sql-programming-guide.md),
in the table under the section "JDBC To Other Databases".
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
