[
https://issues.apache.org/jira/browse/SPARK-16057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reynold Xin resolved SPARK-16057.
---------------------------------
Resolution: Fixed
Assignee: Xiao Li
Fix Version/s: 2.0.0
> JDBC source: Wrong Partition Generation when numPartitions is More than the
> number of rows between upper and lower bounds
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-16057
> URL: https://issues.apache.org/jira/browse/SPARK-16057
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.0
> Reporter: Xiao Li
> Assignee: Xiao Li
> Priority: Critical
> Fix For: 2.0.0
>
>
> {noformat}
> spark.read.jdbc(
> url = urlWithUserAndPass,
> table = "TEST.seq",
> columnName = "id",
> lowerBound = 1,
> upperBound = 5,
> numPartitions = 10,
> connectionProperties = new Properties)
> {noformat}
> Generated partitions are wrong:
> {noformat}
> Partition 0: id < 1 or id is null
> Partition 1: id >= 1 AND id < 1
> Partition 2: id >= 1 AND id < 1
> Partition 3: id >= 1 AND id < 1
> Partition 4: id >= 1 AND id < 1
> Partition 5: id >= 1 AND id < 1
> Partition 6: id >= 1 AND id < 1
> Partition 7: id >= 1 AND id < 1
> Partition 8: id >= 1 AND id < 1
> Partition 9: id >= 1
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
