[
https://issues.apache.org/jira/browse/SPARK-18593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-18593:
----------------------------------
Summary: JDBCRDD fails to filter CHAR type column for PostgreSQL (was:
Unable to use filter function if the data length is one.)
> JDBCRDD fails to filter CHAR type column for PostgreSQL
> -------------------------------------------------------
>
> Key: SPARK-18593
> URL: https://issues.apache.org/jira/browse/SPARK-18593
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.2, 1.6.3
> Reporter: Durga Prasad Gunturu
> Priority: Minor
>
> when using filter using jdbc.. unable to retrieve data if the column value is
> having only value of length one.
> for example:
> Below code works because the filter condition data is of length > 1
> val dataFromTable =
> sqlContext.read.jdbc("jdbc:postgresql://localhost:5433/postgres",
> "stocknames", defaultProps)
> dataFromTable.filter(dataFromTable("grouptype")==="ZP").show()
> result:
> +------------+----------+--------------------+------+---------+---------+------------+--------------------+----------+
> |securitycode|securityid| securityname|status|grouptype|facevalue|
> isinno| industry|instrument|
> +------------+----------+--------------------+------+---------+---------+------------+--------------------+----------+
> | 504370| ZARDIINV|ARDI INVESTMENT &...|Active| ZP| 10.0|
> |Misc.Commercial S...| Equity|
> | 512221| TARCF|TARRIF CINE &...|Active| ZP|
> 10.0|INE177E01010|Finance (includin...| Equity|
> | 512505| MEENST|MEENAKSHI STEEL I...|Active| ZP| 10.0|
> |Iron & Steel ...| Equity|
> | 514060| EVERTEX|Evergreen Textile...|Active| ZP|
> 10.0|INE229N01010| Textiles| Equity|
> | 526085| SGARRES|SAGAR TOURIST RES...|Active| ZP| 10.0|NA
> | Hotels| Equity|
> +------------+----------+--------------------+------+---------+---------+------------+--------------------+----------+
> But if the same filter condition code is of length 1, no data is filtered.
> eg:
> val dataFromTable =
> sqlContext.read.jdbc("jdbc:postgresql://localhost:5433/postgres",
> "stocknames", defaultProps)
> dataFromTable.filter(dataFromTable("grouptype")==="A").show()
> not getting any output
> +------------+----------+------------+------+---------+---------+------+--------+----------+
> |securitycode|securityid|securityname|status|grouptype|facevalue|isinno|industry|instrument|
> +------------+----------+------------+------+---------+---------+------+--------+----------+
> +------------+----------+------------+------+---------+---------+------+--------+----------+
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
