Eugene-Mark commented on PR #36499:
URL: https://github.com/apache/spark/pull/36499#issuecomment-1129569337
@srowen It should be a Teradata specific issue. I tried to read data with
teradata driver, `terajdbc4` and `tdgssconfig` , the data read contains the
fractional part. The code is sth like:
```
Class.forName("com.teradata.jdbc.TeraDriver")
connection = DriverManager.getconnection(s"jdbc:teradata://$host")
new org.apache.commons.dbutils.QueryRunner().query(connection, "select
* from test_db.test_table", new MapListHandler())
```
The `Number` column type is converted to `java.math.BigDecimal` with scale
reserved, which conforms to the mapping relationship per guide
[here](https://docs.teradata.com/r/Teradata-VantageTM-SQL-Data-Definition-Language-Detailed-Topics/March-2019/CREATE-PROCEDURE-and-REPLACE-PROCEDURE-External-Form/Data-Type-Mapping-Between-SQL-and-Java).
I found Spark didn't provide explicit Teradata connector like what has been
taken care like `DB2ConectionProvider`, `MariaDBConnectionProvider`,
`MSSQLConnectionProvider`, etc.
In conclusion, we're using pure built-in JDBC library to invoke Teradata
which might not be a suggested way, and might result in sth unexpected like the
issue listed above. For a quick fix about the issue I met, maybe we can just
merge this PR. At the same time, I will spend some effort to help introduce
TeradataConnectionProvider to Spark if there is no license related issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
