This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 1a536f01ead3 [SPARK-48407][SQL][DOCS] Teradata: Document Type
Conversion rules between Spark SQL and teradata
1a536f01ead3 is described below
commit 1a536f01ead35b770467381c476e093338d81e7c
Author: Kent Yao <y...@apache.org>
AuthorDate: Fri May 24 15:56:19 2024 -0700
[SPARK-48407][SQL][DOCS] Teradata: Document Type Conversion rules between
Spark SQL and teradata
### What changes were proposed in this pull request?
This PR adds documentation for the builtin teradata jdbc dialect's data
type conversion rules
### Why are the changes needed?
doc improvement
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #46728 from yaooqinn/SPARK-48407.
Authored-by: Kent Yao <y...@apache.org>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
docs/sql-data-sources-jdbc.md | 214 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 214 insertions(+)
diff --git a/docs/sql-data-sources-jdbc.md b/docs/sql-data-sources-jdbc.md
index 371dc0595071..9ffd96cd40ee 100644
--- a/docs/sql-data-sources-jdbc.md
+++ b/docs/sql-data-sources-jdbc.md
@@ -1991,3 +1991,217 @@ The Spark Catalyst data types below are not supported
with suitable DB2 types.
- NullType
- ObjectType
- VariantType
+
+### Mapping Spark SQL Data Types from Teradata
+
+The below table describes the data type conversions from Teradata data types
to Spark SQL Data Types,
+when reading data from a Teradata table using the built-in jdbc data source
with the [Teradata JDBC
Driver](https://mvnrepository.com/artifact/com.teradata.jdbc/terajdbc)
+as the activated JDBC Driver.
+
+<table>
+ <thead>
+ <tr>
+ <th><b>Teradata Data Type</b></th>
+ <th><b>Spark SQL Data Type</b></th>
+ <th><b>Remarks</b></th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td>BYTEINT</td>
+ <td>ByteType</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>SMALLINT</td>
+ <td>ShortType</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>INTEGER, INT</td>
+ <td>IntegerType</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>BIGINT</td>
+ <td>LongType</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>REAL, DOUBLE PRECISION, FLOAT</td>
+ <td>DoubleType</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>DECIMAL, NUMERIC, NUMBER</td>
+ <td>DecimalType</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>DATE</td>
+ <td>DateType</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>TIMESTAMP, TIMESTAMP WITH TIME ZONE</td>
+ <td>TimestampType</td>
+ <td>(Default)preferTimestampNTZ=false or
spark.sql.timestampType=TIMESTAMP_LTZ</td>
+ </tr>
+ <tr>
+ <td>TIMESTAMP, TIMESTAMP WITH TIME ZONE</td>
+ <td>TimestampNTZType</td>
+ <td>preferTimestampNTZ=true or spark.sql.timestampType=TIMESTAMP_NTZ</td>
+ </tr>
+ <tr>
+ <td>TIME, TIME WITH TIME ZONE</td>
+ <td>TimestampType</td>
+ <td>(Default)preferTimestampNTZ=false or
spark.sql.timestampType=TIMESTAMP_LTZ</td>
+ </tr>
+ <tr>
+ <td>TIME, TIME WITH TIME ZONE</td>
+ <td>TimestampNTZType</td>
+ <td>preferTimestampNTZ=true or spark.sql.timestampType=TIMESTAMP_NTZ</td>
+ </tr>
+ <tr>
+ <td>CHARACTER(n), CHAR(n), GRAPHIC(n)</td>
+ <td>CharType(n)</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>VARCHAR(n), VARGRAPHIC(n)</td>
+ <td>VarcharType(n)</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>BYTE(n), VARBYTE(n)</td>
+ <td>BinaryType</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>CLOB</td>
+ <td>StringType</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>BLOB</td>
+ <td>BinaryType</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>INTERVAL Data Types</td>
+ <td>-</td>
+ <td>The INTERVAL data types are unknown yet</td>
+ </tr>
+ <tr>
+ <td>Period Data Types, ARRAY, UDT</td>
+ <td>-</td>
+ <td>Not Supported</td>
+ </tr>
+ </tbody>
+</table>
+
+### Mapping Spark SQL Data Types to Teradata
+
+The below table describes the data type conversions from Spark SQL Data Types
to Teradata data types,
+when creating, altering, or writing data to a Teradata table using the
built-in jdbc data source with
+the [Teradata JDBC
Driver](https://mvnrepository.com/artifact/com.teradata.jdbc/terajdbc) as the
activated JDBC Driver.
+
+<table>
+ <thead>
+ <tr>
+ <th><b>Spark SQL Data Type</b></th>
+ <th><b>Teradata Data Type</b></th>
+ <th><b>Remarks</b></th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td>BooleanType</td>
+ <td>CHAR(1)</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>ByteType</td>
+ <td>BYTEINT</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>ShortType</td>
+ <td>SMALLINT</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>IntegerType</td>
+ <td>INTEGER</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>LongType</td>
+ <td>BIGINT</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>FloatType</td>
+ <td>REAL</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>DoubleType</td>
+ <td>DOUBLE PRECISION</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>DecimalType(p, s)</td>
+ <td>DECIMAL(p,s)</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>DateType</td>
+ <td>DATE</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>TimestampType</td>
+ <td>TIMESTAMP</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>TimestampNTZType</td>
+ <td>TIMESTAMP</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>StringType</td>
+ <td>VARCHAR(255)</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>BinaryType</td>
+ <td>BLOB</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>CharType(n)</td>
+ <td>CHAR(n)</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>VarcharType(n)</td>
+ <td>VARCHAR(n)</td>
+ <td></td>
+ </tr>
+ </tbody>
+</table>
+
+The Spark Catalyst data types below are not supported with suitable Teradata
types.
+
+- DayTimeIntervalType
+- YearMonthIntervalType
+- CalendarIntervalType
+- ArrayType
+- MapType
+- StructType
+- UserDefinedType
+- NullType
+- ObjectType
+- VariantType
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
