This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.3 by this push:
new fb58c3e5071 [SPARK-38828][PYTHON] Remove TimestampNTZ type Python
support in Spark 3.3
fb58c3e5071 is described below
commit fb58c3e507113e2e9e398cb77703e54603bfa29a
Author: itholic <[email protected]>
AuthorDate: Wed Apr 20 10:49:07 2022 +0900
[SPARK-38828][PYTHON] Remove TimestampNTZ type Python support in Spark 3.3
This PR proposes to remove `TimestampNTZ` type Python support in Spark 3.3
from documentation and `pyspark.sql.types` module.
The purpose of this PR is just hide `TimestampNTZ` type from end-users.
Because the `TimestampNTZ` project is not finished yet:
- Lack Hive metastore support
- Lack JDBC support
- Need to spend time scanning the codebase to find out any missing support.
The current code usages of TimestampType are larger than TimestampNTZType
No.
The existing tests should cover.
Closes #36255 from itholic/SPARK-38828.
Authored-by: itholic <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit 581000de24377ca373df7fa94b214baa7e9b0462)
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/docs/source/reference/pyspark.sql.rst | 1 -
python/pyspark/sql/types.py | 1 -
2 files changed, 2 deletions(-)
diff --git a/python/docs/source/reference/pyspark.sql.rst
b/python/docs/source/reference/pyspark.sql.rst
index 1d34961a91a..adc1958822e 100644
--- a/python/docs/source/reference/pyspark.sql.rst
+++ b/python/docs/source/reference/pyspark.sql.rst
@@ -302,7 +302,6 @@ Data Types
StringType
StructField
StructType
- TimestampNTZType
TimestampType
DayTimeIntervalType
diff --git a/python/pyspark/sql/types.py b/python/pyspark/sql/types.py
index 23e54eb8889..2a41508d634 100644
--- a/python/pyspark/sql/types.py
+++ b/python/pyspark/sql/types.py
@@ -59,7 +59,6 @@ __all__ = [
"BooleanType",
"DateType",
"TimestampType",
- "TimestampNTZType",
"DecimalType",
"DoubleType",
"FloatType",
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
