Abacn commented on code in PR #37830:
URL: https://github.com/apache/beam/pull/37830#discussion_r2921143297
##########
sdks/python/apache_beam/typehints/schemas.py:
##########
@@ -991,6 +992,33 @@ def to_language_type(self, value):
return Timestamp(seconds=int(value.seconds), micros=int(value.micros))
+# @LogicalType._register_internal
Review Comment:
(pending debugging cleanup)
##########
sdks/python/apache_beam/typehints/schemas.py:
##########
@@ -34,6 +34,7 @@
bytes <-----> BYTES
ByteString ------> BYTES
Timestamp <-----> LogicalType(urn="beam:logical_type:micros_instant:v1")
+ datetime.date <---> LogicalType(urn="beam:logical_type:date:v1")
Review Comment:
Most Java SDK's IO uses joda data and time, and are serialized as joda
timestamp. Then JDBCDateType and JDBCTimeType were created to accomadate them.
WIth the introducation of "ture" portable data/time logical types, we need
to make sure it doesn't break existing Python JdbcIO. We can run
https://github.com/apache/beam/actions/workflows/beam_PostCommit_Python_Xlang_Gcp_Direct.yml
against this PR to test
##########
model/pipeline/src/main/proto/org/apache/beam/model/pipeline/v1/schema.proto:
##########
@@ -174,6 +174,13 @@ message LogicalTypes {
// A variable-length string with its maximum length as the argument.
VAR_CHAR = 7 [(org.apache.beam.model.pipeline.v1.beam_urn) =
"beam:logical_type:var_char:v1"];
+
+ // A URN for Date type
+ // - Representation type: INT64
+ // - A date without a timezone, represented by the number of days
+ // since the epoch.
+ DATE = 8 [(org.apache.beam.model.pipeline.v1.beam_urn) =
Review Comment:
Consider adding TIME as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
