Abacn commented on issue #28359:
URL: https://github.com/apache/beam/issues/28359#issuecomment-1711874721
Note: the following workaround can be used for columns involving Date (or
Time) type, that is register your own logical type:
```python
@LogicalType.register_logical_type
class DateType(LogicalType[datetime.date, MillisInstant, str]):
def __init__(self, unused=""):
pass
@classmethod
def representation_type(cls):
# type: () -> type
return Timestamp
@classmethod
def urn(cls):
return "beam:logical_type:javasdk:v1"
@classmethod
def language_type(cls):
return datetime.date
def to_representation_type(self, value):
# type: (datetime.date) -> Timestamp
return Timestamp.from_utc_datetime(datetime.datetime.combine(value,
datetime.datetime.min.time(), tzinfo=datetime.timezone.utc))
def to_language_type(self, value):
# type: (Timestamp) -> datetime.date
return value.to_utc_datetime().date()
@classmethod
def argument_type(cls):
return str
def argument(self):
return ""
@classmethod
def _from_typing(cls, typ):
return cls()
```
-----
I recall why the Date type support was incomplete. The Java JdbcIO
implemented Date and Time with non-portable logical type backed by joda
Instant, while the Beam portable Date and Time logical type are backed by more
mortan java.time.localDate or localTime. We cannot simply change the Java
JdbcIO due to concern of breaking change. This involves two difficult
compatibility issues
- migrating from joda time to java.time -
[go/beampr/19215](https://goto.google.com/beampr/19215)
- migrate from non-portable to portable logical types
One fix could be done for transition is that adding a flag to IOs that guide
them to produce java.time results and use portable logical types. This would
enable Python implementation (and also eliminate the need of call
`LogicalType.register_logical_type(MillisInstant)` if need to overwrite logical
type mapping.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
