Hi, It now appears that if we want a clean solution then we have to add a fixed size primitive type to Beam atomic types. Or, we then have a millis_instant logical type that does not have to_language_type and to_representation_type implementations. Any suggestions are welcome!
Best, Yi On Thu, Aug 18, 2022 at 10:07 AM Yi Hu <[email protected]> wrote: > > > On Wed, Aug 17, 2022 at 5:14 PM Chamikara Jayalath <[email protected]> > wrote: > >> >> I think this is fine (even though it would add a small perf hit to >> JdbcIO.Read). We also probably should make this conversion a utility method >> that can be used elsewhere when we need to encode datetime fields. >> We should also document that "beam:logical_type:datetime:v1" is not >> portable (till we fix the incompatibility). >> >> > +1 for utility method and documentation. > If we were to change JDBC instead of make millis_instant compatible to > InstantCoder, this would only fix JDBC cross-language timestamps. I expect > for other IO connectors this is still a problem and that is why I would > like to take a generic approach. In general, inside each sdk we would like > to follow the language specific convention of that sdk. I remember a > related discussion about the timestamp types: > https://github.com/apache/beam/pull/17380#discussion_r852422314 which > reached a conclusion that follows the language convention on timestamp > values, e.g. use milli precision (long backed) Instant in Java; micro > precision (float backed) timestamp in python. > > Best, > Yi >
