[
https://issues.apache.org/jira/browse/AIRFLOW-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomasz Urbaszek resolved AIRFLOW-5664.
--------------------------------------
Fix Version/s: 2.0.0
Resolution: Done
> postgres_to_gcs operator drops milliseconds from timestamps
> -----------------------------------------------------------
>
> Key: AIRFLOW-5664
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5664
> Project: Apache Airflow
> Issue Type: Bug
> Components: gcp, operators
> Affects Versions: 1.10.5
> Reporter: Joseph
> Assignee: Oluwafemi Sule
> Priority: Blocker
> Fix For: 2.0.0
>
>
> Postgres stores timestamps with microsecond resolution. When using the
> postgres_to_gcs operator, timestamps are converted to epoch/unix time using
> the datetime.timetuple() method. This method drops the microseconds and so
> you'll end up with a storage object that looks like this:
> {code:java}
> {"id": 1, "last_modified": 1571038537.0}
> {"id": 2, "last_modified": 1571038537.0}
> {"id": 3, "last_modified": 1571038537.0}
> {code}
> When it should look like this:
> {code:java}
> {"id": 1, "last_modified": 1571038537.123}
> {"id": 2, "last_modified": 1571038537.400}
> {"id": 3, "last_modified": 1571038537.455}
> {code}
> It would be useful to keep the timestamps' full resolution.
> I believe the same issue may occur with airflow.operators.mysql_to_gcs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
