Joseph created AIRFLOW-5664: ------------------------------- Summary: postgres_to_gcs operator drops milliseconds from timestamps Key: AIRFLOW-5664 URL: https://issues.apache.org/jira/browse/AIRFLOW-5664 Project: Apache Airflow Issue Type: Bug Components: operators Affects Versions: 1.10.5 Reporter: Joseph
Postgres stores timestamps with microsecond resolution. When using the postgres_to_gcs operator, timestamps are converted to epoch/unix time using the datetime.timetuple() method. This method drops the microseconds and so you'll end up with a storage object that looks like this: {code:java} {"id": 1, "last_modified": 1571038537.0} {"id": 2, "last_modified": 1571038537.0} {"id": 3, "last_modified": 1571038537.0} {code} When it should look like this: {code:java} {"id": 1, "last_modified": 1571038537.123} {"id": 2, "last_modified": 1571038537.400} {"id": 3, "last_modified": 1571038537.455} {code} It would be useful to keep the timestamps' full resolution. I believe the same issue may occur with airflow.operators.mysql_to_gcs. -- This message was sent by Atlassian Jira (v8.3.4#803005)