ahmedabu98 commented on code in PR #39615:
URL: https://github.com/apache/beam/pull/39615#discussion_r3760595406
##########
sdks/python/apache_beam/io/gcp/bigquery_tools.py:
##########
@@ -125,6 +125,13 @@
"GEOGRAPHY": str,
}
+# Duplicated logic with io/gcp/bigquery_change_history.py
+# Default table expiration for auto-created temp datasets: 24 hours in ms.
+# Tables created in the dataset auto-expire after this duration if not
+# explicitly deleted, acting as a safety net for orphaned temp tables
+# (e.g. pipeline crash before cleanup runs).
+_DEFAULT_TABLE_EXPIRATION_MS = 24 * 60 * 60 * 1000
Review Comment:
I haven't encountered a load/export job that takes longer than a day. But I
guess it's possible. A 7-day expiration window would be safer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
