jroachgolf84 commented on PR #50791: URL: https://github.com/apache/airflow/pull/50791#issuecomment-2895204480
> An obvious example is, s3 file lands with timestamp X. Airflow records the event timestamp 10 seconds after X. But between X and X + 10, more files land in the bucket. So if you take airflow's event timestamp as the high watermark, you'll miss files. I included a "safe" timestamp for this, this is most certainly top-of-mind. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
