Scott Wegner created BEAM-5924:
----------------------------------
Summary: Update github sync metadata timestamp to represent actual
sync time
Key: BEAM-5924
URL: https://issues.apache.org/jira/browse/BEAM-5924
Project: Beam
Issue Type: Sub-task
Components: project-management
Reporter: Scott Wegner
Assignee: Mikhail Gryzykhin
The community metrics data ingestion scripts run in a loop and import all new
data since the last run. We have a [Source Data
Freshness|http://104.154.241.245/d/data-freshness/source-data-freshness]
dashboard which tracks the freshness of import data. A Jenkins job checks this
data regularly to ensure our dashboards don't get stale.
Currently the GitHub "last sync" time actually tracks the last activity found
on import. This means that if there is no activity on GitHub our dashboards
will appear stale and our prober job will fail. This can happen in normal
scenarios (i.e. at night, or during weekends / holidays).
We should update the last sync timestamp logic to be the actual timestamp of
last poll, rather than the last data we found.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
