On 11 February 2016 at 01:53, Sam Ruby <[email protected]> wrote: > On Wed, Feb 10, 2016 at 8:23 PM, sebb <[email protected]> wrote: >> Most of the public json data files have dates in them that show when >> the raw data was last updated. >> The files themselves are only replaced when the data in them changes, >> so the If-Modified-Since check works properly. >> >> Now some data sources change quite infrequently, so it can be hard to >> determine of the file really is up to date or the cron job has not >> been run recently. >> >> There are public log files, however these would need to be analysed to >> determine if the job completed OK, and even an empty file could be >> associated with a failed job if the error was not caught and logged - >> i.e. it only went to stderr] > > The cron jobs direct stderr to the log files (2>&1): > > https://github.com/apache/infrastructure-puppet/blob/deployment/modules/whimsy_server/manifests/cronjobs.pp
OK. But the issue still arises - the contents of the log file have to be analysed in order to determine if the output was created OK. However if the job added a marker that would be trivial. >> Possible solutions might be: >> - scripts add a standard entry to each log file on successful completion. >> This could include the start and end times of the job. >> Quite simple to implement, but a bit awkward to use. >> >> - scripts update a shared json file on successful completion. >> More care needed to implement (locking needed), but much easier to use. >> >> Thoughts? > > The log files themselves have an mtime: https://whimsy-test.apache.org/logs/ Yes, but that only shows when cron last ran the job, not when it last completed OK. Also there log file names don't agree with the json files (that could easily be fixed). > status/monitors/public_json.rb could issue a warning (or higher?) if > the log has not been updated recently. Is the monitor itself monitored? > Depending on the level chosen, this could trigger an alert. That is a good idea. > - Sam Ruby
