Ottomata has uploaded a new change for review. (
https://gerrit.wikimedia.org/r/344990 )
Change subject: Prune old druid request logs
......................................................................
Prune old druid request logs
There may be a way to prune request logs using log4j,
but until it is documented, we don't know how!
Using a cron to do so.
Bug: T155491
Change-Id: I7b870029e902606f4865ac607b1e0213e91b72bd
---
M modules/druid/manifests/init.pp
1 file changed, 15 insertions(+), 1 deletion(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/puppet
refs/changes/90/344990/1
diff --git a/modules/druid/manifests/init.pp b/modules/druid/manifests/init.pp
index 85d4867..9bfbc31 100644
--- a/modules/druid/manifests/init.pp
+++ b/modules/druid/manifests/init.pp
@@ -266,7 +266,7 @@
content => template('druid/runtime.properties.erb'),
}
- # Indexing logs are not pruned by Druid. Install a cron job to do so.
+ # Indexing and request logs are not pruned by Druid. Install cron jobs to
do so.
$indexer_log_retention_days = 62
cron { 'prune_old_druid_indexer_logs':
command => "/usr/bin/find
${runtime_properties['druid.indexer.logs.directory']} -mtime
+${indexer_log_retention_days} -exec /bin/rm {} \\;",
@@ -274,4 +274,18 @@
minute => 0,
user => 'druid',
}
+ # Request logs are in /var/log/druid, along with daemon logs. Since
daemon logs are managed
+ # by log4j, we don't want to accidentally prune them with this cron job.
There are two
+ # formats of request log fine names, e.g. 2016-07-22T00:00:00.000Z and
2016-09-13.log.
+ # This cron job will prune both by grepping on a matching pattern and
piping to xargs rm,
+ # instead of using -exec rm.
+ # NOTE: There should be a way to manage Druid request logs via log4j, just
like daemons,
+ # but it is apparently undocumented.
+ $request_log_retention_days = 62
+ cron { 'prune_old_druid_request_logs':
+ command => "/usr/bin/find /var/log/druid -mtime
+${request_log_retention_days} | /bin/grep -E '2.*(Z|.log)$' | /usr/bin/xargs
rm",
+ hour => 0,
+ minute => 0,
+ user => 'druid',
+ }
}
--
To view, visit https://gerrit.wikimedia.org/r/344990
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7b870029e902606f4865ac607b1e0213e91b72bd
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Ottomata <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
