ArielGlenn has submitted this change and it was merged. (
https://gerrit.wikimedia.org/r/393374 )
Change subject: add top level index files to dump status file tarball for rsync
......................................................................
add top level index files to dump status file tarball for rsync
The top level html and json dump files are locally produced on the
generating host, no need to produce them on all the remote servers.
Instead just rsync them over with the rest of the status files.
Change-Id: I626f875e3056f44564f190b2124f607072831262
---
M modules/dumps/files/generation/rsync-to-peers.sh
1 file changed, 4 insertions(+), 2 deletions(-)
Approvals:
ArielGlenn: Looks good to me, approved
jenkins-bot: Verified
diff --git a/modules/dumps/files/generation/rsync-to-peers.sh
b/modules/dumps/files/generation/rsync-to-peers.sh
index ecd5583..d5b8b4e 100644
--- a/modules/dumps/files/generation/rsync-to-peers.sh
+++ b/modules/dumps/files/generation/rsync-to-peers.sh
@@ -51,8 +51,10 @@
latestrun=$( cd "${xmldumpsdir}/public/${latestwiki}" ; ls -d
[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] | sort | tail -1 )
if [ -n "$latestrun" ]; then
- # ( cd /data/xmldatadumps; /usr/bin/find public/ -maxdepth 3
-regextype sed -regex ".*/20171120/.*\(json\|html\|txt\)" )
- ( cd "$xmldumpsdir"; /usr/bin/find "public/" -maxdepth 3 -regextype
sed -regex ".*/${latestrun}/.*\.\(json\|html\|txt\)" | /usr/bin/xargs -s
1048576 /bin/tar cfp "$tarballpath" )
+ # top-level index files first
+ ( cd "$xmldumpsdir"; /bin/tar cfp "$tarballpath" public/*html
public/*json )
+ # add per-wiki files next: ( cd /data/xmldatadumps; /usr/bin/find
public/ -maxdepth 3 -regextype sed -regex ".*/20171120/.*\(json\|html\|txt\)" )
+ ( cd "$xmldumpsdir"; /usr/bin/find "public/" -maxdepth 3 -regextype
sed -regex ".*/${latestrun}/.*\.\(json\|html\|txt\)" | /usr/bin/xargs -s
1048576 /bin/tar rfp "$tarballpath" )
# if no files found, there will be no tarball created either
if [ -f "$tarballpath" ]; then
--
To view, visit https://gerrit.wikimedia.org/r/393374
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I626f875e3056f44564f190b2124f607072831262
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: ArielGlenn <ar...@wikimedia.org>
Gerrit-Reviewer: ArielGlenn <ar...@wikimedia.org>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
