[MediaWiki-commits] [Gerrit] Remove old getclaims script - change (analytics/limn-wikidata-data)

[Addshore (Code Review)]

Addshore has submitted this change and it was merged.

Change subject: Remove old getclaims script
..


Remove old getclaims script

Change-Id: Ibf59226835ebf1fd8a76de4c302ab722b0f2dd47
---
M cron
D graphite/api/getclaims_property_use.sh
2 files changed, 1 insertion(+), 44 deletions(-)

Approvals:
  Addshore: Verified; Looks good to me, approved



diff --git a/cron b/cron
index b39b3a6..1c75d59 100644
--- a/cron
+++ b/cron
@@ -13,4 +13,4 @@
 
 # Logrotate is at 6:25, + time for rsync (hourly?), 12 gives us roughly 6 hours
 # This MUST be run on stat1002
-0 12 * * * ~/wikidata-data/graphite/api/getclaims_property_use.sh
+0 12 * * * php ~/wikidata-data/graphite/api/logScanner.php
diff --git a/graphite/api/getclaims_property_use.sh 
b/graphite/api/getclaims_property_use.sh
deleted file mode 100755
index d32572c..000
--- a/graphite/api/getclaims_property_use.sh
+++ /dev/null
@@ -1,43 +0,0 @@
-#!/bin/bash
-#
-# @author Addshore
-#
-# This shows the number of calls to the wbgetclaims api module broken down by 
property for a 24 hour period.
-# This metric is taken from the api.log file archives generated on fluorine 
that are rsynced to stat1002.
-# The 24 hour period is from 00:00 to 23:59 UTC.
-
-# Run for date given, or yesterday
-if [ $# -eq 0 ]; then
-dateISO=`date --date=yesterday --iso-8601=date`
-else
-dateISO=$1
-fi
-nextDateISO=`date --date="$dateISO + 1 day" --iso-8601=date`
-
-# Get a date stamp to be used later
-dateStamp=`echo $dateISO | tr -d '-'`
-nextDateStamp=`echo $nextDateISO | tr -d '-'`
-
-# Get the location of the last api log
-apilog="/a/mw-log/archive/api/api.log-$dateStamp.gz"
-nextapilog="/a/mw-log/archive/api/api.log-$nextDateStamp.gz"
-
-# Make sure the files we want exist
-if [ ! -f $apilog ]; then
-echo "File not found: (apilog) $apilog"
-exit;
-fi
-if [ ! -f $nextApilog ]; then
-echo "File not found: (nextapilog) $nextapilog"
-exit;
-fi
-
-# Run the main command
-output=`zgrep $dateISO $apilog $nextapilog | grep action=wbgetclaims | grep 
wikidatawiki | egrep -o 'property=P[0-9]+' | sort | uniq -c | sort -nr`
-
-# Iterate over each line and add to graphite
-while read -r line; do
-property=`cut -d "=" -f 2 <<< "$line"`
-value=`cut -d " " -f 1 <<< "$line"`
-echo "daily.wikidata.api.wbgetclaims.properties.$property 
$value `date -d \"$dateStamp\" +%s`" | nc -q0 graphite.eqiad.wmnet 2003
-done <<< "$output"

-- 
To view, visit https://gerrit.wikimedia.org/r/255465
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ibf59226835ebf1fd8a76de4c302ab722b0f2dd47
Gerrit-PatchSet: 1
Gerrit-Project: analytics/limn-wikidata-data
Gerrit-Branch: master
Gerrit-Owner: Addshore 
Gerrit-Reviewer: Addshore 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Remove old getclaims script - change (analytics/limn-wikidata-data)

[Addshore (Code Review)]

Addshore has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/255465

Change subject: Remove old getclaims script
..

Remove old getclaims script

Change-Id: Ibf59226835ebf1fd8a76de4c302ab722b0f2dd47
---
M cron
D graphite/api/getclaims_property_use.sh
2 files changed, 1 insertion(+), 44 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/analytics/limn-wikidata-data 
refs/changes/65/255465/1

diff --git a/cron b/cron
index b39b3a6..1c75d59 100644
--- a/cron
+++ b/cron
@@ -13,4 +13,4 @@
 
 # Logrotate is at 6:25, + time for rsync (hourly?), 12 gives us roughly 6 hours
 # This MUST be run on stat1002
-0 12 * * * ~/wikidata-data/graphite/api/getclaims_property_use.sh
+0 12 * * * php ~/wikidata-data/graphite/api/logScanner.php
diff --git a/graphite/api/getclaims_property_use.sh 
b/graphite/api/getclaims_property_use.sh
deleted file mode 100755
index d32572c..000
--- a/graphite/api/getclaims_property_use.sh
+++ /dev/null
@@ -1,43 +0,0 @@
-#!/bin/bash
-#
-# @author Addshore
-#
-# This shows the number of calls to the wbgetclaims api module broken down by 
property for a 24 hour period.
-# This metric is taken from the api.log file archives generated on fluorine 
that are rsynced to stat1002.
-# The 24 hour period is from 00:00 to 23:59 UTC.
-
-# Run for date given, or yesterday
-if [ $# -eq 0 ]; then
-dateISO=`date --date=yesterday --iso-8601=date`
-else
-dateISO=$1
-fi
-nextDateISO=`date --date="$dateISO + 1 day" --iso-8601=date`
-
-# Get a date stamp to be used later
-dateStamp=`echo $dateISO | tr -d '-'`
-nextDateStamp=`echo $nextDateISO | tr -d '-'`
-
-# Get the location of the last api log
-apilog="/a/mw-log/archive/api/api.log-$dateStamp.gz"
-nextapilog="/a/mw-log/archive/api/api.log-$nextDateStamp.gz"
-
-# Make sure the files we want exist
-if [ ! -f $apilog ]; then
-echo "File not found: (apilog) $apilog"
-exit;
-fi
-if [ ! -f $nextApilog ]; then
-echo "File not found: (nextapilog) $nextapilog"
-exit;
-fi
-
-# Run the main command
-output=`zgrep $dateISO $apilog $nextapilog | grep action=wbgetclaims | grep 
wikidatawiki | egrep -o 'property=P[0-9]+' | sort | uniq -c | sort -nr`
-
-# Iterate over each line and add to graphite
-while read -r line; do
-property=`cut -d "=" -f 2 <<< "$line"`
-value=`cut -d " " -f 1 <<< "$line"`
-echo "daily.wikidata.api.wbgetclaims.properties.$property 
$value `date -d \"$dateStamp\" +%s`" | nc -q0 graphite.eqiad.wmnet 2003
-done <<< "$output"

-- 
To view, visit https://gerrit.wikimedia.org/r/255465
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibf59226835ebf1fd8a76de4c302ab722b0f2dd47
Gerrit-PatchSet: 1
Gerrit-Project: analytics/limn-wikidata-data
Gerrit-Branch: master
Gerrit-Owner: Addshore 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits