Chelsyx has uploaded a new change for review. (
https://gerrit.wikimedia.org/r/374669 )
Change subject: Breakdown API calls by referer class
......................................................................
Breakdown API calls by referer class
Bug: T172452
Change-Id: Ic70d7054e02569eb8545dd347026c7f77321ab2c
---
M modules/api.R
A tab_documentation/referer_breakdown.md
M ui.R
M utils.R
4 files changed, 65 insertions(+), 2 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/wikimedia/discovery/rainbow
refs/changes/69/374669/1
diff --git a/modules/api.R b/modules/api.R
index dc8e332..bfc2350 100644
--- a/modules/api.R
+++ b/modules/api.R
@@ -53,3 +53,22 @@
dyEvent(as.Date("2017-01-01"), "R (reportupdater)", labelLoc = "bottom")
%>%
dyEvent(as.Date("2017-06-29"), "U (new UDF)", labelLoc = "bottom")
})
+
+output$referer_breakdown_plot <- renderDygraph({
+ temp <- split_dataset %>%
+ dplyr::bind_rows(.id = "api") %>%
+ dplyr::group_by(date, referrer) %>%
+ dplyr::summarize(calls = sum(calls, na.rm = TRUE)) %>%
+ tidyr::spread(referrer, calls)
+ if (input$referer_breakdown_prop) {
+ temp <- cbind(temp$date, purrr::map_df(temp[, -c(1, 2)], function(x)
round(100 * x / temp$All, 2)))
+ }
+ temp %>%
+ polloi::smoother(smooth_level =
polloi::smooth_switch(input$smoothing_global,
input$smoothing_referer_breakdown)) %>%
+ polloi::make_dygraph(xlab = "Date",
+ ylab = ifelse(input$referer_breakdown_prop, "API
Calls Share (%)", "API Calls"),
+ title = "Daily API usage by referrer", legend_name =
"API Calls") %>%
+ dyRangeSelector %>%
+ dyEvent(as.Date("2017-01-01"), "R (reportupdater)", labelLoc = "bottom")
%>%
+ dyEvent(as.Date("2017-06-29"), "U (new UDF)", labelLoc = "bottom")
+})
diff --git a/tab_documentation/referer_breakdown.md
b/tab_documentation/referer_breakdown.md
new file mode 100644
index 0000000..0b1f8d0
--- /dev/null
+++ b/tab_documentation/referer_breakdown.md
@@ -0,0 +1,24 @@
+API Calls by Referrer Class
+=======
+
+All types of API calls are aggregated by date and referrer class.
+
+**Internal** is traffic referred by Wikimedia sites, specifically:
mediawiki.org, wikibooks.org, wikidata.org, wikinews.org, wikimedia.org,
wikimediafoundation.org, wikipedia.org, wikiquote.org, wikisource.org,
wikiversity.org, wikivoyage.org, and wiktionary.org (See [Webrequest
source](https://git.wikimedia.org/blob/analytics%2Frefinery%2Fsource.git/master/refinery-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fwikimedia%2Fanalytics%2Frefinery%2Fcore%2FWebrequest.java#L203)
for more information.)
+
+Outages and inaccuracies
+------
+
+* '__R__': on 2017-01-01 we started calculating all of Discovery's metrics
using a new version of [our data retrieval and processing
codebase](https://phabricator.wikimedia.org/diffusion/WDGO/) that we migrated
to [Wikimedia Analytics](https://www.mediawiki.org/wiki/Analytics)'
[Reportupdater
infrastructure](https://wikitech.wikimedia.org/wiki/Analytics/Reportupdater).
See [T150915](https://phabricator.wikimedia.org/T150915) for more details.
Furthermore, we switched to an updated UDF for counting API calls -- the
previous version was undercounting full-text and geo search API calls (see
[Gerrit change 315503](https://gerrit.wikimedia.org/r/#/c/315503/) for more
details).
+* '__U__': on 2017-06-29 we started to use a new UDF to get the type of search
API (see [Gerrit change 345863](https://gerrit.wikimedia.org/r/#/c/345863/) for
more details) and break down the API calls by referer class.
+
+Questions, bug reports, and feature suggestions
+------
+For technical, non-bug questions, [email
Mikhail](mailto:[email protected]?subject=Dashboard%20Question) or
[Chelsy](mailto:[email protected]?subject=Dashboard%20Question). If you
experience a bug or notice something wrong or have a suggestion, [open a ticket
in
Phabricator](https://phabricator.wikimedia.org/maniphest/task/create/?projects=Discovery)
in the Discovery board or [email
Deb](mailto:[email protected]?subject=Dashboard%20Question).
+
+<hr style="border-color: gray;">
+<p style="font-size: small;">
+ <strong>Link to this dashboard:</strong> <a
href="https://discovery.wmflabs.org/metrics/#referer_breakdown";>https://discovery.wmflabs.org/metrics/#referer_breakdown</a>
+ | Page is available under <a
href="https://creativecommons.org/licenses/by-sa/3.0/"; title="Creative Commons
Attribution-ShareAlike License">CC-BY-SA 3.0</a>
+ | <a href="https://phabricator.wikimedia.org/diffusion/WDRN/"; title="Search
Metrics Dashboard source code repository">Code</a> is licensed under <a
href="https://phabricator.wikimedia.org/diffusion/WDRN/browse/master/LICENSE.md";
title="MIT License">MIT</a>
+ | Part of <a href="https://discovery.wmflabs.org/";>Discovery Dashboards</a>
+</p>
diff --git a/ui.R b/ui.R
index 8b20615..7131e7e 100644
--- a/ui.R
+++ b/ui.R
@@ -60,7 +60,8 @@
menuSubItem(text = "Open Search", tabName =
"open_search"),
menuSubItem(text = "Geo Search", tabName =
"geo_search"),
menuSubItem(text = "Prefix Search", tabName =
"prefix_search"),
- menuSubItem(text = "Language Search", tabName =
"language_search")),
+ menuSubItem(text = "Language Search", tabName =
"language_search"),
+ menuSubItem(text = "Referer Breakdown", tabName =
"referer_breakdown")),
menuItem(text = "Zero Results",
menuSubItem(text = "Summary", tabName =
"failure_rate"),
menuSubItem(text = "Search Type Breakdown", tabName
= "failure_breakdown"),
@@ -248,6 +249,18 @@
dygraphOutput("language_aggregate"),
includeMarkdown("./tab_documentation/language_basic.md")
),
+ tabItem(tabName = "referer_breakdown",
+ fluidRow(
+ column(
+ HTML("<label class = \"control-label\" style =
\"margin-bottom:-30px;\">Type</label>"),
+ shiny::checkboxInput("referer_breakdown_prop", label =
"Use Proportion", value = FALSE),
+ width = 2
+ ),
+ column(polloi::smooth_select("smoothing_referer_breakdown"),
width = 10)
+ ),
+ dygraphOutput("referer_breakdown_plot"),
+ includeMarkdown("./tab_documentation/referer_breakdown.md")
+ ),
tabItem(tabName = "failure_rate",
polloi::smooth_select("smoothing_failure_rate"),
polloi::automata_select(input_id = "failure_rate_automata"),
diff --git a/utils.R b/utils.R
index 70db0ed..3d9ad46 100644
--- a/utils.R
+++ b/utils.R
@@ -91,7 +91,14 @@
dplyr::filter(!is.na(api), !is.na(referer_class), !is.na(calls)) %>%
dplyr::distinct(date, api, referer_class, .keep_all = TRUE) %>%
dplyr::arrange(api, date) %>%
- dplyr::mutate(referer_class =
polloi::capitalize_first_letter(referer_class)) %>%
+ dplyr::mutate(referer_class = forcats::fct_recode(
+ referer_class,
+ `None (direct)` = "none",
+ `Search engine` = "external (search engine)",
+ `External (but not search engine)` = "external",
+ Internal = "internal",
+ Unknown = "unknown"
+ )) %>%
tidyr::spread(referer_class, calls) %>%
dplyr::mutate(All = ifelse(is.na(All), rowSums(.[, -c(1, 2)], na.rm =
TRUE), All)) %>%
tidyr::gather(referrer, calls, -c(date, api)) %>%
--
To view, visit https://gerrit.wikimedia.org/r/374669
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic70d7054e02569eb8545dd347026c7f77321ab2c
Gerrit-PatchSet: 1
Gerrit-Project: wikimedia/discovery/rainbow
Gerrit-Branch: develop
Gerrit-Owner: Chelsyx <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
