Re: [Wikitech-l] How to get number of articles or related metrics for indic wiki's ?

[Shrinivasan T]

Thanks Bryan,

Exploring the links you gave.

In the meantime, rolled out my own server to setup Prometheus and grafana.

Wrote a custom exporter to get counts for indic Wikipedia sites.

http://139.59.47.5:3000/d/kx1Pb36Zz/indic-wiki-stats

Check the above dashboard.

Will check if we can get the similar comparative graphs with wikistats2.

If this really goes well and community like to keep the dashboards for long
time, will check with cloud vps team to host it.

Thanks.
Shrini
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] How to get number of articles or related metrics for indic wiki's ?

[Bryan Davis]

On Fri, May 8, 2020 at 2:03 PM Shrinivasan T  wrote:
>
> Hello all,
>
>  I am planning to build a grafana dashboard using Prometheus for the counts
> of all indic wiki articles.

The dashboards you are thinking of may actually already exist.
Discovery of tools is a real problem in our community, and one I hope
to be able to work on more in the coming months.

Please take a look at

as an example of the data that the Wikimedia Foundation's Analytics
team publishes to help folks keep track of trends across the Wikimedia
movement's project wikis. More information on the "Wikistats 2"
project can be found at

including information on how you can contribute to this project.

> Have to get all the counts and write a custom exporter.
>
>
> Planning for a dashboard showing counts for articles in all indic languages.
>
> Another dashboard to show counts for all wiki projects for selected
> language.
>
> Have few queries.
>
> 1. How to get the number of articles in a wiki for example, tamil wikipedia
> ?  Any api is there to get numbers?

Basic information on article counts can be fetched from each wiki
using the Action API's action=query=siteinfo endpoint. See
 for more information
about this API.

See 

for an example usage on tawiki.

The Wikistats 2 project actually pulls its data from a public API as
well! The dashboard I linked above fetches data from
.
This is part of what is known as the "Wikimedia REST API". See
 for more
information on this API collection.


> 2. Can we run a sparql query from our own server?
>
> 3. Once these dashboards are built, can we host custom exporter, Prometheus
> and grafana in tool server or any wiki cloud server? Whom to contact for
> hosting these ?

Toolforge is probably not a great place to host a Prometheus server
simply because the local disk that you would have available to store
the data sets would be hosted on the shared NFS server which provides
$HOME directories for Toolforge maintainers and their tools.

A Cloud VPS project would be capable of hosting the general software
described. See 
for more information about what a Cloud VPS project is and how you
might apply to create one for your project.

Please be aware that a request to create the project described in this
email would likely receive a response encouraging you to collaborate
with the Wikistats 2 project to achieve your goals rather than making
a new project.

> Will do these in remote Hackathon this weekend.

I hope my answers here don't spoil your hackathon! Maybe try playing
around with Wikistats 2 and the APIs it uses and think of ways that
you could either add new features to Wikistats 2 or make a tool that
uses data from the same APIs that would be helpful to the Indic
language community.


Bryan
-- 
Bryan Davis  Technical Engagement  Wikimedia Foundation
Principal Software Engineer   Boise, ID USA
[[m:User:BDavis_(WMF)]]  irc: bd808

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] How to get number of articles or related metrics for indic wiki's ?

[Shrinivasan T]

Hello all,

 I am planning to build a grafana dashboard using Prometheus for the counts
of all indic wiki articles.

Have to get all the counts and write a custom exporter.


Planning for a dashboard showing counts for articles in all indic languages.

Another dashboard to show counts for all wiki projects for selected
language.

Have few queries.

1. How to get the number of articles in a wiki for example, tamil wikipedia
?  Any api is there to get numbers?

2. Can we run a sparql query from our own server?

3. Once these dashboards are built, can we host custom exporter, Prometheus
and grafana in tool server or any wiki cloud server? Whom to contact for
hosting these ?

Will do these in remote Hackathon this weekend.

Shrini
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l