Ladsgroup added a comment.
Given the recent issues with WDQS I would like this to have higher priority.
Currently the Wikidata's top requests in any time I checked is from WDQS:
0: jdbc:hive2://an-coord1001.eqiad.wmnet:1000> select user_agent, count(*)
as hitcount from wmf.webrequest where uri_host = 'www.wikidata.org' and year =
2019 and month = '08' and day = 6 and hour = 05 and (dt like
'2019-08-06T05:38%') group by user_agent order by hitcount desc limit 50;
<...>
user_agent hitcount
Wikidata Query Service Updater 14892
Object Revision Evaluation Service <[email protected]> 3480
The ratio of the top external request is one order of magnitude smaller then
the WDQS (ORES is also internal, but that's another issue).
I highly disagree with timing isolated requests and benchmarking just one
reuqest. Wikidata caches lots of bits of every request, making the second
request to the same thing is usually faster. Not to mention database caches.
The other thing I want to mention and was missing here is overhead of
encryption and TLS handshakes. In the @BBlack's example, we still use TLS but
if you use plain http request, it's considerably faster (in both overhead of
encryption and decryption):
ladsgroup@mwmaint1002:~$ time curl -H 'Host: www.wikidata.org'
'http://appservers-ro.discovery.wmnet/wiki/Special:EntityData/Q7251.ttl?revision=992109551&flavor=dump'
> /dev/null
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
100 123k 0 123k 0 0 508k 0 --:--:-- --:--:-- --:--:--
510k
real 0m0.256s
user 0m0.008s
sys 0m0.004s
Unless there's any reason to encrypt requests internally, I think this would
help us greatly.
TASK DETAIL
https://phabricator.wikimedia.org/T199219
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Ladsgroup
Cc: Ladsgroup, akosiaris, BBlack, Aklapper, Smalyshev, Gehel, darthmon_wmde,
DannyS712, Nandana, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic,
QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Jonas, Xmlizer,
jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
_______________________________________________
Wikidata-bugs mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
