bking created this task.
bking added projects: Wikidata-Query-Service, Data-Platform-SRE.
Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION
We had a brief outage of the RDF streaming updater today. Users could have
been impacted from about ~2022 - 2050 UTC. Specifically
bking added a comment.
We could also deploy via a new namespace, but I wonder what implications that
would have for our monitoring/tooling etc. Open to feedback/suggestions on this
one.
TASK DETAIL
https://phabricator.wikimedia.org/T349095
EMAIL PREFERENCES
https
bking added a subtask: T349095: Migrate staging rdf-streaming-updater to flink
operator.
TASK DETAIL
https://phabricator.wikimedia.org/T326409
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Gehel, BTullis, JMeybohm, gmodena, Ottomata
bking added a parent task: T326409: Migrate the wdqs streaming updater flink
jobs to flink-k8s-operator deployment model.
TASK DETAIL
https://phabricator.wikimedia.org/T349095
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Aklapper, bking
bking created this task.
bking added projects: Wikidata-Query-Service, Data-Platform-SRE.
Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION
We have tested the flink operator mode in dse-k8s
<https://phabricator.wikimedia.org/T342149> . Our next step is to m
bking updated the task description.
TASK DETAIL
https://phabricator.wikimedia.org/T326409
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Gehel, BTullis, JMeybohm, gmodena, Ottomata, bking, Aklapper, dcausse,
Danny_Benjafield_WMDE
bking added a comment.
Apologies for not catching this earlier:
- to be aligned with the dumps that are imported into hdfs we must select a
particular set of files, when selecting a "-all" file the preceding "lexemes"
one must be taken, e.g. if wikidata-20230
bking added a subtask: T349011: Improve data-reload cookbook based on graph
split needs.
TASK DETAIL
https://phabricator.wikimedia.org/T347504
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: RKemper, dcausse, Aklapper, bking
bking reopened subtask T342149: Test common operations in the flink
operator/k8s/Flink ZK environment as In Progress.
TASK DETAIL
https://phabricator.wikimedia.org/T326409
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Gehel, BTullis
bking added a comment.
I started a data reload for hosts`wdqs1022-1024`. These are running in a tmux
window under my user on`cumin1001`.
Based on T323096 <https://phabricator.wikimedia.org/T323096> , we expect this
process to fail multiple times, which is why we're running in on
bking closed subtask T342149: Test common operations in the flink
operator/k8s/Flink ZK environment as Resolved.
TASK DETAIL
https://phabricator.wikimedia.org/T326409
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Gehel, BTullis, JMeybohm
bking changed the status of subtask T342149: Test common operations in the
flink operator/k8s/Flink ZK environment from Open to In
Progress.
TASK DETAIL
https://phabricator.wikimedia.org/T326409
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
bking updated the task description.
TASK DETAIL
https://phabricator.wikimedia.org/T347505
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: dcausse, Aklapper, bking, Danny_Benjafield_WMDE, Isabelladantes1983,
Themindcoder, Adamm71, Jersione
bking claimed this task.
bking moved this task from Incoming to In Progress on the Data-Platform-SRE
board.
TASK DETAIL
https://phabricator.wikimedia.org/T347504
WORKBOARD
https://phabricator.wikimedia.org/project/board/6524/
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings
bking updated the task description.
TASK DETAIL
https://phabricator.wikimedia.org/T347504
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: RKemper, dcausse, Aklapper, bking, Danny_Benjafield_WMDE, Astuthiodit_1,
AWesterinen, BTullis
bking added a comment.
More details related to dump loading in T325114
<https://phabricator.wikimedia.org/T325114> .
TASK DETAIL
https://phabricator.wikimedia.org/T347504
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: RKemper, d
bking updated the task description.
TASK DETAIL
https://phabricator.wikimedia.org/T347505
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: dcausse, Aklapper, bking, Danny_Benjafield_WMDE, Astuthiodit_1,
AWesterinen, BTullis, karapayneWMDE
bking added a comment.
Checksum does not match the version from `wdqs1016`, which is:
sha1sum wikidata.jnl.zst
e3197eb5177dcd1aa0956824cd8dc4afc2d8796c wikidata.jnl.zst
I also downloaded the file locally after putting it up in Cloudflare, which
has a different checksum
bking added a parent task: T340548: [EPIC] Deployment of the Search Update
Pipeline on Flink / k8s.
TASK DETAIL
https://phabricator.wikimedia.org/T326914
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: dcausse, bking
Cc: dr0ptp4kt, bking, dcausse
bking added a parent task: T342149: Test common operations in the flink
operator/k8s/Flink ZK environment.
TASK DETAIL
https://phabricator.wikimedia.org/T326914
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: dcausse, bking
Cc: dr0ptp4kt, bking
bking added a comment.
A few notes on this process:
- I used zstd compression <https://facebook.github.io/zstd/> to compress
the JNL file, as it supposedly offers the best speed. I used `zstd -T0 -19
wikidata.jnl` as my compression command (all cores, maximum compression),
bking added a comment.
@dr0ptp4kt and I were looking at this today and it occurred to me that the
JNL file is uncompressed.
Thus, I gzipped the main wikidata JNL file from `wdqs1016`, which takes ~4
hours using pigz at maximum compression rate, and we end up with a ~400 GB file
bking updated the task description.
TASK DETAIL
https://phabricator.wikimedia.org/T347605
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Addshore, dr0ptp4kt, Aklapper, bking, Danny_Benjafield_WMDE, Astuthiodit_1,
AWesterinen, BTullis
bking added a project: Data-Platform-SRE.
TASK DETAIL
https://phabricator.wikimedia.org/T344905
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: bking, Krinkle, dr0ptp4kt, Abbe98, Gehel, Addshore, Aklapper,
Danny_Benjafield_WMDE, Mohamed
bking updated the task description.
TASK DETAIL
https://phabricator.wikimedia.org/T347605
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Addshore, dr0ptp4kt, Aklapper, bking, AWesterinen, BTullis, Namenlos314,
Gq86, Lucas_Werkmeister_WMDE
bking updated the task description.
TASK DETAIL
https://phabricator.wikimedia.org/T347605
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Aklapper, bking, AWesterinen, BTullis, Namenlos314, Gq86,
Lucas_Werkmeister_WMDE, EBjune, merbst
bking created this task.
bking added projects: Wikidata-Query-Service, Data-Platform-SRE.
Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION
Blazegraph (the application that serves WDQS) stores all its data in a single
JNL file. The WDQS file is very large (~1.2TB) so moving
bking added a subscriber: dcausse.
bking added a comment.
@dcausse couple of questions:
- Are we OK to start the data load as soon as these hosts are in production?
- Does each host need its data loaded, or can we load on one and
data-transfer to the others?
TASK DETAIL
https
bking updated the task description.
TASK DETAIL
https://phabricator.wikimedia.org/T347505
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: dcausse, Aklapper, bking, Danny_Benjafield_WMDE, Astuthiodit_1,
AWesterinen, BTullis, karapayneWMDE
bking updated the task description.
TASK DETAIL
https://phabricator.wikimedia.org/T347504
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Aklapper, bking, Danny_Benjafield_WMDE, Astuthiodit_1, AWesterinen,
BTullis, karapayneWMDE, Invadibot
bking added a subtask: T347505: Prepare new WDQS hosts for graph splitting.
TASK DETAIL
https://phabricator.wikimedia.org/T337013
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: dr0ptp4kt, RKemper, bking, tfmorris, elal, karapayneWMDE
bking added a parent task: T337013: [Epic] Splitting the graph in WDQS.
TASK DETAIL
https://phabricator.wikimedia.org/T347505
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Aklapper, bking, AWesterinen, BTullis, Namenlos314, Gq86
bking added a parent task: T347505: Prepare new WDQS hosts for graph splitting.
TASK DETAIL
https://phabricator.wikimedia.org/T347504
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Aklapper, bking, Danny_Benjafield_WMDE, Astuthiodit_1
bking added a subtask: T347504: WDQS graph split: load data from dumps into new
hosts.
TASK DETAIL
https://phabricator.wikimedia.org/T347505
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Aklapper, bking, AWesterinen, BTullis, Namenlos314
bking removed a subtask: T347505: Prepare new WDQS hosts for graph splitting.
TASK DETAIL
https://phabricator.wikimedia.org/T347504
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Aklapper, bking, Danny_Benjafield_WMDE, Astuthiodit_1
bking removed a parent task: T347504: WDQS graph split: load data from dumps
into new hosts.
TASK DETAIL
https://phabricator.wikimedia.org/T347505
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Aklapper, bking, AWesterinen, BTullis
bking removed a subtask: T345475: Service implementation for
wdqs202[3-5].codfw.wmnet.
TASK DETAIL
https://phabricator.wikimedia.org/T337013
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: dr0ptp4kt, RKemper, bking, tfmorris, elal
bking added a subtask: T347505: Prepare new WDQS hosts for graph splitting.
TASK DETAIL
https://phabricator.wikimedia.org/T347504
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Aklapper, bking, Danny_Benjafield_WMDE, Astuthiodit_1
bking added a parent task: T347504: WDQS graph split: load data from dumps into
new hosts.
TASK DETAIL
https://phabricator.wikimedia.org/T347505
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Aklapper, bking, AWesterinen, BTullis
bking created this task.
bking added projects: Data-Platform-SRE, Wikidata-Query-Service.
Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION
wdqs1017-1024 are newly-deployed WDQS hosts.
Let's claim 2022-2024 for the graph splitting. These will be a new tier of
WDQS
bking added a parent task: T337013: [Epic] Splitting the graph in WDQS.
TASK DETAIL
https://phabricator.wikimedia.org/T347504
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Aklapper, bking, AWesterinen, BTullis, Namenlos314, Gq86
bking added a subtask: T347504: WDQS graph split: load data from dumps into new
hosts.
TASK DETAIL
https://phabricator.wikimedia.org/T337013
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: dr0ptp4kt, RKemper, bking, tfmorris, elal
bking created this task.
bking added projects: Wikidata-Query-Service, Data-Platform-SRE.
Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION
In order to proceed on T337013 <https://phabricator.wikimedia.org/T337013> ,
we need to do a full data reload on three hosts, s
bking added a subtask: T347355: Create alerts for
https://query.wikidata.org/bigdata/ldf.
TASK DETAIL
https://phabricator.wikimedia.org/T347284
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: RKemper, bking, MisterSynergy, dcausse
bking added a parent task: T347284: Restore service for
https://query.wikidata.org/bigdata/ldf.
TASK DETAIL
https://phabricator.wikimedia.org/T347355
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Aklapper, bking, AWesterinen, BTullis
bking created this task.
bking added projects: Data-Platform-SRE, Wikidata-Query-Service.
Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION
Per T347284 <https://phabricator.wikimedia.org/T347284> , we lost the LDF
endpoint for a few days. Creating this ticket to add
bking closed this task as "Resolved".
bking moved this task from Incoming to Done on the Data-Platform-SRE board.
bking claimed this task.
bking added a comment.
@MisterSynergy After applying the last patch,
https://query.wikidata.org/bigdata/ldf seems to be back up. Thus, I'm
bking added a project: Data-Platform-SRE.
TASK DETAIL
https://phabricator.wikimedia.org/T347284
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: RKemper, bking, MisterSynergy, dcausse, Aklapper, Danny_Benjafield_WMDE,
Astuthiodit_1
bking renamed this task from "https://query.wikidata.org/bigdata/ldf is broken"
to "Restore service for https://query.wikidata.org/bigdata/ldf;.
TASK DETAIL
https://phabricator.wikimedia.org/T347284
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/e
bking updated the task description.
TASK DETAIL
https://phabricator.wikimedia.org/T326409
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Gehel, BTullis, JMeybohm, gmodena, Ottomata, bking, Aklapper, dcausse,
Danny_Benjafield_WMDE
bking updated the task description.
TASK DETAIL
https://phabricator.wikimedia.org/T326409
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Gehel, BTullis, JMeybohm, gmodena, Ottomata, bking, Aklapper, dcausse,
Danny_Benjafield_WMDE
bking updated the task description.
TASK DETAIL
https://phabricator.wikimedia.org/T326409
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Gehel, BTullis, JMeybohm, gmodena, Ottomata, bking, Aklapper, dcausse,
Danny_Benjafield_WMDE
bking added a subtask: T342149: Test common operations in the flink
operator/k8s/Flink ZK environment.
TASK DETAIL
https://phabricator.wikimedia.org/T326409
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Gehel, BTullis, JMeybohm, gmodena
bking closed subtask T341792: Provision Zookeeper Cluster for storing Flink HA
data as Resolved.
TASK DETAIL
https://phabricator.wikimedia.org/T326409
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Gehel, BTullis, JMeybohm, gmodena
bking added a parent task: T342149: Test common operations in the flink
operator/k8s/Flink ZK environment.
TASK DETAIL
https://phabricator.wikimedia.org/T346456
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Aklapper, bking
bking added a subscriber: RKemper.
bking added a comment.
When T345475 <https://phabricator.wikimedia.org/T345475> is done, we should
have 3 new WDQS hosts in CODFW that could be used for the graph splitting
experiment. @RKemper let us know if you have any objections to this plan.
bking added a subtask: T345475: Service implementation for
wdqs202[3-5].codfw.wmnet.
TASK DETAIL
https://phabricator.wikimedia.org/T337013
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: RKemper, bking, tfmorris, elal, karapayneWMDE
bking moved this task from Ready for Work to Done on the Data-Platform-SRE
board.
bking added a comment.
We successfully deployed this yesterday; moving to "Done" on the workboard.
TASK DETAIL
https://phabricator.wikimedia.org/T344284
WORKBOARD
https://phabricator.wikimedia.o
bking added a project: Data-Platform-SRE.
TASK DETAIL
https://phabricator.wikimedia.org/T326914
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: bking, dcausse, Aklapper, Danny_Benjafield_WMDE, Astuthiodit_1,
AWesterinen, BTullis
bking moved this task from Ready for Work to Done on the Data-Platform-SRE
board.
bking added a comment.
Hello,
I've fixed wdqs1003 as well and it appears we have the same amount of triples
now
<https://grafana.wikimedia.org/d/00489/wikidata-query-service?orgId=1=1m=now-24h=now_n
bking updated the task description.
TASK DETAIL
https://phabricator.wikimedia.org/T326409
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Gehel, BTullis, JMeybohm, gmodena, Ottomata, bking, Aklapper, dcausse,
Danny_Benjafield_WMDE
bking added a comment.
Thanks for the quick response. It could very well be our fault, as we just
changed how we deploy our federation allowlist
<https://phabricator.wikimedia.org/T343856> . Thus, we'll continue to
troubleshoot from our side as well.
TASK DETAIL
bking claimed this task.
TASK DETAIL
https://phabricator.wikimedia.org/T326409
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Gehel, BTullis, JMeybohm, gmodena, Ottomata, bking, Aklapper, dcausse,
Danny_Benjafield_WMDE, fbalicchia
bking added a comment.
@Epidosis sorry for the delay on this ticket. We've added your endpoint, can
you please test it and let us know if it works? Thanks for your patience.
TASK DETAIL
https://phabricator.wikimedia.org/T337296
EMAIL PREFERENCES
https://phabricator.wikimedia.org
bking claimed this task.
bking updated Other Assignee, added: RKemper.
TASK DETAIL
https://phabricator.wikimedia.org/T337296
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Aklapper, EBernhardson, dcausse, Epidosis, Danny_Benjafield_WMDE
bking added a comment.
I've repooled wcqs1001 and wcqs1002 after verifying they have the correct
amount of triples. We've left wcqs1003 depooled as we troubleshoot further. Let
us know if you notice any other issues.
TASK DETAIL
https://phabricator.wikimedia.org/T344882
EMAIL PREFERENCES
bking added a comment.
Thanks for bringing this to our attention, and sorry for the inconvenience. I
have depooled the eqiad datacenter while we work to address the issue. Please
test our your queries and let us know if you're still getting inconsistent
results.
TASK DETAIL
https
bking updated the task description.
TASK DETAIL
https://phabricator.wikimedia.org/T343856
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: RKemper, bking
Cc: Gehel, Reedy, bking, Aklapper, Danny_Benjafield_WMDE, Isabelladantes1983,
Themindcoder
bking added a subtask: T344614: Add Zookeeper config to 'flink-app' test
service.
TASK DETAIL
https://phabricator.wikimedia.org/T326409
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Gehel, BTullis, JMeybohm, gmodena, Ottomata, bking
bking updated the task description.
TASK DETAIL
https://phabricator.wikimedia.org/T343856
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Gehel, Reedy, bking, Aklapper, Danny_Benjafield_WMDE, Isabelladantes1983,
Themindcoder, Adamm71
bking closed subtask T337801: WDQS: Document procedure for switching between
Kubernetes and Yarn Streaming Updater as Resolved.
TASK DETAIL
https://phabricator.wikimedia.org/T336134
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc
bking added a subtask: T337801: WDQS: Document procedure for switching between
Kubernetes and Yarn Streaming Updater.
TASK DETAIL
https://phabricator.wikimedia.org/T336134
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: karapayneWMDE
bking added a comment.
@WolfgangFahl We've whitelisted the endpoints, but the query you linked above
<https://w.wiki/6q2i> still does not work. Can you verify that is it working as
expected? My teammate mentioned "it's returning application/sparql-results+xml
but we on
bking closed subtask T330714: Document SRE steps for deploying a new WDQS (and
WCQS) host as Resolved.
TASK DETAIL
https://phabricator.wikimedia.org/T332314
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: RKemper, Gehel, Aklapper
bking added a comment.
Update: `wdqs2016.codfw.wmnet` is the last host that needs to be configured
for production.
`wdqs2020.codfw.wmnet` has been receiving production traffic for a week now,
with no observed issues.
We should be able to finish the rest pretty soon and start
bking added a comment.
Update: I forgot to target 2013 in my last command, here is the latest list
of hosts that need a data transfer and a deploy:
(4) wdqs[2013-2016].codfw.wmnet
- OUTPUT of 'du -hcxs /srv/de...6756ebe194261756' -
132M
/srv/deployment/wdqs/wdqs
bking set the point value for this task to "1".
TASK DETAIL
https://phabricator.wikimedia.org/T339347
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: bking, Aklapper, WolfgangFahl, Astuthiodit_1, AWesterinen, BTullis,
kar
bking added a comment.
Update: wdqs[2017-2021].codfw.wmnet are now production ready:
= NODE GROUP =
(4) wdqs[2014-2016,2022].codfw.wmnet
- OUTPUT of 'du -hcxs /srv/de...6756ebe194261756' -
132M
/srv/deployment/wdqs/wdqs-cache/revs
bking added a comment.
Current state: 2019 and 2020 are production-ready. The others need a data
transfer and/or scap deploy to be complete. The command below checks the
deployment directory size. If the directory size is smaller than 471M, that
means `git-fat` isn't working and the host
bking added a subtask: T330714: Document SRE steps for deploying a new WDQS
(and WCQS) host.
TASK DETAIL
https://phabricator.wikimedia.org/T332314
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: RKemper, Gehel, Aklapper, Astuthiodit_1
bking reopened this task as "In Progress".
TASK DETAIL
https://phabricator.wikimedia.org/T321605
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Vgutierrez, RKemper, Volans, Aklapper, bking, Astuthiodit_1, AWesterinen,
kar
bking claimed this task.
TASK DETAIL
https://phabricator.wikimedia.org/T332314
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Gehel, Aklapper, Astuthiodit_1, AWesterinen, BTullis, karapayneWMDE,
Invadibot, MPhamWMF, maantietaja
bking added a project: Discovery-Search (Current work).
TASK DETAIL
https://phabricator.wikimedia.org/T321605
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Vgutierrez, RKemper, Volans, Aklapper, bking, Isabelladantes1983,
Themindcoder
bking moved this task from monitoring to in progress on the Wikidata board.
bking closed this task as "Resolved".
bking claimed this task.
bking added a comment.
I believe this is complete; moving to 'needs review' status.
TASK DETAIL
https://phabricator.wikimedia.org/T321605
bking added a comment.
@Epidosis sorry for the long turnaround on this. Can you try with 'https://'
as opposed to 'http://' ? Here <https://w.wiki/6puk> is the result we get when
using 'https://'.
TASK DETAIL
https://phabricator.wikimedia.org/T336709
EMAIL PREFERENCES
bking added a comment.
I've been working on this a bit more lately. The Transfer.py documentation
<https://doc.wikimedia.org/transferpy/master/transferpy/transferpy.html#module-transferpy.Firewall>
mentions "remote_execution" but does not mention that it's a required argumen
bking added a comment.
Per today's SRE meeting, the larger SRE org is working on a comprehensive
alert review <https://etherpad.wikimedia.org/p/alert-review-may-2023> . We
should work with the SREs to help out and use their methods to review our own
alerts.
TASK DETAIL
bking added projects: Sustainability, SRE-OnFire.
TASK DETAIL
https://phabricator.wikimedia.org/T336577
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: bking, dcausse, Gehel, Aklapper, Astuthiodit_1, AWesterinen, karapayneWMDE,
Invadibot
bking edited projects, added Sustainability, SRE-OnFire; removed Sustainability
(Incident Followup).
TASK DETAIL
https://phabricator.wikimedia.org/T336574
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: bking, Aklapper, Gehel
bking added a comment.
@DL2204 Thanks for your patience. We've added
https://lila-erc.eu/sparql/lila_knowledge_base/sparql as an allowed SPARQL
endpoint for WDQS.
Please test this out and respond here with your results-good or bad.
TASK DETAIL
https://phabricator.wikimedia.org
bking added a comment.
@Nikki Thanks for your patience. We've added
https://vocabularies.unesco.org/sparql as an allowed SPARQL endpoint for WDQS.
Please test this out and respond here with your results-good or bad.
TASK DETAIL
https://phabricator.wikimedia.org/T335994
EMAIL
bking added a comment.
@Epidosis Thanks for your patience. We've added
https://digitale.bncf.firenze.sbn.it/openrdf-workbench/repositories/NS/query as
an allowed SPARQL endpoint for WDQS. Please test this out and respond here with
your results-good or bad.
TASK DETAIL
https
bking added a comment.
Other action items:
- Add link to new WDQS superset dashboard to WDQS runbook page.
- Fix dead logstash link on WDQS runbook page
<https://wikitech.wikimedia.org/wiki/Wikidata_Query_Service/Runbook#Timeouts>
- Better documentation of throttling be
bking claimed this task.
TASK DETAIL
https://phabricator.wikimedia.org/T336577
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: bking, dcausse, Gehel, Aklapper, Astuthiodit_1, AWesterinen, karapayneWMDE,
Invadibot, Zabe, MPhamWMF
bking added subscribers: dcausse, bking.
bking added a comment.
Updated the Streaming Updater operations docs
<https://wikitech.wikimedia.org/wiki/Wikidata_Query_Service/Streaming_Updater#The_consumers_are_backlogged>
after today's pairing session with @dcausse . We'll continue to
bking added a comment.
Revised totals for alerts in the last year after looking at Logstash:
`RdfStreamingUpdaterHighConsumerUpdateLag` 373
`RdfStreamingUpdaterFlinkProcessingLatencyIsHigh` 63
`RdfStreamingUpdaterFlinkJobUnstable` 125
The majority of all three alert types fired
bking added a comment.
Quick notes here before I forget. Checking my "alerts" email folder for the
past year (not the most reliable source), I have:
- 89 alerts with title RdfStreamingUpdaterHighConsumerUpdateLag
- 72 alerts with title RdfStreamingUpdaterFlinkProcessingLat
bking removed a project: Discovery-Search (Current work).
TASK DETAIL
https://phabricator.wikimedia.org/T325602
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: Gehel, Aklapper, RKemper, bking, Astuthiodit_1, AWesterinen, karapayneWMDE
bking removed a project: Discovery-Search (Current work).
TASK DETAIL
https://phabricator.wikimedia.org/T193473
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: bking, Aklapper, Smalyshev, Gehel, Astuthiodit_1, AWesterinen,
karapayneWMDE
bking edited projects, added Discovery-Search; removed Discovery-Search
(Current work).
TASK DETAIL
https://phabricator.wikimedia.org/T274270
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: bking
Cc: bking, RKemper, Gehel, Aklapper, Astuthiodit_1
101 - 200 of 312 matches
Mail list logo
