[Wikidata-bugs] [Maniphest] T362920: Benchmark Blazegraph import with increased buffer capacity (and other factors)

dr0ptp4kt added a comment. @RKemper I think that's captured in P54284 . If you need to get a copy of the files, there's a pointer in T350106#9381611 for how one might go about copying from HDFS

[Wikidata-bugs] [Maniphest] T362920: Benchmark Blazegraph import with increased buffer capacity (and other factors)

dr0ptp4kt added a comment. Another thing that can be nice for figuring out stuff later is to add some timing and a simple log file. A command like the following was helpful when I was trying this out on the gaming-class desktop (you may not need this if your tmux session lets you scroll

[Wikidata-bugs] [Maniphest] T349069: Design and implement a WDQS data-reload mechanism that sources its data from HDFS instead of the snapshot servers

bking added a comment. @dcausse It looks like there's plenty of disk space on `/srv`: bking@stat1007:~$ df -h | grep srv /dev/mapper/stat1007--vg-data 7.2T 4.4T 2.5T 65% /srv I don't own the stats servers though. @BTullis are there any other concerns besides disk space we

[Wikidata-bugs] [Maniphest] T316876: wdqs: replace git-fat with git-lfs

gerritbot added a comment. Change #1020911 **merged** by Bking: [wikidata/query/deploy@master] Convert git-fat to git-lfs https://gerrit.wikimedia.org/r/1020911 TASK DETAIL https://phabricator.wikimedia.org/T316876 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T362920: Benchmark Blazegraph import with increased buffer capacity (and other factors)

gerritbot added a comment. Change #1026668 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper): [operations/puppet@production] wdqs: enable nfs data reloads on wdqs1021 https://gerrit.wikimedia.org/r/1026668 TASK DETAIL https://phabricator.wikimedia.org/T362920

[Wikidata-bugs] [Maniphest] T362920: Benchmark Blazegraph import with increased buffer capacity (and other factors)

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T362920 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: gerritbot Cc: RKemper, bking, Aklapper, dr0ptp4kt, Danny_Benjafield_WMDE, Isabelladantes1983,

[Wikidata-bugs] [Maniphest] T316876: wdqs: replace git-fat with git-lfs

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T316876 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bking, Maintenance_bot Cc: BTullis, Gehel, dancy, RKemper, bking, demon, Aklapper,

[Wikidata-bugs] [Maniphest] T333656: Decommission query-preview.wikidata.org

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T333656 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, gerritbot Cc: Lydia_Pintscher, Dzahn, RKemper, Aklapper, Gehel, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T333656: Decommission query-preview.wikidata.org

gerritbot added a comment. Change #1026622 had a related patch set uploaded (by Dzahn; author: Dzahn): [operations/puppet@production] delete cert for query-preview.wikidata.org https://gerrit.wikimedia.org/r/1026622 TASK DETAIL https://phabricator.wikimedia.org/T333656 EMAIL

[Wikidata-bugs] [Maniphest] T333656: Decommission query-preview.wikidata.org

gerritbot added a comment. Change #1026623 **merged** by Dzahn: [labs/private@master] delete fake key for query-preview.wikidata.org https://gerrit.wikimedia.org/r/1026623 TASK DETAIL https://phabricator.wikimedia.org/T333656 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T362920: Benchmark Blazegraph import with increased buffer capacity (and other factors)

RKemper added a subscriber: dcausse. RKemper added a comment. I'm realizing I don't remember enough about how we load specific graph splits (scholarly vs main). But it's possible we won't need the above nfs patch if our previous process was to manually download the relevant dump file.

[Wikidata-bugs] [Maniphest] T364039: Lift IP cap on this dates 14/05; 28/05; 04/06; 11/06; 18/06; 25/06 for edit-a-thon for eswiki, commons and wikidata

Peachey88 edited projects, added Wikimedia-Site-requests; removed Wikidata, Spanish-Sites, Commons. TASK DETAIL https://phabricator.wikimedia.org/T364039 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Peachey88 Cc: Ucriesidelplata, Aklapper,

[Wikidata-bugs] [Maniphest] T333656: Decommission query-preview.wikidata.org

gerritbot added a comment. Change #1026623 had a related patch set uploaded (by Dzahn; author: Dzahn): [labs/private@master] delete fake key for query-preview.wikidata.org https://gerrit.wikimedia.org/r/1026623 TASK DETAIL https://phabricator.wikimedia.org/T333656 EMAIL

[Wikidata-bugs] [Maniphest] T349069: Design and implement a WDQS data-reload mechanism that sources its data from HDFS instead of the snapshot servers

dcausse added a comment. @BTullis @bking I plan to use a cookbook to transfer some data out of hdfs to blazegraph machines, a naive approach I thought about was to use a temp folder somewhere in `/srv` of a stat100x machine and then re-use the transferpy

[Wikidata-bugs] [Maniphest] T364039: Lift IP cap on this dates 14/05; 28/05; 04/06; 11/06; 18/06; 25/06 for edit-a-thon for eswiki, commons and wikidata

Ezarate created this task. Ezarate added projects: Commons, Spanish-Sites, Wikidata. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION An activity is scheduled the dates 2024-05-14, 2024-05-28, 2024-06-04, 2024-06-18 and 2024-06-25 by Facultad de Ciencias Naturales y Museo,

[Wikidata-bugs] [Maniphest] T362920: Benchmark Blazegraph import with increased buffer capacity (and other factors)

RKemper added a comment. Actually we'll use `wdqs1021` since we're not confident NFS will work seamlessly between eqiad and codfw (we use nsf to procure the dumps that the data-reload is run from). Gearing up to kick off a data reload on `wdqs1021` after a quick patch to enable NFS on

[Wikidata-bugs] [Maniphest] T362060: Generalize ScholarlyArticleSplitter

gerritbot added a comment. Change #1020871 **merged** by jenkins-bot: [wikidata/query/rdf@master] Split based on subgraph rules https://gerrit.wikimedia.org/r/1020871 TASK DETAIL https://phabricator.wikimedia.org/T362060 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T363538: Decide what to do about Manual of Style pseudo-namespaces conflicting with Mooré Wikipedia

Dcljr added a comment. In T363538#9748082 , @jhsoby wrote: >> - Create the wiki at mos.wikipedia.org and add hacks to the dumpInterwiki script in #mediawiki-extensions-wikimediamaintenance

[Wikidata-bugs] [Maniphest] T362060: Generalize ScholarlyArticleSplitter

gerritbot added a comment. Change #1024414 **merged** by jenkins-bot: [wikidata/query/rdf@master] ScholarlyArticleSplit: add support for stubs https://gerrit.wikimedia.org/r/1024414 TASK DETAIL https://phabricator.wikimedia.org/T362060 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T362060: Generalize ScholarlyArticleSplitter

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T362060 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, Maintenance_bot Cc: dr0ptp4kt, dcausse, Aklapper, Danny_Benjafield_WMDE, S8321414,

[Wikidata-bugs] [Maniphest] T364050: Wikidata Query Builder Ticket Template language issue

Maintenance_bot added a project: Wikidata. TASK DETAIL https://phabricator.wikimedia.org/T364050 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Maintenance_bot Cc: Aklapper, PhilbonQC, Danny_Benjafield_WMDE, S8321414, Astuthiodit_1, karapayneWMDE,

[Wikidata-bugs] [Maniphest] T363538: Decide what to do about Manual of Style pseudo-namespaces conflicting with Mooré Wikipedia

Pppery added a comment. "m:mos" would indeed. I have no idea how Wikidata interacts with this system, though. TASK DETAIL https://phabricator.wikimedia.org/T363538 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Pppery Cc: Dcljr,

[Wikidata-bugs] [Maniphest] T363538: Decide what to do about Manual of Style pseudo-namespaces conflicting with Mooré Wikipedia

Pppery added a comment. Although it looks like Meta is currently treated by the interwiki system as a Wikipedia even though it isn't one, so if the decision is to de-interwiki "mos:" on all Wikipedias (as opposed to de-interwikiing it only on en, zh, and dag wikipedias) it will need more