[Wikidata-bugs] [Maniphest] T291168: [Debian bullseye image] APT source-list file is not up to date

2022-01-04 Thread Andrew
Andrew closed this task as "Resolved". Andrew claimed this task. Andrew added subscribers: Majavah, Dzahn. Andrew added a comment. @Majavah reminded me about this issue and caught me up with the latest. A few points: - there are no Wikimedia mirrors for the Debian security rep

[Wikidata-bugs] [Maniphest] T291168: [Debian bullseye image] APT source-list file is not up to date

2022-01-04 Thread Andrew
Andrew added a parent task: T264311: Prepare for puppetizing /etc/apt/sources.list. TASK DETAIL https://phabricator.wikimedia.org/T291168 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: Tim-moody, bd808, Andrew, RhinosF1, fgiunchedi

[Wikidata-bugs] [Maniphest] T291168: [Debian bullseye image] APT source-list file is not up to date

2022-01-04 Thread Andrew
Andrew added a comment. I've just confirmed that this still happens TASK DETAIL https://phabricator.wikimedia.org/T291168 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: Tim-moody, bd808, Andrew, RhinosF1, fgiunchedi, Aklapper, Hjfocs

[Wikidata-bugs] [Maniphest] T283175: Should WMCS be getting CF protection?

2021-05-19 Thread Andrew
Andrew created this task. Andrew added projects: Security-Team, Security. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: cloud-services-team (Kanban). TASK DESCRIPTION Right now most WMCS things (specifically, public IPs associated with VMs and all

[Wikidata-bugs] [Maniphest] T265615: Terminating MySQL takes several minutes in (Wikibase?) CI jobs

2021-04-13 Thread Andrew
Andrew closed subtask T266777: integration instances suffer from high IO latency due to Ceph as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T265615 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: hashar, Andrew Cc: Aklapper, dduvall

[Wikidata-bugs] [Maniphest] T273636: Blazegraph journal for wcqs is too big

2021-02-12 Thread Andrew
Andrew added a comment. Just a note about these dedicated hypervisors: I've downtimed the disk space alerts on all cloudvirt-wdqs hosts. This is to avoid getting an alert in the scenario where the VM image has grown to 'fill' the physical raid even when the VM volume itself is mostly empty

[Wikidata-bugs] [Maniphest] T208416: Check whether wikidata-dev project requires NFS or not

2020-10-13 Thread Andrew
Andrew closed this task as "Resolved". Andrew added a comment. Thank you! You will probably need to reboot your VMs for this change to take effect; they may refuse to let you log in in the meantime. TASK DETAIL https://phabricator.wikimedia.org/T208416 EMAIL PREFERENC

[Wikidata-bugs] [Maniphest] T206636: Provide a way to have test servers on real hardware, isolated from production for Wikidata Query Service

2020-08-28 Thread Andrew
Andrew added a subscriber: EBernhardson. Andrew added a comment. I would like to delete the flavors named t206636 and t206636-testing and the VM named wcqs-beta-01 which uses the latter flavor. Is that ok? TASK DETAIL https://phabricator.wikimedia.org/T206636 EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] [Updated] T257336: Request increased quota for wikidata-query Cloud VPS project

2020-07-08 Thread Andrew
Andrew added a comment. This flavor was broken mostly because it asked for way too many cores (as well as slightly to much RAM). I've adjusted it as needed and things should be working better now. Note that due to a different snafu (recorded in T219078 <ht

[Wikidata-bugs] [Maniphest] [Commented On] T257336: Request increased quota for wikidata-query Cloud VPS project

2020-07-08 Thread Andrew
Andrew added a comment. I set aggregate_instance_extra_specs:wdqs='true' on that flavor; if you try recreating it should end up on the right hardware. TASK DETAIL https://phabricator.wikimedia.org/T257336 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Commented On] T208416: Check whether wikidata-dev project requires NFS or not

2020-07-07 Thread Andrew
Andrew added a comment. Sounds good! lmk when everyone is ready. TASK DETAIL https://phabricator.wikimedia.org/T208416 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: Andrew, Ladsgroup, Tarrow, ItamarWMDE, Bstorm

[Wikidata-bugs] [Maniphest] [Commented On] T208416: Check whether wikidata-dev project requires NFS or not

2020-05-20 Thread Andrew
Andrew added a comment. A few weeks is just fine. Thank you! TASK DETAIL https://phabricator.wikimedia.org/T208416 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore, Andrew Cc: Andrew, Ladsgroup, Tarrow, ItamarWMDE, Bstorm

[Wikidata-bugs] [Maniphest] [Commented On] T208416: Check whether wikidata-dev project requires NFS or not

2020-05-20 Thread Andrew
Andrew added a comment. If you're able to muster everyone into backing up their homedirs, I'd love to remove the mount. TASK DETAIL https://phabricator.wikimedia.org/T208416 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: Ladsgroup

[Wikidata-bugs] [Maniphest] [Commented On] T208416: Check whether wikidata-dev project requires NFS or not

2020-05-20 Thread Andrew
Andrew added a comment. I've removed all the mounts other than /home from all VMs in this project. TASK DETAIL https://phabricator.wikimedia.org/T208416 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: Ladsgroup, Tarrow, ItamarWMDE

[Wikidata-bugs] [Maniphest] [Commented On] T208416: Check whether wikidata-dev project requires NFS or not

2020-05-19 Thread Andrew
Andrew added a comment. There's not a straightforward way to preserve the contents of /home outside of manual copying. For now I'll remove all the mounts other than that one, then you can ping if/when y'all think you're clear of it. Thanks for the response! TASK DETAIL https

[Wikidata-bugs] [Maniphest] [Commented On] T208416: Check whether wikidata-dev project requires NFS or not

2020-05-19 Thread Andrew
Andrew added a comment. Hello again @Addshore! The ask for this remains the same as always. VMs in this project have quite a few NFS volumes mounted, including: /data/project /data/scratch /mnt/nfs/dumps-labstore1006.wikimedia.org /public/dumps /home /mnt/nfs/ I

[Wikidata-bugs] [Maniphest] [Unblock] T206636: Provide a way to have test servers on real hardware, isolated from production for Wikidata Query Service

2020-05-16 Thread Andrew
Andrew closed subtask T221631: Dedicated servers on WMCS to test WDQS scalability strategy as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T206636 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel, Andrew Cc: Iamamz3, bd808, Addshore

[Wikidata-bugs] [Maniphest] [Closed] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

2020-05-16 Thread Andrew
Andrew closed this task as "Resolved". Andrew added a comment. @EBernhardson I've switched the wdqs hosts back to local storage so you should be able to recreate the VMs that you need any time. Let me know if you run into any trouble! TASK DETAIL https://phabricator.wikimedia.o

[Wikidata-bugs] [Maniphest] [Closed] T252784: Remove Ceph from cloudvirt-wdqs100x, Add ceph to cloudvirt1004 and cloudvirt1006

2020-05-16 Thread Andrew
Andrew closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T252784 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: RhinosF1, Aklapper, Andrew, CBogen, Phamhi, darthmon_wmde, Nandana, sietec, Bstorm,

[Wikidata-bugs] [Maniphest] [Unblock] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

2020-05-16 Thread Andrew
Andrew closed subtask T252784: Remove Ceph from cloudvirt-wdqs100x, Add ceph to cloudvirt1004 and cloudvirt1006 as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T221631 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc

[Wikidata-bugs] [Maniphest] [Updated] T252784: Remove Ceph from cloudvirt-wdqs100x, Add ceph to cloudvirt1004 and cloudvirt1006

2020-05-16 Thread Andrew
Andrew added a parent task: T221631: Dedicated servers on WMCS to test WDQS scalability strategy. TASK DETAIL https://phabricator.wikimedia.org/T252784 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: RhinosF1, Aklapper, Andrew

[Wikidata-bugs] [Maniphest] [Updated] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

2020-05-16 Thread Andrew
Andrew added a subtask: T252784: Remove Ceph from cloudvirt-wdqs100x, Add ceph to cloudvirt1004 and cloudvirt1006. TASK DETAIL https://phabricator.wikimedia.org/T221631 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: EBernhardson, JHedden

[Wikidata-bugs] [Maniphest] [Commented On] T252784: Remove Ceph from cloudvirt-wdqs100x, Add ceph to cloudvirt1004 and cloudvirt1006

2020-05-15 Thread Andrew
Andrew added a comment. 1004 and 1006 are now running ceph; I've drained the wdqs nodes but not rebuilt them yet. TASK DETAIL https://phabricator.wikimedia.org/T252784 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: RhinosF1, Aklapper

[Wikidata-bugs] [Maniphest] [Updated] T252784: Remove Ceph from cloudvirt-wdqs100x, Add ceph to cloudvirt1004 and cloudvirt1006

2020-05-14 Thread Andrew
Andrew added projects: Wikidata-Query-Service, cloud-services-team (Kanban). Restricted Application added a project: Wikidata. TASK DETAIL https://phabricator.wikimedia.org/T252784 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: Aklapper

[Wikidata-bugs] [Maniphest] [Claimed] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

2020-05-14 Thread Andrew
Andrew claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T221631 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: EBernhardson, JHedden, Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, Aklapper

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

2020-05-14 Thread Andrew
Andrew added a comment. @EBernhardson sorry for the delay in responding! We can return these hosts to you at any time -- I'll start working on that shortly unless you're back to not needing them :) TASK DETAIL https://phabricator.wikimedia.org/T221631 EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] [Commented On] T208416: Check whether wikidata-dev project requires NFS or not

2020-02-28 Thread Andrew
Andrew added a comment. thanks @WMDE-leszek. the ask here isn't really to do anything other than document here which NFS mounts your project is using, and for what. TASK DETAIL https://phabricator.wikimedia.org/T208416 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

2020-02-28 Thread Andrew
Andrew added a comment. breadcrumbs for whoever picks this up in June: I created a 'wdqs-scaling ' project and a custom flavor 'wdqs-scaling' which is associated with the 'wdqs' host aggregate and is just big enough to fill up one of these servers with a single giant VM. TASK DETAIL

[Wikidata-bugs] [Maniphest] [Unassigned] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

2020-02-28 Thread Andrew
Andrew removed Andrew as the assignee of this task. TASK DETAIL https://phabricator.wikimedia.org/T221631 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, Aklapper

[Wikidata-bugs] [Maniphest] [Changed Status] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

2020-02-28 Thread Andrew
Andrew changed the task status from "Open" to "Stalled". Andrew added a comment. Gehel responded on IRC: > we don't need those cloud wdqs servers for at least another 3 month, maybe more (priority changed). So, this task is stalled until 2020-06-01. @Gehel, i

[Wikidata-bugs] [Maniphest] [Blocker] T206636: Provide a way to have test servers on real hardware, isolated from production for Wikidata Query Service

2020-02-28 Thread Andrew
Andrew changed the status of subtask T221631: Dedicated servers on WMCS to test WDQS scalability strategy from Open to Stalled. TASK DETAIL https://phabricator.wikimedia.org/T206636 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel, Andrew Cc

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

2020-02-27 Thread Andrew
Andrew added a comment. @Gehel after a long time in dc-limbo these hosts are just about ready to go. I'm in the process of setting up a project and custom VM flavor for you to use., Do you have a sense of when you'll have time to pay attention to the new systems? If at this point you

[Wikidata-bugs] [Maniphest] [Triaged] T208416: Check whether wikidata-dev project requires NFS or not

2020-02-18 Thread Andrew
Andrew triaged this task as "Low" priority. TASK DETAIL https://phabricator.wikimedia.org/T208416 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: WMDE-leszek, Addshore, Aklapper, Krenair, Phamhi, darthmon_wmde, Nandana, JKSTNK

[Wikidata-bugs] [Maniphest] [Closed] T208417: Check whether wikidata-primary-source-tool project requires NFS or not

2020-02-12 Thread Andrew
Andrew closed this task as "Resolved". Andrew added a comment. thanks, done! TASK DETAIL https://phabricator.wikimedia.org/T208417 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: Hjfocs, Un1tY, Phamhi, Hook696, Daryl-TTMG, Ro

[Wikidata-bugs] [Maniphest] [Commented On] T208416: Check whether wikidata-dev project requires NFS or not

2020-02-11 Thread Andrew
Andrew added a comment. @Addshore, can you please follow up on this? I see @https://phabricator.wikimedia.org/p/LucasWerkmeister/ creating a fair number of files in /home but not much in /data/project. TASK DETAIL https://phabricator.wikimedia.org/T208416 EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T208417: Check whether wikidata-primary-source-tool project requires NFS or not

2020-02-11 Thread Andrew
Andrew edited subscribers, added: Hjfocs; removed: Aklapper, Krenair. Andrew added a comment. @Hjfocs can you comment on whether or not the wikidata-primary-source-tool makes use of the nfs 'dumps' mount that's provided? We're trying to eliminate unnecessary mounts. TASK DETAIL https

[Wikidata-bugs] [Maniphest] [Claimed] T208417: Check whether wikidata-primary-source-tool project requires NFS or not

2020-02-11 Thread Andrew
Andrew claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T208417 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: Aklapper, Krenair, Phamhi, darthmon_wmde, Nandana, JKSTNK, Fheredia, sietec, Jony, Zylc, Bstorm, 1978Gage2001

[Wikidata-bugs] [Maniphest] [Claimed] T208416: Check whether wikidata-dev project requires NFS or not

2020-02-11 Thread Andrew
Andrew claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T208416 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: Addshore, Aklapper, Krenair, Phamhi, darthmon_wmde, Nandana, JKSTNK, Fheredia, sietec, Jony, Zylc, Bstorm

[Wikidata-bugs] [Maniphest] [Created] T238153: puppet breakage in the wikidata-query VPS project

2019-11-12 Thread Andrew
Andrew created this task. Andrew added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION Three VMs in wikidata-query have broken puppet: sdcquery, dcatap and wdqs-test; they are broken

[Wikidata-bugs] [Maniphest] [Updated] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

2019-09-11 Thread Andrew
Andrew added a subtask: T232654: eqiad: three clouvirt-wdqs servers for WDQS testing. TASK DETAIL https://phabricator.wikimedia.org/T221631 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc: GreenReaper, RazShuty, Krenair, Smalyshev, Andrew

[Wikidata-bugs] [Maniphest] [Commented On] T225312: role::wikibase in wikidata-dev Cloud VPS project broken

2019-06-07 Thread Andrew
Andrew added a comment. I wouldn't say that the role is necessarily broken; it may just be that it needs the hiera args provided if the class is applied. You could provide a good default in the git puppet tree, though, if there is a good universal default. Easier would be to just add

[Wikidata-bugs] [Maniphest] [Commented On] T221063: Request creation of Wikimedia developer account to join existing VPS project

2019-05-03 Thread Andrew
Andrew added a comment. Hm, I seem not to have gotten notified when this was last updated -- sorry! The good news is that self-serve account creation is now back online, so Tupino should be able to just create a new account on wikitech. TASK DETAIL https://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] [Commented On] T221063: Request creation of Wikimedia developer account to join existing VPS project

2019-04-25 Thread Andrew
Andrew added a comment. To do this I need: on-wiki name (presumably 'Tupini07') shell name (could be 'tupini07') email address Once the account is created I will email a password, which you will need to change immediately. TASK DETAIL https://phabricator.wikimedia.org/T221063

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

2019-04-23 Thread Andrew
Andrew added a comment. Can you tell me more about what these hosts would actually do? Are they virt hosts hosting one big VM each? TASK DETAIL https://phabricator.wikimedia.org/T221631 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Andrew Cc

[Wikidata-bugs] [Maniphest] [Commented On] T216404: deployment-db03.deployment-prep.eqiad.wmflabs instance can not start

2019-02-19 Thread Andrew
Andrew added a comment. (For broad context: https://phabricator.wikimedia.org/T215217)TASK DETAILhttps://phabricator.wikimedia.org/T216404EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AndrewCc: bd808, GTirloni, Addshore, Legoktm, Peachey88, Bawolff

[Wikidata-bugs] [Maniphest] [Closed] T210772: wdqs-test.wikidata-query.eqiad.wmflabs is of unknown flavor, needs migrating

2018-11-30 Thread Andrew
Andrew closed this task as "Resolved".Andrew claimed this task.Andrew added a comment. Thanks @Smalyshev ! I'm going to delete that VM but you'll have quota to recreate it in eqiad1.TASK DETAILhttps://phabricator.wikimedia.org/T210772EMAIL PREFERENCEShttps://phabricator.wikimedia.or

[Wikidata-bugs] [Maniphest] [Unblock] T206636: Provide a way to have test servers on real hardware, isolated from production for Wikidata Query Service

2018-11-29 Thread Andrew
Andrew closed subtask T207101: delete t206636-3 VM and revert quota bumps for project wikidata-query as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T206636EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AndrewCc: Addshore, Andrew

[Wikidata-bugs] [Maniphest] [Closed] T207101: delete t206636-3 VM and revert quota bumps for project wikidata-query

2018-11-29 Thread Andrew
Andrew closed this task as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T207101EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AndrewCc: Gehel, Smalyshev, Aklapper, Addshore, Andrew, Legado_Shulgin, Nandana, thifranc, AndyT

[Wikidata-bugs] [Maniphest] [Created] T210772: wdqs-test.wikidata-query.eqiad.wmflabs is of unknown flavor, needs migrating

2018-11-29 Thread Andrew
Andrew created this task.Andrew added projects: Wikidata-Query-Service, cloud-services-team (Kanban).Restricted Application added a subscriber: Aklapper.Restricted Application added a project: Wikidata. TASK DESCRIPTIONToday I was moving the 'wikidata-query' project to the eqiad1-r region

[Wikidata-bugs] [Maniphest] [Commented On] T206636: Provide a way to have test servers on real hardware, isolated from production for Wikidata Query Service

2018-11-29 Thread Andrew
Andrew added a comment. So it sounds like you will need dedicated hardware to make this work (if the original test with the t206636 VM was adequate.) The analytics team is experimenting with this model currently so we'll see how it goes. You would need to figure out about budgeting/procurement

[Wikidata-bugs] [Maniphest] [Unblock] T195121: Contribution from the IGN to Structured Data on Commons

2018-11-14 Thread Andrew
Andrew closed subtask T203072: Request creation of ign2commons VPS project as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T195121EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AndrewCc: fgiunchedi, Reedy, bd808, Aklapper, aborrero, S

[Wikidata-bugs] [Maniphest] [Retitled] T207101: delete t206636-3 VM and revert quota bumps for project wikidata-query

2018-10-29 Thread Andrew
Andrew renamed this task from "delete t206636 VM and revert quota bumps for project wikidata-query" to "delete t206636-3 VM and revert quota bumps for project wikidata-query". TASK DETAILhttps://phabricator.wikimedia.org/T207101EMAIL PREFERENCEShttps://phabricator.wikimedi

[Wikidata-bugs] [Maniphest] [Commented On] T206636: Provide a way to have test servers on real hardware, isolated from production for Wikidata Query Service

2018-10-25 Thread Andrew
Andrew added a comment. In T206636#4690379, @Smalyshev wrote: I've created a new VM, t206636-2.wikidata-query.eqiad.wmflabs I see that t206636-2 is listed as having 4 VPU and 24G RAM. That sounds too small for what I'd need - is this accurate? That's my mistake -- I've build a new VM t206636-3

[Wikidata-bugs] [Maniphest] [Commented On] T206636: Provide a way to have test servers on real hardware, isolated from production for Wikidata Query Service

2018-10-22 Thread Andrew
Andrew added a comment. I've created a new VM, t206636-2.wikidata-query.eqiad.wmflabs. This is in the older region, on a host that is not super busy but is supporting quite a few other VMs. If your tests look good there too then we're probably in good shape and can avoid needing special hardware

[Wikidata-bugs] [Maniphest] [Commented On] T206636: Provide a way to have test servers on real hardware, isolated from production for Wikidata Query Service

2018-10-21 Thread Andrew
Andrew added a comment. Thanks, Stas. There are two ways I think we can go forward with this: Run a second set of tests on a similar VM that shares a host with other active cloud instances (to see if we can meet your needs as a standard user), or Start talking about buying you a private virt

[Wikidata-bugs] [Maniphest] [Commented On] T206636: Provide a way to have test servers on real hardware, isolated from production for Wikidata Query Service

2018-10-18 Thread Andrew
Andrew added a comment. note to self, I can merge https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/468377/ after Stas releases this VM (or at least stops caring about resource contention)TASK DETAILhttps://phabricator.wikimedia.org/T206636EMAIL PREFERENCEShttps://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] [Commented On] T206636: Provide a way to have test servers on real hardware, isolated from production for Wikidata Query Service

2018-10-16 Thread Andrew
Andrew added a comment. @Smalyshev, dumps should be available on that host now.TASK DETAILhttps://phabricator.wikimedia.org/T206636EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AndrewCc: Addshore, Andrew, Aklapper, Smalyshev, Gehel, Nandana, thifranc

[Wikidata-bugs] [Maniphest] [Commented On] T207184: dumps mounts don't show up on eqiad1-r VMs

2018-10-16 Thread Andrew
Andrew added a comment. looks fixed to me!TASK DETAILhttps://phabricator.wikimedia.org/T207184EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Bstorm, AndrewCc: gerritbot, Gehel, Smalyshev, Aklapper, Addshore, Andrew, CucyNoiD, Nandana, NebulousIris, thifranc

[Wikidata-bugs] [Maniphest] [Commented On] T207184: dumps mounts don't show up on eqiad1-r VMs

2018-10-16 Thread Andrew
Andrew added a comment. we discussed this on IRC but for the record, the additional range we need is 172.16.0.0/21TASK DETAILhttps://phabricator.wikimedia.org/T207184EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Bstorm, AndrewCc: Gehel, Smalyshev, Aklapper

[Wikidata-bugs] [Maniphest] [Commented On] T207184: dumps mounts don't show up on eqiad1-r VMs

2018-10-16 Thread Andrew
Andrew added a comment. We might need to check if other things (e.g scratch) have this same issue.TASK DETAILhttps://phabricator.wikimedia.org/T207184EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Bstorm, AndrewCc: Gehel, Smalyshev, Aklapper, Addshore

[Wikidata-bugs] [Maniphest] [Created] T207184: dumps mounts don't show up on eqiad1-r VMs

2018-10-16 Thread Andrew
Andrew created this task.Andrew triaged this task as "High" priority.Andrew added projects: Wikidata-Query-Service, Operations, cloud-services-team, Wikidata. TASK DESCRIPTIONbstorm_> The exports use this for VPS: 10.68.0.0/16TASK DETAILhttps://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] [Commented On] T206636: Provide a way to have test servers on real hardware, isolated from production for Wikidata Query Service

2018-10-16 Thread Andrew
Andrew added a comment. Sorry! Since you mentioned having tested this on a VM in the past I assumed you had a plan in mind to set something up for testing. All of the storage for that instance is hosted on SSDs. I can partition the extra space for you if you want -- you would just want to add

[Wikidata-bugs] [Maniphest] [Commented On] T206636: Provide a way to have test servers on real hardware, isolated from production

2018-10-15 Thread Andrew
Andrew added a comment. I've created a temporary VM for this: t206636.wikidata-query.eqiad.wmflabs -- it's on a new and currently unused host, cloudvirt1023. This test box should be largely uncontested for resources until at least Friday. Let me know if it seems like it might do the trick

[Wikidata-bugs] [Maniphest] [Created] T207101: delete t206636 VM and revert quota bumps for project wikidata-query

2018-10-15 Thread Andrew
Andrew created this task.Andrew triaged this task as "High" priority.Andrew added projects: Wikidata-Query-Service, Operations, cloud-services-team, Wikidata. TASK DESCRIPTIONAs a temporary test for t206636 I've bumped the eqiad1-r quotas for wikidata-query and created a special giant

[Wikidata-bugs] [Maniphest] [Commented On] T206636: Provide a way to have test servers on real hardware, isolated from production

2018-10-15 Thread Andrew
Andrew added a comment. Sorry, my question was poorly phrased. How long would you need to test a given VM box in order to determine whether or not it's an adequate solution for your long-term needs? I can't offer you a permanent setup now but we need to figure out what will work before we talk

[Wikidata-bugs] [Maniphest] [Commented On] T206636: Provide a way to have test servers on real hardware, isolated from production

2018-10-15 Thread Andrew
Andrew added a comment. How long would you need to test? I can allocate this space in a VM for a little while but most existing space is already allocated in the coming weeks.TASK DETAILhttps://phabricator.wikimedia.org/T206636EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Commented On] T206636: Provide a way to have test servers on real hardware, isolated from production

2018-10-15 Thread Andrew
Andrew added a comment. We're unlikely to support bare metal in the near future. I would like to try a couple of things, though... Ensure that your tests are running on an SSD host (the cloud is currently a mix of ssds and spinning disks) Try giving you a private cloudvirt host so your use

[Wikidata-bugs] [Maniphest] [Commented On] T206636: Provide a way to have test servers on real hardware, isolated from production

2018-10-11 Thread Andrew
Andrew added a comment. Hello @Gehel! We're unlikely to support bare metal on Labs in the near future, largely because our pilot program for this years ago determined that we were able to meet almost every actual use case using VMs. Can you tell me a bit more about what you need that a VM can't

[Wikidata-bugs] [Maniphest] [Reassigned] T167154: Delete wikidata-ldf project on wmflabs

2018-08-01 Thread Andrew
Andrew reassigned this task from Andrew to Bstorm.Andrew added a comment. Brooke, I updated the docs about how to do this here: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Projects_lifecycle#Deleting_a_projectTASK DETAILhttps://phabricator.wikimedia.org/T167154EMAIL PREFERENCEShttps

[Wikidata-bugs] [Maniphest] [Closed] T181836: Delete the wikidata-build cloud VPS project

2017-12-12 Thread Andrew
Andrew claimed this task.Andrew closed this task as "Resolved".Andrew added a comment. deletedTASK DETAILhttps://phabricator.wikimedia.org/T181836EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AndrewCc: Andrew, Stashbot, bd808, Aklapper, gerritbot

[Wikidata-bugs] [Maniphest] [Unblock] T181706: Kill the automated wikidata build on labs

2017-12-12 Thread Andrew
Andrew closed subtask T181836: Delete the wikidata-build cloud VPS project as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T181706EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Addshore, AndrewCc: Ladsgroup, gerritbot, Aklapper, Adds

[Wikidata-bugs] [Maniphest] [Commented On] T178661: Drop wb_entity_per_page views in Wiki Replicas

2017-11-26 Thread Andrew
Andrew added a comment. @Marostegui this is on my plate but I have no idea how to proceed... is this timeout due to some long-running job that I need to locate and kill?TASK DETAILhttps://phabricator.wikimedia.org/T178661EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Closed] T180347: Request creation of wikidata-primary-sources-tool VPS project

2017-11-22 Thread Andrew
Andrew closed this task as "Resolved".Andrew claimed this task.Andrew added a comment. @Hjfocs I've created the new project; you should be able to access it and add additional users or project admins as needed.TASK DETAILhttps://phabricator.wikimedia.org/T180347EMAIL PREFER

[Wikidata-bugs] [Maniphest] [Commented On] T167114: Open view for term_full_entity_id in wb_terms table in labs

2017-10-10 Thread Andrew
Andrew added a comment. ok -- I was expecting this table to be present in enwiki. If it's wikidata-specific then we're probably done. @Ladsgroup can you confirm?TASK DETAILhttps://phabricator.wikimedia.org/T167114EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T167114: Open view for term_full_entity_id in wb_terms table in labs

2017-10-10 Thread Andrew
Andrew added a subscriber: Marostegui.Andrew added a comment. I've run maintain-views, but the wb_terms table isn't getting replicated at all. I don't see any evidence of filtering in the sanitarium files but I may be looking in the wrong place... @Marostegui, any ideas?TASK DETAILhttps

[Wikidata-bugs] [Maniphest] [Commented On] T169133: WDQS testing setup platform sizing

2017-09-05 Thread Andrew
Andrew added a comment. ok, flavor added. I'm a bit nervous about how the scheduler will handle this, so please ping me after your instance is created but before you get attached to it so I can make sure it landed someplace reasonable.TASK DETAILhttps://phabricator.wikimedia.org/T169133EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T169133: WDQS testing setup platform sizing

2017-09-05 Thread Andrew
Andrew added a comment. I don't see a project named 'WDQS' -- can you clarify what actual VPS project we're talking about here? I see a few things it could be...TASK DETAILhttps://phabricator.wikimedia.org/T169133EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Updated] T169133: WDQS testing setup platform sizing

2017-08-28 Thread Andrew
Andrew added a project: Cloud-VPS (Quota-requests). TASK DETAILhttps://phabricator.wikimedia.org/T169133EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AndrewCc: daniel, Lydia_Pintscher, bd808, chasemp, Gehel, Aklapper, Smalyshev, GoranSMilovanovic, QZanden

[Wikidata-bugs] [Maniphest] [Closed] T170927: Make wbqc_constraints table available on Quarry et al.

2017-08-25 Thread Andrew
Andrew closed this task as "Resolved".Andrew added a comment. In T170927#3553979, @Marostegui wrote: In T170927#3553623, @Andrew wrote: Blocked by https://phabricator.wikimedia.org/T174182 That above task has been fixed (but that is about hiwikiversity though, not sure how is th

[Wikidata-bugs] [Maniphest] [Updated] T170927: Make wbqc_constraints table available on Quarry et al.

2017-08-25 Thread Andrew
Andrew added a subtask: T174182: set up hiwikiversity on labsdb1010. TASK DETAILhttps://phabricator.wikimedia.org/T170927EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: madhuvishy, AndrewCc: Andrew, madhuvishy, zhuyifei1999, Reedy, Bawolff, dpatrick

[Wikidata-bugs] [Maniphest] [Reopened] T170927: Make wbqc_constraints table available on Quarry et al.

2017-08-25 Thread Andrew
Andrew reopened this task as "Open".Andrew added a comment. Blocked by https://phabricator.wikimedia.org/T174182TASK DETAILhttps://phabricator.wikimedia.org/T170927EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: madhuvishy, AndrewCc: Andrew,

[Wikidata-bugs] [Maniphest] [Unblock] T159215: Wikidata documentation sprint

2017-06-05 Thread Andrew
Andrew closed subtask T165725: Can we search namespaces on wikitech? as "Declined". TASK DETAILhttps://phabricator.wikimedia.org/T159215EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: johl, AndrewCc: Fuzheado, Alicia_Fagerving_WMSE, Crang115, nichti

[Wikidata-bugs] [Maniphest] [Closed] T165728: Organize Labs/Openstack/VPS maintenance docs

2017-05-26 Thread Andrew
Andrew closed this task as "Resolved".Andrew added a comment. I did this, a little bit.TASK DETAILhttps://phabricator.wikimedia.org/T165728EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AndrewCc: chasemp, Aklapper, Lea_Lacroix_WMDE, Trizek-

[Wikidata-bugs] [Maniphest] [Unblock] T159215: Wikidata documentation sprint

2017-05-26 Thread Andrew
Andrew closed subtask T165728: Organize Labs/Openstack/VPS maintenance docs as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T159215EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: johl, AndrewCc: Fuzheado, Alicia_Fagerving_WMSE, Crang115

[Wikidata-bugs] [Maniphest] [Updated] T165962: Request creation of wikidata-documentation labs project

2017-05-26 Thread Andrew
Andrew removed a parent task: T76375: New Labs project requests (tracking). TASK DETAILhttps://phabricator.wikimedia.org/T165962EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Spinster, AndrewCc: Andrew, Aklapper, Spinster, GoranSMilovanovic, Ivana_Isadora

[Wikidata-bugs] [Maniphest] [Commented On] T165962: Request creation of wikidata-documentation labs project

2017-05-26 Thread Andrew
Andrew added a comment. Since this is not actually about a Labs project I'm going to remove the associated parent task.TASK DETAILhttps://phabricator.wikimedia.org/T165962EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Spinster, AndrewCc: Andrew, Aklapper

[Wikidata-bugs] [Maniphest] [Created] T165728: Organize Labs/Openstack/VPS maintenance docs

2017-05-19 Thread Andrew
Andrew created this task.Andrew added projects: Wikimedia-Hackathon-2017, Wikidata, Documentation. TASK DESCRIPTIONThe old wikitech page https://wikitech.wikimedia.org/wiki/OpenStack is 95% obsolete. We need to salvage the useful bits and banish the rest of the content to the Obsolete: namespace

[Wikidata-bugs] [Maniphest] [Block] T159215: Wikidata documentation sprint

2017-05-19 Thread Andrew
Andrew created subtask T165728: Organize Labs/Openstack/VPS maintenance docs. TASK DETAILhttps://phabricator.wikimedia.org/T159215EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: johl, AndrewCc: Fuzheado, Alicia_Fagerving_WMSE, Crang115, nichtich, Tarrow

[Wikidata-bugs] [Maniphest] [Updated] T159215: Wikidata documentation sprint

2017-05-19 Thread Andrew
Andrew added a subtask: T165725: Can we search namespaces on wikitech?. TASK DETAILhttps://phabricator.wikimedia.org/T159215EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: johl, AndrewCc: Fuzheado, Alicia_Fagerving_WMSE, Crang115, nichtich, Tarrow, Rfarrand

[Wikidata-bugs] [Maniphest] [Closed] T155042: Increase quota for wikidata-dev project

2017-01-24 Thread Andrew
Andrew closed this task as "Resolved".Andrew claimed this task.Andrew added a comment. It looks to me like there's quite a bit of quota room in wikidata-dev already. I increased the max instance count by 1, which is probably already more than necessary... Can you create what you nee

[Wikidata-bugs] [Maniphest] [Unblock] T151993: Implement ChangeDispatchCoordinator based on RedisLockManager

2017-01-24 Thread Andrew
Andrew closed subtask T155042: Increase quota for wikidata-dev project as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T151993EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, AndrewCc: gerritbot, Ladsgroup, hoo, Lydia_Pintsc

[Wikidata-bugs] [Maniphest] [Unblock] T148713: Create test system on labs for the Cognate extension

2016-10-24 Thread Andrew
Andrew closed subtask T148889: Request creation of Cognate labs project as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T148713EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Addshore, AndrewCc: Aklapper, Addshore, D3r1ck01, Izno, Wik

[Wikidata-bugs] [Maniphest] [Commented On] T112151: Support POST for SPARQL query endpoint

2015-10-20 Thread Andrew
Andrew added a subscriber: Andrew. Andrew added a comment. WDQS is in internal tool, so if its API doesn't conform to clients' expectations then this is a bug in WDQS, yes? Would it not be better to fix that rather than adding mysterious middleware obfuscation? TASK DETAIL https