[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[EBernhardson]

EBernhardson added a comment.


  Thanks!

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew, EBernhardson
Cc: EBernhardson, JHedden, Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, 
Andrew, Gehel, bd808, Aklapper, CBogen, Phamhi, darthmon_wmde, Nandana, sietec, 
Bstorm, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, 
Allthingsgo, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, 
Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[Andrew]

Andrew added a comment.


  @EBernhardson sorry for the delay in responding!  We can return these hosts 
to you at any time -- I'll start working on that shortly unless you're back to 
not needing them :)

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew
Cc: EBernhardson, JHedden, Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, 
Andrew, Gehel, bd808, Aklapper, CBogen, Phamhi, darthmon_wmde, Nandana, sietec, 
Bstorm, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, 
Allthingsgo, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, 
Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[EBernhardson]

EBernhardson added a comment.


  Priorities have changed again, commons query service is back at the top of 
the stack. How are we in terms of  a timeline making these available? Have we 
finished up the ceph testing?

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: EBernhardson
Cc: EBernhardson, JHedden, Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, 
Andrew, Gehel, bd808, Aklapper, CBogen, Phamhi, darthmon_wmde, Nandana, sietec, 
Bstorm, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, 
Allthingsgo, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, 
Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[Stashbot]

Stashbot added a comment.


  Mentioned in SAL (#wikimedia-cloud) [2020-03-04T20:03:31Z]  add 
cloudvirt-wdqs100[123] to ceph host aggregate T221631 


TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Stashbot
Cc: JHedden, Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, 
bd808, Aklapper, Beast1978, CBogen, Un1tY, Phamhi, Hook696, Daryl-TTMG, 
RomaAmorRoma, E.S.A-Sheild, darthmon_wmde, Meekrab2012, joker88john, CucyNoiD, 
Nandana, NebulousIris, Gaboe420, sietec, Versusxo, Majesticalreaper22, 
Giuliamocci, Bstorm, Adrian1985, Cpaulf30, Lahi, aborrero, Gq86, Af420, 
Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, 
Adik2382, Chicocvenancio, Allthingsgo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, 
EBjune, merbst, LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, 
rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, 
Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[gerritbot]

gerritbot added a comment.


  Change 576903 **merged** by Jhedden:
  [operations/puppet@production] openstack: switch cloudvirt-wdqs servers to 
Ceph
  
  https://gerrit.wikimedia.org/r/576903

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: gerritbot
Cc: JHedden, Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, 
bd808, Aklapper, Beast1978, CBogen, Un1tY, Phamhi, Hook696, Daryl-TTMG, 
RomaAmorRoma, E.S.A-Sheild, darthmon_wmde, Meekrab2012, joker88john, CucyNoiD, 
Nandana, NebulousIris, Gaboe420, sietec, Versusxo, Majesticalreaper22, 
Giuliamocci, Bstorm, Adrian1985, Cpaulf30, Lahi, aborrero, Gq86, Af420, 
Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, 
Adik2382, Chicocvenancio, Allthingsgo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, 
EBjune, merbst, LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, 
rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, 
Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[JHedden]

JHedden added a comment.


  These hosts will temporarily be used for testing CloudVPS's new Ceph storage 
cluster. To switch back to local storage, we'll need to migrate or delete all 
running virtual machines and revert this patch 
https://gerrit.wikimedia.org/r/c/operations/puppet/+/576903

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: JHedden
Cc: JHedden, Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, 
bd808, Aklapper, Beast1978, CBogen, Un1tY, Phamhi, Hook696, Daryl-TTMG, 
RomaAmorRoma, E.S.A-Sheild, darthmon_wmde, Meekrab2012, joker88john, CucyNoiD, 
Nandana, NebulousIris, Gaboe420, sietec, Versusxo, Majesticalreaper22, 
Giuliamocci, Bstorm, Adrian1985, Cpaulf30, Lahi, aborrero, Gq86, Af420, 
Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, 
Adik2382, Chicocvenancio, Allthingsgo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, 
EBjune, merbst, LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, 
rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, 
Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[gerritbot]

gerritbot added a comment.


  Change 576903 had a related patch set uploaded (by Jhedden; owner: Jhedden):
  [operations/puppet@production] openstack: switch cloudvirt-wdqs servers to 
Ceph
  
  https://gerrit.wikimedia.org/r/576903

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: gerritbot
Cc: Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, 
Aklapper, CBogen, Phamhi, darthmon_wmde, Nandana, sietec, Bstorm, Lahi, 
aborrero, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, 
Allthingsgo, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, 
Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[Andrew]

Andrew added a comment.


  breadcrumbs for whoever picks this up in June:  I created a 'wdqs-scaling ' 
project and a custom flavor 'wdqs-scaling' which is associated with the 'wdqs' 
host aggregate and is just big enough to fill up one of these servers with a 
single giant VM.

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew
Cc: Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, 
Aklapper, Phamhi, darthmon_wmde, Nandana, sietec, Bstorm, Lahi, aborrero, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, 
Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[Andrew]

Andrew added a comment.


  @Gehel after a long time in dc-limbo these hosts are just about ready to go.  
I'm in the process of setting up a project and custom VM flavor for you to use.,
  
  Do you have a sense of when you'll have time to pay attention to the new 
systems?  If at this point you don't want to think about them for a month or 
two we might repurpose them as temporary test boxes; on the other hand if you 
want to start hacking right away I can have them ready for you by tomorrow or 
Monday.

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew
Cc: Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, 
Aklapper, Phamhi, darthmon_wmde, Nandana, sietec, Bstorm, Lahi, aborrero, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, 
Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[ops-monitoring-bot]

ops-monitoring-bot added a comment.


  Completed auto-reimage of hosts:
  
['cloudvirt-wdqs1002.eqiad.wmnet']
  
  and were **ALL** successful.

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew, ops-monitoring-bot
Cc: Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, 
Aklapper, Phamhi, darthmon_wmde, Nandana, sietec, Bstorm, Lahi, aborrero, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, 
Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[ops-monitoring-bot]

ops-monitoring-bot added a comment.


  Completed auto-reimage of hosts:
  
['cloudvirt-wdqs1003.eqiad.wmnet']
  
  and were **ALL** successful.

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew, ops-monitoring-bot
Cc: Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, 
Aklapper, Phamhi, darthmon_wmde, Nandana, sietec, Bstorm, Lahi, aborrero, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, 
Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[ops-monitoring-bot]

ops-monitoring-bot added a comment.


  Script wmf-auto-reimage was launched by andrew on cumin1001.eqiad.wmnet for 
hosts:
  
cloudvirt-wdqs1002.eqiad.wmnet
  
  The log can be found in 
`/var/log/wmf-auto-reimage/202002272018_andrew_134870_cloudvirt-wdqs1002_eqiad_wmnet.log`.

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew, ops-monitoring-bot
Cc: Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, 
Aklapper, Phamhi, darthmon_wmde, Nandana, sietec, Bstorm, Lahi, aborrero, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, 
Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[ops-monitoring-bot]

ops-monitoring-bot added a comment.


  Script wmf-auto-reimage was launched by andrew on cumin1001.eqiad.wmnet for 
hosts:
  
cloudvirt-wdqs1003.eqiad.wmnet
  
  The log can be found in 
`/var/log/wmf-auto-reimage/202002272013_andrew_133073_cloudvirt-wdqs1003_eqiad_wmnet.log`.

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew, ops-monitoring-bot
Cc: Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, 
Aklapper, Phamhi, darthmon_wmde, Nandana, sietec, Bstorm, Lahi, aborrero, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, 
Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[ops-monitoring-bot]

ops-monitoring-bot added a comment.


  Completed auto-reimage of hosts:
  
['cloudvirt-wdqs1003.eqiad.wmnet']
  
  Of which those **FAILED**:
  
['cloudvirt-wdqs1003.eqiad.wmnet']

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew, ops-monitoring-bot
Cc: Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, 
Aklapper, Beast1978, Un1tY, Phamhi, Hook696, Daryl-TTMG, RomaAmorRoma, 
E.S.A-Sheild, darthmon_wmde, Meekrab2012, joker88john, CucyNoiD, Nandana, 
NebulousIris, Gaboe420, sietec, Versusxo, Majesticalreaper22, Giuliamocci, 
Bstorm, Adrian1985, Cpaulf30, Lahi, aborrero, Gq86, Af420, Darkminds3113, 
Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, 
Chicocvenancio, Allthingsgo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, 
merbst, LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, 
Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[ops-monitoring-bot]

ops-monitoring-bot added a comment.


  Completed auto-reimage of hosts:
  
['cloudvirt-wdqs1002.eqiad.wmnet']
  
  Of which those **FAILED**:
  
['cloudvirt-wdqs1002.eqiad.wmnet']

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew, ops-monitoring-bot
Cc: Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, 
Aklapper, Beast1978, Un1tY, Phamhi, Hook696, Daryl-TTMG, RomaAmorRoma, 
E.S.A-Sheild, darthmon_wmde, Meekrab2012, joker88john, CucyNoiD, Nandana, 
NebulousIris, Gaboe420, sietec, Versusxo, Majesticalreaper22, Giuliamocci, 
Bstorm, Adrian1985, Cpaulf30, Lahi, aborrero, Gq86, Af420, Darkminds3113, 
Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, 
Chicocvenancio, Allthingsgo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, 
merbst, LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, 
Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[ops-monitoring-bot]

ops-monitoring-bot added a comment.


  Completed auto-reimage of hosts:
  
['cloudvirt-wdqs1002.eqiad.wmnet']
  
  Of which those **FAILED**:
  
['cloudvirt-wdqs1002.eqiad.wmnet']

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew, ops-monitoring-bot
Cc: Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, 
Aklapper, Beast1978, Un1tY, Phamhi, Hook696, Daryl-TTMG, RomaAmorRoma, 
E.S.A-Sheild, darthmon_wmde, Meekrab2012, joker88john, CucyNoiD, Nandana, 
NebulousIris, Gaboe420, sietec, Versusxo, Majesticalreaper22, Giuliamocci, 
Bstorm, Adrian1985, Cpaulf30, Lahi, aborrero, Gq86, Af420, Darkminds3113, 
Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, 
Chicocvenancio, Allthingsgo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, 
merbst, LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, 
Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[ops-monitoring-bot]

ops-monitoring-bot added a comment.


  Script wmf-auto-reimage was launched by andrew on cumin1001.eqiad.wmnet for 
hosts:
  
cloudvirt-wdqs1003.eqiad.wmnet
  
  The log can be found in 
`/var/log/wmf-auto-reimage/202002271950_andrew_128639_cloudvirt-wdqs1003_eqiad_wmnet.log`.

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew, ops-monitoring-bot
Cc: Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, 
Aklapper, Beast1978, Un1tY, Phamhi, Hook696, Daryl-TTMG, RomaAmorRoma, 
E.S.A-Sheild, darthmon_wmde, Meekrab2012, joker88john, CucyNoiD, Nandana, 
NebulousIris, Gaboe420, sietec, Versusxo, Majesticalreaper22, Giuliamocci, 
Bstorm, Adrian1985, Cpaulf30, Lahi, aborrero, Gq86, Af420, Darkminds3113, 
Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, 
Chicocvenancio, Allthingsgo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, 
merbst, LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, 
Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[ops-monitoring-bot]

ops-monitoring-bot added a comment.


  Script wmf-auto-reimage was launched by andrew on cumin1001.eqiad.wmnet for 
hosts:
  
cloudvirt-wdqs1002.eqiad.wmnet
  
  The log can be found in 
`/var/log/wmf-auto-reimage/202002271950_andrew_128610_cloudvirt-wdqs1002_eqiad_wmnet.log`.

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew, ops-monitoring-bot
Cc: Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, 
Aklapper, Beast1978, Un1tY, Phamhi, Hook696, Daryl-TTMG, RomaAmorRoma, 
E.S.A-Sheild, darthmon_wmde, Meekrab2012, joker88john, CucyNoiD, Nandana, 
NebulousIris, Gaboe420, sietec, Versusxo, Majesticalreaper22, Giuliamocci, 
Bstorm, Adrian1985, Cpaulf30, Lahi, aborrero, Gq86, Af420, Darkminds3113, 
Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, 
Chicocvenancio, Allthingsgo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, 
merbst, LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, 
Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[ops-monitoring-bot]

ops-monitoring-bot added a comment.


  Script wmf-auto-reimage was launched by andrew on cumin1001.eqiad.wmnet for 
hosts:
  
cloudvirt-wdqs1002.eqiad.wmnet
  
  The log can be found in 
`/var/log/wmf-auto-reimage/202002271949_andrew_128500_cloudvirt-wdqs1002_eqiad_wmnet.log`.

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew, ops-monitoring-bot
Cc: Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, 
Aklapper, Beast1978, Un1tY, Phamhi, Hook696, Daryl-TTMG, RomaAmorRoma, 
E.S.A-Sheild, darthmon_wmde, Meekrab2012, joker88john, CucyNoiD, Nandana, 
NebulousIris, Gaboe420, sietec, Versusxo, Majesticalreaper22, Giuliamocci, 
Bstorm, Adrian1985, Cpaulf30, Lahi, aborrero, Gq86, Af420, Darkminds3113, 
Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, 
Chicocvenancio, Allthingsgo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, 
merbst, LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, 
Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[gerritbot]

gerritbot added a comment.


  Change 575312 **merged** by Andrew Bogott:
  [operations/puppet@production] Add cloudvirt-wdqs hosts
  
  https://gerrit.wikimedia.org/r/575312

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew, gerritbot
Cc: Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, 
Aklapper, Beast1978, Un1tY, Phamhi, Hook696, Daryl-TTMG, RomaAmorRoma, 
E.S.A-Sheild, darthmon_wmde, Meekrab2012, joker88john, CucyNoiD, Nandana, 
NebulousIris, Gaboe420, sietec, Versusxo, Majesticalreaper22, Giuliamocci, 
Bstorm, Adrian1985, Cpaulf30, Lahi, aborrero, Gq86, Af420, Darkminds3113, 
Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, 
Chicocvenancio, Allthingsgo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, 
merbst, LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, 
Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[gerritbot]

gerritbot added a comment.


  Change 575312 had a related patch set uploaded (by Andrew Bogott; owner: 
Andrew Bogott):
  [operations/puppet@production] Add cloudvirt-wdqs hosts
  
  https://gerrit.wikimedia.org/r/575312

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew, gerritbot
Cc: Iamamz3, GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, 
Aklapper, Phamhi, darthmon_wmde, Nandana, sietec, Bstorm, Lahi, aborrero, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, 
Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[Smalyshev]

Smalyshev added a comment.


  Both evaluating Virtuoso and other solutions (like JanusGraph) would require 
that. @Gehel should know the details.

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: GreenReaper, RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, Aklapper, 
darthmon_wmde, DannyS712, Nandana, sietec, Lahi, aborrero, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, QZanden, EBjune, 
merbst, LawExplorer, _jensen, rosalieper, Jonas, Xmlizer, jkroll, 
Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331, chasemp
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[bd808]

bd808 added a comment.


  I verified today that we have budget for this build in FY19/20 Q1 
(July-September 2019). Before I start a procurement request, @Gehel can you 
confirm that this test cluster is still desired by the Search team?

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: bd808
Cc: RazShuty, Krenair, Smalyshev, Andrew, Gehel, bd808, Aklapper, 
darthmon_wmde, DannyS712, Nandana, sietec, Lahi, aborrero, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, QZanden, EBjune, 
merbst, LawExplorer, _jensen, rosalieper, Jonas, Xmlizer, jkroll, 
Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331, chasemp
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[bd808]

bd808 added a comment.


  In T221631#5131369 , 
@Gehel wrote:
  
  > So my preference would be to have dedicated servers, with a single VM on 
each, basically using our cloud infrastructure only as an isolation layer. I'm 
not sure how much sense this makes and we're very much open to suggestions!
  
  
  Yes, it makes sense. We are budgeting for the 3 hosts in Q1 of FY19/20.

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: bd808
Cc: Krenair, Smalyshev, Andrew, Gehel, bd808, Aklapper, alaa_wmde, Nandana, 
sietec, Lahi, aborrero, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, 
Chicocvenancio, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, 
Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, 
Mbch331, chasemp
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[Gehel]

Gehel added a comment.


  Yes, this is the continuation of T206636 
.
  
  In T221631#5131322 , 
@bd808 wrote:
  
  > 1. Try to map these requirements into our "standard" cloudvirt hosts and 
expand our FY2019/2020 planned growth to include the needed expansion
  > 2. Spec and rack specific hardware sized for these instances using the 
"fake ironic" model of making them virt servers with a single instance each
  
  
  In term of constraints:
  
  - We know that IO is critical to the performance of WDQS in its current 
implementation, and we suspect the same thing about any replacement solution. 
So I'm not sure that sharing the same IO on the 3 VMs is a great solution.
  - We need to test the distributed nature of whatever replacement solution. By 
hosting the 3 VMs on the same physical host, we will probably hide whatever 
issue might come from increased latency between different physical servers.
  
  So my preference would be to have dedicated servers, with a single VM on 
each, basically using our cloud infrastructure only as an isolation layer. I'm 
not sure how much sense this makes and we're very much open to suggestions!

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Gehel
Cc: Krenair, Smalyshev, Andrew, Gehel, bd808, Aklapper, alaa_wmde, Nandana, 
sietec, Lahi, aborrero, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, 
Chicocvenancio, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, 
Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, 
Mbch331, chasemp
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[bd808]

bd808 added a comment.


  I see 2 ways to approach this:
  
  1. Try to map these requirements into our "standard" cloudvirt hosts and 
expand our FY2019/2020 planned growth to include the needed expansion
  2. Spec and rack specific hardware sized for these instances using the "fake 
ironic" model of making them virt servers with a single instance each
  
  A single cloudvirt identical to cloudvirt10[25-30] would provide all of the 
CPU and RAM needed for all 3 instances. It would not provide the amount of disk 
being asked for however (9T usable). We are actively discussing moving instance 
storage to some shared storage cluster (currently looking into Ceph), but do 
not currently know how this will work out, how long it will take, etc. It is 
also very unknown what kind of IOPS this will provide.

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: bd808
Cc: Krenair, Smalyshev, Andrew, Gehel, bd808, Aklapper, alaa_wmde, Nandana, 
sietec, Lahi, aborrero, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, 
Chicocvenancio, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, 
Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, 
Mbch331, chasemp
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[bd808]

bd808 added a comment.


  In T206636#4941969 , 
@bd808 wrote:
  
  > The next step here is to determine the hardware requirements and then find 
out if there is current fiscal year budget to cover procuring the required 
hardware, or if we can allocate a dedicated server from the existing cloudvirt 
pool and backfill that capacity.
  >
  > The wdqs10{09,10} prod hosts each have 8 cores, 128GB RAM, 4x800GB SSD 
(1.6TB RAID10). Our most recent cloudvirt servers (cloudvirt10[25-30]) are 36 
physical cores, 512GB RAM, 6x1.92TB SSD (5.7TB RAID10). I think this means that 
a single cloudvirt would be more than enough hardware if it was not shared with 
other arbitrary workloads.
  
  
  
  
  In T206636#4946372 , 
@Smalyshev wrote:
  
  > Yes, judging from our preliminary test, if we get uncontested use of the 
server or even a certain chunk of it maybe (not sure if possible?) it would be 
enough. Note that an interesting scenario that we want to test in foreseeable 
future involves cluster setup, so we'd want at least 2 hosts (not sure whether 
they have to be on 2 separate hardware machines) with requirements close to 
what wdqs hosts have. It could splitting cloudvirt host into two VMs 
exclusively used by these test hosts would be ok. Not sure if virtualization 
that we do now allows such kind of fixed resource allocations (probably also 
I/O resources need to be taken care of?)
  
  
  
  
  In T206636#4948039 , 
@bd808 wrote:
  
  > In T206636#4946372 , 
@Smalyshev wrote:
  >
  > > Not sure if virtualization that we do now allows such kind of fixed 
resource allocations (probably also I/O resources need to be taken care of?)
  >
  >
  > We have a few special instances that we do this for today. It is something 
that we are experimenting with for virtualization of shared Cloud Services 
systems such as the ToolsDB databases. The process that requires cloud-root 
assistance to create the initial instances using cli magic, but it is possible. 
Basically we mark the cloudvirt as unavailable for selection by the normal 
OpenStack scheduler and then force create the desired instances there manually.

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: bd808
Cc: Krenair, Smalyshev, Andrew, Gehel, bd808, Aklapper, alaa_wmde, Nandana, 
sietec, Lahi, aborrero, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, 
Chicocvenancio, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, 
Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, 
Mbch331, chasemp
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T221631: Dedicated servers on WMCS to test WDQS scalability strategy

[Andrew]

Andrew added a comment.


  Can you tell me more about what these hosts would actually do?  Are they virt 
hosts hosting one big VM each?

TASK DETAIL
  https://phabricator.wikimedia.org/T221631

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Andrew
Cc: Andrew, Gehel, bd808, Aklapper, alaa_wmde, Nandana, sietec, Lahi, aborrero, 
Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, QZanden, 
EBjune, merbst, LawExplorer, _jensen, rosalieper, Jonas, Xmlizer, jkroll, 
Smalyshev, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331, 
chasemp
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs