[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[jcrespo]

jcrespo added a comment.


  labsdb1009.mgmt (stress on management interface) is down according to icinga 
for 14 hours (around net maintenance), maybe a loose cable or misconfiguration? 
Not a huge blocker, but better make sure it is not an intended/known state.

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, jcrespo
Cc: jcrespo, Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, 
fgiunchedi, hashar, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, 
Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 496721 abandoned by Marostegui:
  db-eqiad.php: Failover db1066 to db1076 on s2
  
  Reason:
  Not needed
  
  https://gerrit.wikimedia.org/r/496721

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, Stashbot, 
Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 496720 abandoned by Marostegui:
  mariadb: Failover db1066 to db1076 on s2
  
  Reason:
  Not needed
  
  https://gerrit.wikimedia.org/r/496720

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, Stashbot, 
Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 496723 abandoned by Marostegui:
  mariadb: Promote db1120 as x1 master
  
  Reason:
  Not needed
  
  https://gerrit.wikimedia.org/r/496723

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, Stashbot, 
Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 496724 abandoned by Marostegui:
  db-eqiad.php: Promote db1120 as x1 master
  
  Reason:
  Not needed
  
  https://gerrit.wikimedia.org/r/496724

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, Stashbot, 
Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[Stashbot]

Stashbot added a comment.


  Mentioned in SAL (#wikimedia-operations) [2019-03-20T06:09:58Z] 
 Synchronized wmf-config/db-eqiad.php: Repool databases 
in row A - T187960  (duration: 00m 
49s)

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, Stashbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, Stashbot, 
Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[Marostegui]

Marostegui added a comment.


  For the record, read only times on s2:
  Read only ON: 15:21:10
  Read only OFF: 15:27:56

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, Marostegui
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, Stashbot, 
Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[Stashbot]

Stashbot added a comment.


  Mentioned in SAL (#wikimedia-operations) [2019-03-19T15:27:56Z] 
 Synchronized wmf-config/db-eqiad.php: Set s2 read only 
OFF - T187960  (duration: 00m 26s)

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, Stashbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, Stashbot, 
Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[Stashbot]

Stashbot added a comment.


  Mentioned in SAL (#wikimedia-operations) [2019-03-19T15:21:10Z] 
 Synchronized wmf-config/db-eqiad.php: Set s2 database 
master on read only - T187960  
(duration: 00m 48s)

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, Stashbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, Stashbot, 
Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 497472 **merged** by jenkins-bot:
  [operations/mediawiki-config@master] db-eqiad.php: Set s2 on read only
  
  https://gerrit.wikimedia.org/r/497472

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, Stashbot, 
Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[Stashbot]

Stashbot added a comment.


  Mentioned in SAL (#wikimedia-operations) [2019-03-19T15:12:24Z]  
eqiad A7 servers uplink move - T187960 


TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, Stashbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, Krinkle, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, 
Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[Stashbot]

Stashbot added a comment.


  Mentioned in SAL (#wikimedia-operations) [2019-03-19T14:44:51Z] 
 Synchronized wmf-config/db-eqiad.php: Depool databases 
in row A - T187960  (duration: 00m 
48s)

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, Stashbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, Krinkle, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, 
Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 497469 **merged** by jenkins-bot:
  [operations/mediawiki-config@master] db-eqiad.php: Depool hosts in row A
  
  https://gerrit.wikimedia.org/r/497469

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, Krinkle, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, 
Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 497472 had a related patch set uploaded (by Marostegui; owner: 
Marostegui):
  [operations/mediawiki-config@master] db-eqiad.php: Set s2 on read only
  
  https://gerrit.wikimedia.org/r/497472

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, Krinkle, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, 
Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 497469 had a related patch set uploaded (by Marostegui; owner: 
Marostegui):
  [operations/mediawiki-config@master] db-eqiad.php: Depool hosts in row A
  
  https://gerrit.wikimedia.org/r/497469

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, Krinkle, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, 
Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 497319 **merged** by Bstorm:
  [operations/puppet@production] dumps distribution: failing over the web only 
for network changes
  
  https://gerrit.wikimedia.org/r/497319

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, Krinkle, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, 
Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 497420 **merged** by Bstorm:
  [operations/dns@master] dumps distribution: fail over dumps to labstore1007
  
  https://gerrit.wikimedia.org/r/497420

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, Krinkle, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, 
Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 497420 had a related patch set uploaded (by Bstorm; owner: Bstorm):
  [operations/dns@master] dumps distribution: fail over dumps to labstore1007
  
  https://gerrit.wikimedia.org/r/497420

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, Krinkle, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, 
Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 497328 **merged** by Bstorm:
  [operations/dns@master] dumps distribution: reduce TTL on labstore1006 for 
failover
  
  https://gerrit.wikimedia.org/r/497328

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, Krinkle, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, 
Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 497328 had a related patch set uploaded (by Bstorm; owner: Bstorm):
  [operations/dns@master] dumps distribution: reduce TTL on labstore1006 for 
failover
  
  https://gerrit.wikimedia.org/r/497328

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, Krinkle, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, 
Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 497319 had a related patch set uploaded (by Bstorm; owner: Bstorm):
  [operations/puppet@production] dumps distribution: failing over the web only 
for network changes
  
  https://gerrit.wikimedia.org/r/497319

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, Krinkle, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, 
Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 496724 had a related patch set uploaded (by Marostegui; owner: 
Marostegui):
  [operations/mediawiki-config@master] db-eqiad.php: Promote db1120 as x1 master
  
  https://gerrit.wikimedia.org/r/496724

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, Krinkle, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, 
Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 496723 had a related patch set uploaded (by Marostegui; owner: 
Marostegui):
  [operations/puppet@production] mariadb: Promote db1120 as x1 master
  
  https://gerrit.wikimedia.org/r/496723

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, Krinkle, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, 
Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 496721 had a related patch set uploaded (by Marostegui; owner: 
Marostegui):
  [operations/mediawiki-config@master] db-eqiad.php: Failover db1066 to db1076 
on s2
  
  https://gerrit.wikimedia.org/r/496721

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, Krinkle, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, 
Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[gerritbot]

gerritbot added a comment.


  Change 496720 had a related patch set uploaded (by Marostegui; owner: 
Marostegui):
  [operations/puppet@production] mariadb: Failover db1066 to db1076 on s2
  
  https://gerrit.wikimedia.org/r/496720

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, gerritbot
Cc: Tgr, Dbrant, JoeWalsh, Bstorm, thcipriani, akosiaris, Joe, fgiunchedi, 
hashar, Krinkle, ArielGlenn, jijiki, Addshore, Catrope, elukey, Marostegui, 
Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, 
Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, 
Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[ayounsi]

ayounsi added a comment.


  Thank you all for the quick replies!
  
  In T187960#5000934 , 
@hashar wrote:
  
  > Is that just about re cabling the server from a switch to another? If so I 
would expect the downtime to be rather minimal.
  
  
  Correct, ~5s if no issue
  
  > Note that iirc contint1001 is limited to 100Mbps, I think on purpose to 
avoid having CI to overflow the network layer somehow.
  
  The link is negotiated at 1000Mbps, and there is no rate limiter on the 
switch side. So unless there is rate-limiter on the server side, this operates 
at 1G.
  Maybe it was true on an older infra, but now the network is able to handle 1G 
hosts.
  
  > Will have to announce it ahead of time and gracefully shutdown CI, I am not 
sure how the stack would react on a network interruption.
  
  Better figure it out during a planned maintenance than a 4am outage ;)
  
  In T187960#5001105 , 
@elukey wrote:
  
  >   IIUC this maintenance will not require any host shutdown [...] (please 
@ayounsi confirm that :)
  
  
  Confirmed. Server list updated to include your comments.

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, ayounsi
Cc: thcipriani, akosiaris, Joe, fgiunchedi, hashar, Krinkle, ArielGlenn, 
jijiki, Addshore, MMiller_WMF, Catrope, elukey, Marostegui, Stashbot, Paladox, 
gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, Legado_Shulgin, 
CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, Gaboe420, 
Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, 
Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[elukey]

elukey added a comment.


ge-6/0/25 - mc1019
ge-6/0/26 - mc1020
ge-6/0/27 - mc1021
ge-6/0/28 - mc1022
ge-6/0/29 - mc1023
  
  The above ones are holding the eqiad mediawiki object cache, they are 
extremely important. IIUC this maintenance will not require any host shutdown, 
that would mean stopping memcached and wipe the cache (please @ayounsi confirm 
that :), so a network blip of one at the time will be fine. Please do it one at 
the time, checking with me or Joe that nothing is exploding on the mediawiki 
side while the maintenance is ongoing.

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, elukey
Cc: thcipriani, akosiaris, Joe, fgiunchedi, hashar, Krinkle, ArielGlenn, 
jijiki, Addshore, MMiller_WMF, Catrope, elukey, Marostegui, Stashbot, Paladox, 
gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, Legado_Shulgin, 
CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, Gaboe420, 
Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, 
Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[elukey]

elukey added a comment.


  About Analytics nodes:
  
  - ge-1/0/7  - kafka-jumbo1001 -> Kafka needs to be stopped ~10/15 minutes 
beforehand to have a graceful shutdown (if possible)
  - ge-2/0/12 - kafka-jumbo1002 -> same thing, and it is better to have 
kafka-jumbo1001 up and running (with Kafka partitions sync recovered) before 
doing maintenance on 1002. We can survive two brokers down but since we control 
the maintenance I'd prefer not :)
  
  - ge-2/0/17 - analytics1012
  - ge-2/0/18 - analytics1013
  - ge-2/0/19 - notebook1002 -> all these are decommed nodes, should we move 
them or just unrack them?
  
  - ge-3/0/26 - analytics1052
  - ge-3/0/27 - analytics1053
  - ge-3/0/28 - analytics1054
  - ge-3/0/29 - analytics1055
  - ge-3/0/30 - analytics1056
  - ge-3/0/31 - analytics1057
  - ge-3/0/33 - analytics1059
  - ge-3/0/34 - analytics1060  -> all hadoop worker nodes, one at the time 
would be great
  
  - ge-4/0/0  - druid1001 - depool va pybal --> it is not pooled in pybal 
(druid100[4-6] are), some heads up beforehand to gracefully stop the daemons 
would be good.
  
  - ge-4/0/1  - aqs1004 - depool va pybal -> some heads up beforehand to 
gracefully stop cassandra would be good.
  
  - ge-4/0/6  - analytics1070 -> hadoop worker node, no special needs
  
  - ge-4/0/25 - kafka1001 - depool from pybal --> Kafka would need to be 
gracefully stopped if possible beforehand
  
  - ge-6/0/12 - druid1004 - depool via pybal -> heads up beforehand would be 
nice to gracefully stop daemons
  
  - ge-6/0/15 - an-master1001 - status planned -> This is the hadoop master 
node, extremely important, I need to perform a failover beforehand.
  
  - ge-6/0/24 - aqs1007 - depool from pybal -> need some time to gracefully 
stop cassandra, and please do not do it when aqs1004 is down as well.

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, elukey
Cc: thcipriani, akosiaris, Joe, fgiunchedi, hashar, Krinkle, ArielGlenn, 
jijiki, Addshore, MMiller_WMF, Catrope, elukey, Marostegui, Stashbot, Paladox, 
gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, Legado_Shulgin, 
CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, Gaboe420, 
Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, 
Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[Marostegui]

Marostegui added a comment.


  Will do! Thanks!

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, Marostegui
Cc: Addshore, MMiller_WMF, Catrope, elukey, Marostegui, Stashbot, Paladox, 
gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, Legado_Shulgin, 
CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, Gaboe420, 
Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, 
Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg, akosiaris, fgiunchedi
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[Addshore]

Addshore added a comment.


  In T187960#4998831 , 
@Marostegui wrote:
  
  > In T187960#4998807 , 
@Addshore wrote:
  >
  > > From where I am sat setting $wgReadOnly for the few seconds would be the 
best thing for cognate, and would avoid having to run the maintenance scripts.
  >
  >
  > Thanks for the explanation! If you consider that is the best approach, I am 
fine with that! Can you be available during the maintenance window to 
coordinate? :)
  
  
  Tuesday the 19th, I sure can, please ping me  then :)

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, Addshore
Cc: Addshore, MMiller_WMF, Catrope, elukey, Marostegui, Stashbot, Paladox, 
gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, Legado_Shulgin, 
CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, Gaboe420, 
Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, 
Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg, akosiaris, fgiunchedi
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack

[Marostegui]

Marostegui added a comment.


  In T187960#4998807 , 
@Addshore wrote:
  
  > From where I am sat setting $wgReadOnly for the few seconds would be the 
best thing for cognate, and would avoid having to run the maintenance scripts.
  
  
  Thanks for the explanation! If you consider that is the best approach, I am 
fine with that! Can you be available during the maintenance window to 
coordinate? :)

TASK DETAIL
  https://phabricator.wikimedia.org/T187960

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Cmjohnson, Marostegui
Cc: Addshore, MMiller_WMF, Catrope, elukey, Marostegui, Stashbot, Paladox, 
gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, Legado_Shulgin, 
CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, Gaboe420, 
Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, 
Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, 
LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, 
Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, 
mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, 
KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg, akosiaris, fgiunchedi
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs