[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2022-01-27 Thread dcausse 
dcausse closed this task as "Resolved".
dcausse added a comment.


  Thanks for the report, blazegraph died I restarted it, should be available 
again now.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, dcausse
Cc: dcausse, Theklan, Marsupium, Vojtech.dostal, Base, RhinosF1, Majavah, 
aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2022-01-27 Thread Fuzheado 
Fuzheado added a comment.


  Agree, here's the specific error coming back:
  
  
  502 Bad Gateway
  
  502 Bad Gateway
  nginx/1.13.9
  
  
  

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, Fuzheado
Cc: Theklan, Marsupium, Vojtech.dostal, Base, RhinosF1, Majavah, aborrero, 
GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2022-01-26 Thread Dominicbm 
Dominicbm added a comment.


  I reopened this, as WCQS is again giving 502 Bad Gateway error, though 
slightly different behavior for me. wcqs-beta.wmflabs.org resolves, but gives 
the error as the response when clicking the "Run" button.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, Dominicbm
Cc: Theklan, Marsupium, Vojtech.dostal, Base, RhinosF1, Majavah, aborrero, 
GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2022-01-26 Thread Dominicbm 
Dominicbm reopened this task as "Open".

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, Dominicbm
Cc: Theklan, Marsupium, Vojtech.dostal, Base, RhinosF1, Majavah, aborrero, 
GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2022-01-10 Thread dcaro 
dcaro added a comment.


  If you create any new VMs, or re-image the existing ones, use that new flavor 
and it will avoid getting the host out of space (it has a slightly smaller 
disk).

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro
Cc: Theklan, Marsupium, Vojtech.dostal, Base, RhinosF1, Majavah, aborrero, 
GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2022-01-10 Thread Stashbot 
Stashbot added a comment.


  Mentioned in SAL (#wikimedia-cloud) [2022-01-10T13:56:24Z]  Replace 
too big flavor t206636 for a one with a smaller disk, t206636v2 (T297454 
)

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, Stashbot
Cc: Theklan, Marsupium, Vojtech.dostal, Base, RhinosF1, Majavah, aborrero, 
GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2022-01-10 Thread Gehel 
Gehel closed this task as "Resolved".

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, Gehel
Cc: Theklan, Marsupium, Vojtech.dostal, Base, RhinosF1, Majavah, aborrero, 
GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2022-01-04 Thread EBernhardson 
EBernhardson added a comment.


  The beta service looks to have unintentionally picked up some of the 
configuration of the production cluster.  I've put the configuration back and 
disabled the beta instance (puppet) from updating itself which should keep 
things in the current state as we make changes to roll out the production 
service.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, EBernhardson
Cc: Theklan, Marsupium, Vojtech.dostal, Base, RhinosF1, Majavah, aborrero, 
GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-30 Thread RhinosF1 
RhinosF1 added a comment.


  Assuming it is released on time, yes.
  
  The search team aren't going to put something unstable in production.
  
  The production service will use its own dedicated and seperate hardware to 
beta though.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, RhinosF1
Cc: Theklan, Marsupium, Vojtech.dostal, Base, RhinosF1, Majavah, aborrero, 
GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-30 Thread Sj 
Sj added a comment.


  I get a 500 error now... will this be back up before the general release on 
Feb 1?

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, Sj
Cc: Theklan, Marsupium, Vojtech.dostal, Base, RhinosF1, Majavah, aborrero, 
GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-20 Thread Gehel 
Gehel moved this task from Incoming to Current work on the 
Wikidata-Query-Service board.
Gehel added a project: Discovery-Search (Current work).

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

WORKBOARD
  https://phabricator.wikimedia.org/project/board/891/

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, Gehel
Cc: Base, RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, 
Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, 
Jarekt, Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, 
Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, 
Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, 
Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, 
Jdouglas, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, 
Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-15 Thread EBernhardson 
EBernhardson added a comment.


  In T297454#7572089 , 
@dcaro wrote:
  
  > I had to stop (shutoff) and start the VM again (to clear the error state), 
the VM is up and running, and what's using all the space is:
  >
  >   root@wcqs-beta-01:/srv/wdqs-data# du -hs *
  >   4.0Kaliases.map
  >   4.0Kdumps
  >   28G latest-mediainfo.ttl.gz
  >   28G munged
  >   72K sdoc.jnl
  >   4.0Ktarget
  >   3.1Twcqs.jnl
  >
  > I'll turn off the VM again, so you can turn it on and cleanup before it 
fills up again, let me know how it goes.
  
  Thanks! I was able to start the instance this time around. The `wcqs.jnl` 
can't be shrunk, it's a known problem of the beta installation. We've deleted 
the existing journal and started a fresh data load. It will take a few days 
before fully coming back online. Thi data load will  fill 1T or so.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, EBernhardson
Cc: Base, RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, 
Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, 
Jarekt, Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, 
Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, 
Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, 
Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, 
Jdouglas, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, 
Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-15 Thread Stashbot 
Stashbot added a comment.


  Mentioned in SAL (#wikimedia-cloud) [2021-12-15T12:44:25Z]  Downtiming 
cloudvirt-wdqs1001 as it has no VMs running until disk space is fixed (T297454 
)

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, Stashbot
Cc: Base, RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, 
Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, 
Jarekt, Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, 
Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, 
Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, 
Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, 
Jdouglas, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, 
Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-15 Thread Stashbot 
Stashbot added a comment.


  Mentioned in SAL (#wikimedia-operations) [2021-12-15T12:43:27Z] 
 END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 30 
days, 0:00:00 on cloudvirt-wdqs1001.eqiad.wmnet with reason: T297454 


TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, Stashbot
Cc: Base, RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, 
Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, 
Jarekt, Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, 
Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, 
Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, 
Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, 
Jdouglas, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, 
Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-15 Thread Stashbot 
Stashbot added a comment.


  Mentioned in SAL (#wikimedia-operations) [2021-12-15T12:43:23Z] 
 START - Cookbook sre.hosts.downtime for 30 days, 0:00:00 on 
cloudvirt-wdqs1001.eqiad.wmnet with reason: T297454 


TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, Stashbot
Cc: Base, RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, 
Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, 
Jarekt, Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, 
Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, 
Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, 
Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, 
Jdouglas, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, 
Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-15 Thread Stashbot 
Stashbot added a comment.


  Mentioned in SAL (#wikimedia-operations) [2021-12-15T12:43:14Z] 
 END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 
4:00:00 on cloudvirt-wdqs1001.eqiad.wmnet with reason: T297454 


TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, Stashbot
Cc: Base, RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, 
Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, 
Jarekt, Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, 
Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, 
Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, 
Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, 
Jdouglas, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, 
Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-15 Thread Stashbot 
Stashbot added a comment.


  Mentioned in SAL (#wikimedia-operations) [2021-12-15T12:43:09Z] 
 START - Cookbook sre.hosts.downtime for 4:00:00 on 
cloudvirt-wdqs1001.eqiad.wmnet with reason: T297454 


TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, Stashbot
Cc: Base, RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, 
Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, 
Jarekt, Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, 
Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, 
Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, 
Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, 
Jdouglas, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, 
Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-15 Thread dcaro 
dcaro added a comment.


  I had to stop (shutoff) and start the VM again (to clear the error state), 
the VM is up and running, and what's using all the space is:
  
root@wcqs-beta-01:/srv/wdqs-data# du -hs *
4.0Kaliases.map
4.0Kdumps
28G latest-mediainfo.ttl.gz
28G munged
72K sdoc.jnl
4.0Ktarget
3.1Twcqs.jnl
  
  I'll turn off the VM again, so you can turn it on and cleanup before it fills 
up again, let me know how it goes.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro
Cc: Base, RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, 
Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, 
Jarekt, Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, 
Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, 
Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, 
Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, 
Jdouglas, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, 
Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-14 Thread EBernhardson 
EBernhardson added a comment.


  In T297454#7568856 , 
@dcaro wrote:
  
  > I've moved the other VM around, so there's a little bit of space free now, 
should be enough to start the VM but you'll have to make sure to cleanup right 
when it comes back up, if it's not possible to get it to less than half it's 
size, we will not be able to shrink the image though.
  >
  > Let me know how it goes.
  
  Oddly it's still giving the initial error when starting the instance from 
horizon, potentially that is a second issue unrelated to the instance becoming 
unavailable?:
  
Error: You are not allowed to start instance: wcqs-beta-01

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, EBernhardson
Cc: Base, RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, 
Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, 
Jarekt, Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, 
Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, 
Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, 
Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, 
Jdouglas, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, 
Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-14 Thread Stashbot 
Stashbot added a comment.


  Mentioned in SAL (#wikimedia-cloud) [2021-12-14T10:26:14Z]  Moved the 
nova cache (/var/lib/nova/instances/_base) and the canary image local data 
(/var/lib/nova/instance/) to the root disk on 
cloudvirt-wdqs1001 to temporary free some space (T297454 
)

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, Stashbot
Cc: Base, RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, 
Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, 
Jarekt, Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, 
Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, 
Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, 
Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, 
Jdouglas, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, 
Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-14 Thread dcaro 
dcaro added a comment.


  I've moved the other VM around, so there's a little bit of space free now, 
should be enough to start the VM but you'll have to make sure to cleanup right 
when it comes back up, if it's not possible to get it to less than half it's 
size, we will not be able to shrink the image though.
  
  Let me know how it goes.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro
Cc: Base, RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, 
Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, 
Jarekt, Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, 
Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, 
Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, 
Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, 
Jdouglas, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, 
Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-14 Thread dcaro 
dcaro reopened this task as "Open".
dcaro added a comment.


  Sorry about that.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro
Cc: Base, RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, 
Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, 
Jarekt, Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, 
Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, 
Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, 
Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, 
Jdouglas, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, 
Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-14 Thread dcaro 
dcaro closed this task as "Resolved".

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro
Cc: Base, RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, 
Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, 
Jarekt, Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, 
Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, 
Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, 
Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, 
Jdouglas, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, 
Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-13 Thread dcaro 
dcaro added a comment.


  The underlying filesystem is full, the disk size was set too big (3.4T), 
there's a cloudcanary instance taking 20G that can be removed to try to start 
the VM and cleanup stuff, but we'll need to resize the disk later once the 
sapce is freed. Not sure if that's possible, but will check.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro
Cc: Base, RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, 
Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, 
Jarekt, Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, 
Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, 
Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, 
Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, 
Jdouglas, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, 
Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-13 Thread dcaro 
dcaro changed the task status from "In Progress" to "Open".

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro
Cc: RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, 
Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, 
Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, 
JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, 
SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, 
Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, 
Jdouglas, Base, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, 
Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-13 Thread Majavah 
Majavah added a comment.


  The hypervisor box it's running on has apparently filled its disk:
  
taavi@cloudvirt-wdqs1001 ~ $ df -h /var/lib/nova/instances/
Filesystem Size  Used Avail Use% Mounted on
/dev/mapper/tank-data  3.3T  3.3T   20K 100% /var/lib/nova/instances

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, Majavah
Cc: Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, 
HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, 
MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, 
Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-13 Thread EBernhardson 
EBernhardson added a comment.


  In T297454#7567004 , @Sj 
wrote:
  
  > Is this monitored by any of the status tools?  Does it just need to be 
restarted?
  
  As a beta service running outside of production this does not have access to 
much of the tooling we generally use to ensure uptime. Downtime of the beta 
service is considered routine and expected.
  
  > (and is it normal for projectadmin's to not have the ability to restart, or 
a feature of this being  complex service?)
  
  This likely has more to do with the beta service running on non-standard 
cloudvirt's (the underlying hardware is different than the rest of WMCS) that 
are special cased for this one service. In general projectadmin's can 
start/stop/create/delete instances at will.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, EBernhardson
Cc: aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-13 Thread aborrero 
aborrero added a comment.


  the VM is in error state:
  
  This is what I get from the openstack API:
  
{'code': 500, 'created': '2021-12-10T19:19:35Z', 'message': 'OSError', 
'details': 'Traceback (most recent call last):\n  File 
"/usr/lib/python3/dist-packages/nova/compute/manager.py", line 205, in 
decorated_function\nreturn function(self, context, *args, **kwargs)\n  File 
"/usr/lib/python3/dist-packages/nova/compute/manager.py", line 3743, in 
reboot_instance\ndo_reboot_instance(context, instance, block_device_info, 
reboot_type)\n  File 
"/usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py", line 360, in 
inner\nreturn f(*args, **kwargs)\n  File 
"/usr/lib/python3/dist-packages/nova/compute/manager.py", line 3742, in 
do_reboot_instance\nreboot_type)\n  File 
"/usr/lib/python3/dist-packages/nova/compute/manager.py", line 3835, in 
_reboot_instance\nself._set_instance_obj_error_state(instance)\n  File 
"/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in 
__exit__\nself.force_reraise()\n  File 
"/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in 
force_reraise\nsix.reraise(self.type_, self.value, self.tb)\n  File 
"/usr/lib/python3/dist-packages/six.py", line 703, in reraise\nraise 
value\n  File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 
3810, in _reboot_instance\nbad_volumes_callback=bad_volumes_callback)\n  
File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 3250, 
in reboot\nblock_device_info, accel_info)\n  File 
"/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 3343, in 
_hard_reboot\nmdevs=mdevs, accel_info=accel_info)\n  File 
"/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6501, in 
_get_guest_xml\ncontext, mdevs, accel_info)\n  File 
"/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6146, in 
_get_guest_config\nflavor, guest.os_type)\n  File 
"/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 4817, in 
_get_guest_storage_config\ninst_type)\n  File 
"/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 4724, in 
_get_guest_disk_config\nconf = disk.libvirt_info(disk_info, 
self.disk_cachemode,\n  File 
"/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 556, in 
disk_cachemode\nif not 
nova.privsep.utils.supports_direct_io(CONF.instances_path):\n  File 
"/usr/lib/python3/dist-packages/nova/privsep/utils.py", line 74, in 
supports_direct_io\n{\'path\': dirpath, \'ex\': e})\n  File 
"/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in 
__exit__\nself.force_reraise()\n  File 
"/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in 
force_reraise\nsix.reraise(self.type_, self.value, self.tb)\n  File 
"/usr/lib/python3/dist-packages/six.py", line 703, in reraise\nraise 
value\n  File "/usr/lib/python3/dist-packages/nova/privsep/utils.py", line 57, 
in supports_direct_io\nfd = os.open(testfile, os.O_CREAT | os.O_WRONLY | 
os.O_DIRECT)\n  File "/usr/lib/python3/dist-packages/eventlet/green/os.py", 
line 118, in open\nfd = __original_open__(file, flags, mode)\nOSError: 
[Errno 28] No space left on device: 
\'/var/lib/nova/instances/.directio.test.5199023175537772324\'\n'}
  
  I don't know what happened. First time I see this error.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro, aborrero
Cc: aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-13 Thread dcaro 
dcaro added a project: Cloud-Services-Origin-User.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro
Cc: GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-13 Thread dcaro 
dcaro added a project: Cloud-Services-Worktype-Unplanned.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro
Cc: GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-13 Thread dcaro 
dcaro claimed this task.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro
Cc: GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-13 Thread dcaro 
dcaro changed the task status from "Open" to "In Progress".

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro
Cc: GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-13 Thread dcaro 
dcaro added a project: User-dcaro.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcaro
Cc: GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, 
Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-13 Thread Sj 
Sj added a comment.


  Is this monitored by any of the status tools?  Does it just need to be 
restarted?  
  (and is it normal for projectadmin's to not have the ability to restart, or a 
feature of this being  complex service?)

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Sj
Cc: Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, 
EBernhardson, Aklapper, Jarekt, Invadibot, GFontenelle_WMF, MPhamWMF, 
maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-10 Thread EBernhardson 
EBernhardson added a comment.


  This is still the correct URL. I don't have exact logs, but it looks like 
around 2021-12-09T17:44 the instance stopped running and upon inspection 
reports a status of error and power state of paused.  Around 20 minutes after I 
started poking it the power state changed to No State. I have `projectadmin` 
rights for the project, but attempting to start the instance reports I don't 
have appropriate rights. Will likely need a wmcs admin to poke it.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: EBernhardson
Cc: EBernhardson, Aklapper, Jarekt, Invadibot, GFontenelle_WMF, MPhamWMF, 
maantietaja, FRomeo_WMF, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, 
Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-09 Thread Maintenance_bot 
Maintenance_bot added a project: Wikidata.

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Maintenance_bot
Cc: Aklapper, Jarekt, Invadibot, GFontenelle_WMF, MPhamWMF, maantietaja, 
FRomeo_WMF, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, 
Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, 
merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, 
Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, 
matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, 
Lydia_Pintscher, Raymond, Steinsplitter, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"

2021-12-09 Thread Jarekt 
Jarekt created this task.
Jarekt added projects: Wikidata-Query-Service, SDC General.
Restricted Application added a subscriber: Aklapper.

TASK DESCRIPTION
  Wikimedia Commons Query Service seems to be down, as 
https://wcqs-beta.wmflabs.org/ is giving me "502 Bad Gateway Error". I have not 
used WCQS in a bit so maybe something changed, like URL (?).

TASK DETAIL
  https://phabricator.wikimedia.org/T297454

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Jarekt
Cc: Aklapper, Jarekt, GFontenelle_WMF, MPhamWMF, FRomeo_WMF, CBogen, 
Nintendofan885, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, 
Cparle, Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, EBjune, Tramullas, Acer, 
merbst, Salgo60, Silverfish, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, 
Wikidata-bugs, Jdouglas, Base, matthiasmullie, aude, Tobias1984, Manybubbles, 
Ricordisamoa, Wesalius, Lydia_Pintscher, Raymond, Steinsplitter
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org