[Wikidata-bugs] [Maniphest] [Commented On] T255410: Termbox SSR connection terminated very often

[Pablo-WMDE]

Pablo-WMDE added a comment.


  > termbox can possibly make multiple requests to MediaWiki depending on the 
state of the cache. I guess these each have a timeout?
  
  Yes. All alike AFAIK.
  
  
https://gerrit.wikimedia.org/r/plugins/gitiles/wikibase/termbox/+/21be68a/src/server/server.ts#48
  
https://gerrit.wikimedia.org/r/plugins/gitiles/operations/deployment-charts/+/059eef0/helmfile.d/services/codfw/termbox/values.yaml#7

TASK DETAIL
  https://phabricator.wikimedia.org/T255410

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Pablo-WMDE
Cc: Sakretsu, akosiaris, JMeybohm, WMDE-leszek, Pablo-WMDE, Tarrow, Jakob_WMDE, 
Addshore, Aklapper, Michael, darthmon_wmde, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T255410: Termbox SSR connection terminated very often

[Addshore]

Addshore added a comment.


  So one thing relating to this ticket and timeouts that we realized is that it 
would likely make sense to have the initial MW -> termbox timeout be longer 
than the termbox -> MW timeouts.
  I guess in an ideal situation MW when making the first request would receive 
an error message from termbox service saying the inner MediaWiki request 
failed, rather than the current behaviour which is the first MW -> termbox 
request times out before that happens.
  Another thing to take into account is the fact that termbox can possibly make 
multiple requests to MediaWiki depending on the state of the cache. I guess 
these each have a timeout?
  
  Summary, we should look at all of this and figure out the correct values.

TASK DETAIL
  https://phabricator.wikimedia.org/T255410

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: Sakretsu, akosiaris, JMeybohm, WMDE-leszek, Pablo-WMDE, Tarrow, Jakob_WMDE, 
Addshore, Aklapper, Michael, darthmon_wmde, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T255410: Termbox SSR connection terminated very often

[Michael]

Michael added a comment.


  In T255410#6227218 , 
@JMeybohm wrote:
  
  > @Michael thanks for writing this up!
  >
  > So, if it is safe to assume the MW -> termbox timeout is 3s I would suggest 
we configure the envoys accordingly by setting `tls.upstream_timeout: "3s"` in 
termbox values.yaml as well as `timeout: "3s"` in appservers envoy config (also 
lowering `keepalive` there).
  
  That sounds sensible to me, but I'm really not the expert here. What do 
@Addshore and @Tarrow think? Note that in context of T255436 
, we might look at timeouts again, 
but this has not yet been prioritized, so I cannot speculate when that might be.

TASK DETAIL
  https://phabricator.wikimedia.org/T255410

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael
Cc: akosiaris, JMeybohm, WMDE-leszek, Pablo-WMDE, Tarrow, Jakob_WMDE, Addshore, 
Aklapper, Michael, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, 
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T255410: Termbox SSR connection terminated very often

[JMeybohm]

JMeybohm added a comment.


  @Michael thanks for writing this up!
  
  So, if it is safe to assume the MW -> termbox timeout is 3s I would suggest 
we configure the envoys accordingly by setting `tls.upstream_timeout: "3s"` in 
termbox values.yaml as well as `timeout: "3s"` in appservers envoy config (also 
lowering `keepalive` there).

TASK DETAIL
  https://phabricator.wikimedia.org/T255410

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: JMeybohm
Cc: akosiaris, JMeybohm, WMDE-leszek, Pablo-WMDE, Tarrow, Jakob_WMDE, Addshore, 
Aklapper, Michael, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, 
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T255410: Termbox SSR connection terminated very often

[Michael]

Michael added a comment.


  @akosiaris I edited my comment and it now has links that hopefully should 
work. There should be only a single event behind each of the searches linked 
there.
  
  Also, I added another follow-up task to understand those k8s timeouts 
themselves.

TASK DETAIL
  https://phabricator.wikimedia.org/T255410

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael
Cc: akosiaris, JMeybohm, WMDE-leszek, Pablo-WMDE, Tarrow, Jakob_WMDE, Addshore, 
Aklapper, Michael, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, 
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T255410: Termbox SSR connection terminated very often

[Michael]

Michael added a comment.


  In T255410#6224290 , 
@akosiaris wrote:
  
  > Logstash links are unfortunately not copy-pastable. Anyone clicking on 
those links will get a `Unable to completely restore the URL, be sure to use 
the share functionality.`
  
  Meh! Thanks for pointing that out. I still have those tabs open. I'll try to 
see if there is some persistent share functionality there.

TASK DETAIL
  https://phabricator.wikimedia.org/T255410

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael
Cc: akosiaris, JMeybohm, WMDE-leszek, Pablo-WMDE, Tarrow, Jakob_WMDE, Addshore, 
Aklapper, Michael, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, 
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T255410: Termbox SSR connection terminated very often

[akosiaris]

akosiaris added a comment.


  In T255410#6224239 , 
@Michael wrote:
  
  > 
  
  
  
  > Termbox/k8s: 
https://logstash.wikimedia.org/app/kibana#/doc/logstash-*/logstash-syslog-2020.06.15/syslog?id=AXK3_osLZmYAikdJbyT-&_g=h@e3739c2
  > Mediawiki: 
https://logstash.wikimedia.org/app/kibana#/doc/logstash-*/logstash-mediawiki-2020.06.15/mediawiki?id=AXK3_omyK6TSR36GrxZd&_g=h@e1c60c6
  
  Logstash links are unfortunately not copy-pastable. Anyone clicking on those 
links will get a `Unable to completely restore the URL, be sure to use the 
share functionality.`

TASK DETAIL
  https://phabricator.wikimedia.org/T255410

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: akosiaris
Cc: akosiaris, JMeybohm, WMDE-leszek, Pablo-WMDE, Tarrow, Jakob_WMDE, Addshore, 
Aklapper, Michael, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, 
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T255410: Termbox SSR connection terminated very often

[gerritbot]

gerritbot added a comment.


  Change 605554 **merged** by Alexandros Kosiaris:
  [operations/puppet@production] services_proxy: Add keepalive, retries
  
  https://gerrit.wikimedia.org/r/605554

TASK DETAIL
  https://phabricator.wikimedia.org/T255410

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: gerritbot
Cc: Pablo-WMDE, Tarrow, Jakob_WMDE, Addshore, Aklapper, Michael, Blissjay007, 
Oblanco79, Alter-paule, Beast1978, Un1tY, Hook696, Daryl-TTMG, RomaAmorRoma, 
E.S.A-Sheild, darthmon_wmde, Kent7301, Meekrab2012, joker88john, CucyNoiD, 
Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, 
Adrian1985, Cpaulf30, Lahi, Gq86, Af420, Darkminds3113, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, 
LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T255410: Termbox SSR connection terminated very often

[gerritbot]

gerritbot added a comment.


  Change 605554 had a related patch set uploaded (by Alexandros Kosiaris; 
owner: Alexandros Kosiaris):
  [operations/puppet@production] services_proxy: Add keepalive, retries
  
  https://gerrit.wikimedia.org/r/605554

TASK DETAIL
  https://phabricator.wikimedia.org/T255410

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: gerritbot
Cc: Pablo-WMDE, Tarrow, Jakob_WMDE, Addshore, Aklapper, Michael, darthmon_wmde, 
Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, 
rosalieper, Scott_WUaS, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs