[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

akosiaris added a comment. In T212189#5289724 , @Tarrow wrote: > @akosiaris Yep; we've interpreted it as something we really need before exposing it to real traffic. We've got a ticket open about it that we'll be picking up real soon:

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

akosiaris added a comment. @WMDE-leszek, @Tarrow. Any feedback on the comment above? TASK DETAIL https://phabricator.wikimedia.org/T212189 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: akosiaris Cc: Mholloway, RazShuty, sbassett,

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

akosiaris added a comment. @WMDE-leszek, @Tarrow. I 've noticed we are missing one thing. We have a dashboard for the service's metrics in https://grafana.wikimedia.org/d/AJf0z_7Wz/termbox but it looks like the service isn't sending request metrics to the local statsd instance. It is

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

WMDE-leszek added a comment. Sounds good, thanks @akosiaris ! We monitor T220402 then. TASK DETAIL https://phabricator.wikimedia.org/T212189 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: WMDE-leszek

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

WMDE-leszek added a comment. Disclaimer: I do understand that SRE and others have been pretty busy last weeks, and I would absolutely take "we cannot really say this week" as an answer. As it is Q4/Q2 now, I wondered @akosiaris whether we'd be able to narrow down somehow a timeline of

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

RazShuty added a comment. Thanks a lot! TASK DETAIL https://phabricator.wikimedia.org/T212189 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RazShuty Cc: RazShuty, sbassett, thcipriani, Tarrow, Smalyshev, jijiki, fsero, CDanis, akosiaris,

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Joe added a comment. In T212189#5053087 , @RazShuty wrote: > Hey @akosiaris, not sure I see it in there, maybe I'm lost a bit... can you point me out to where the SSR is in

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

RazShuty added a comment. Hey @akosiaris, not sure I see it in there, maybe I'm lost a bit... can you point me out to where the SSR is in https://www.mediawiki.org/w/index.php?title=Wikimedia_Technology/Annual_Plans/FY2019/TEC3:_Deployment_Pipeline/Goals#Q4_Goals ? TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

akosiaris added a comment. In T212189#5044451 , @Addshore wrote: > In T212189#5020187 , @akosiaris wrote: > > > Thanks for the understanding. We are drafting next quarter goals this

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Addshore added a comment. In T212189#5020187 , @akosiaris wrote: > Thanks for the understanding. We are drafting next quarter goals this week, I 'll make sure to add this. Just poking to double check that this was added (I would

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

WMDE-leszek added a comment. > Final question and just for verification, this ain't going to be exposed directly to the internet after all, right? Rather this will be called by mediawiki, per the architectural diagrams attached to this task. That's correct! TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

akosiaris added a comment. In T212189#5017053 , @Tarrow wrote: > In T212189#5011311 , @akosiaris wrote: > > > I have to say I am wondering a bit about the latency as the low end

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

WMDE-leszek added a comment. In T212189#5011311 , @akosiaris wrote: > I have to say I am wondering a bit about the latency as the low end seems to be quite high (500ms?). It's your service of course and I am fine with it. This

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Tarrow added a comment. In T212189#5011311 , @akosiaris wrote: > I have to say I am wondering a bit about the latency as the low end seems to be quite high (500ms?). It's your service of course and I am fine with it. There is a

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

akosiaris added a comment. That for this, it's appreciated. Note that we haven't still decided over which time window the availability will be calculated, but it's probably gonna be quarertly (3months that is). I have to say I am wondering a bit about the latency as the low end seems to

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

akosiaris added a comment. In T212189#4998182 , @Tarrow wrote: > I am indeed already working on it. > > Just so you know the current state: we are already using blubber for the CI i.e. we have 'service-pipeline-test' run in

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Tarrow added a comment. I am indeed already working on it. Just so you know the current state: we are already using blubber for the CI i.e. we have 'service-pipeline-test' run in zuul/layout.yaml. I suppose this needs to soon include '-and-publish'? Our blubber will obviously need

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

WMDE-leszek added a comment. We do not intend to maintain the "proper" UI logic in PHP. SSR service will render the page on the server side which will (via MediaWiki etc) make it to reader's browser. We do want Wikidata readers and editors have an access to data even if they decide to disable JS

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

daniel added a comment. @Smalyshev My understanding (which may be dated or incomplete) is this: there would be no PHP rendering, JS rendering would need to happen either in the browser on on the server. If the server supports SSR, you can read with a non-JS browser. If the server does not support

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Smalyshev added a comment. Thanks, T213318 makes it a bit clearer though not entirely 100% clear which parts stay in PHP and which parts move to JS. Would it also be true that there is no way to render Wikibase content (even without editing) on non-_javascript_ browsers? The SPA approach in

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

WMDE-leszek added a comment. @Smalyshev I believe the approach we are suggesting really makes a difference when thinking beyond just rendering a template for a particular part of an Item page. I can definitely be blamed for not making it clear enough in the description of this task that we are NOT

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Smalyshev added a comment. I get the idea of server-side HTML rendering to avoid delays. But I am kinda questioning whether the advantage of splitting code outside of PHP into separate service and all the complexities that follow from that. Maybe there's an easier way to achieve the same benefit

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

daniel added a comment. The reason this is a dedicated service is the language it is written in (typescript), which was chosen because it allows us to create an implementation which can be compiled/transpiled to work on both server and client - something not possible with PHP. For the concrete

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Pablo-WMDE added a comment. @Smalyshev No reason to be sorry for asking the right questions! If we truly wanted* to boil the reason down to one sentence: The reason this is a dedicated service is the language it is written in (typescript), which was chosen because it allows us to create an

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Smalyshev added a comment. I've tried to read all of it and maybe I've missed something, but I am still not sure what added value having such separate service gives us. We are creating pretty complex patten of interaction between Mediawiki and outside service, and I am not sure why this service is

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

WMDE-leszek added a comment. It did (today, not on Monday though). I hope the outcome is I hope that @Joe and @akosiaris have a better understanding of what are we have in mind. What we talked about (Wikibase front-end architecture) is also going to be turned into an RFC in next 24 hours.TASK

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

daniel added a comment. Did the chat with @Joe happen? What was the outcome?TASK DETAILhttps://phabricator.wikimedia.org/T212189EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: fselles, CDanis, akosiaris, Krinkle, Milimetric, daniel, mobrovac, Joe,

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

WMDE-leszek added a comment. Thanks everyone for comments so far. This ticket in its current state is definitely not a ready RFC, you're right. We're going to turn it into one/create a separate RFC ticket in upcoming days. As a preparation, we're going to have a little chat with @Joe on Monday, to

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Milimetric added a comment. I agree that we can't go back on decisions that are 3 years in the making. But I do like Timo's point that we should state the problem. Here's an attempt: "Implementing interaction in JS on top of static html rendered by PHP is inflexible and complicated. It leads

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

daniel added a comment. @Krinkle said: Rather, it starts out on the assumption that we're going to have UI code in production (based on Vue.js) written in a way that contains too much business logic in its templating code. That is correct: the Wikidata team made the determination that they want

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Krinkle added a comment. In T212189#4840406, @daniel wrote: So, overall, it seems like the solution proposed by the Wikidata team is the only one viable at this time. I'm not very happy about this, but I don't really see an alternative. [..] I'll look into this RFC in more detail at a later

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

daniel added a comment. @Joe said: the SSR service should not need to call the mediawiki api. It should accept all the information needed to render the termbox in the call from mediawiki. After talking to the Wikidta folks, I realized that this is not easy at all. It requires the callingg code

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Milimetric added a comment. @WMDE-leszek ok, we're on the same page, except the crazy part of my proposal. I was saying directly routed to SSR service as in, without ever hitting mediawiki and spinning up the mediawiki context. So this would expose SSR publicly. The fallback stub HTML generated

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Joe added a comment. In T212189#4840039, @WMDE-leszek wrote: To avoid misunderstandings: I was not questioning MediaWiki's action API being performant. By "lightweight" I was referring to "PHP has high startup time" point @daniel made above as one of the reason why no service should call MW API.

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

WMDE-leszek added a comment. To avoid misunderstandings: I was not questioning MediaWiki's action API being performant. By "lightweight" I was referring to "PHP has high startup time" point @daniel made above as one of the reason why no service should call MW API.TASK

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Joe added a comment. In T212189#4839848, @WMDE-leszek wrote: The intention of introducing the service is not to have a service that call Mediawiki. As discussed above, it is needed for the service to ask for some data, and this data shall be provided by some API. Currently, the only API that

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Joe added a comment. Also, if we're going to build microservices, I'd like to not see applications that "grow", at least in terms of what they can do. A microservice should do one thing and do it well. In this case, it's using data from mediawiki to render an HTML fragment; unless you want to make

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Joe added a comment. In T212189#4838090, @Addshore wrote: The "termbox" is more of an application than a template. Only it knows which data it needs - actively "sending" data to it requires knowledge of which information is needed. While seemingly trivial in the beginning this will, as the

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Joe added a comment. Let me state it again: the SSR service should not need to call the mediawiki api. It should accept all the information needed to render the termbox in the call from mediawiki. So we should have something like: Mediawiki makes a POST request to SSR, sending the entity data,

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

WMDE-leszek added a comment. the request to index.php is conditionally routed directly to the SSR service. In our world, the SSR service is there, so we configure it in Varnish, it returns html, and Vue takes over client-side. For other mediawiki installations, index.php knows to render a basic

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Milimetric added a comment. Thanks @Jakob_WMDE, I think we're saying the same thing in slightly different terms, and it's because I'm not being precise. It's ok for the client and server to have different implementations, but you're saying the have the same capabilities, right? I was thinking

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Jakob_WMDE added a comment. In T212189#4838123, @Milimetric wrote: My question here was more, how can the client render everything it needs, when some of the logic, for example for data-access, is only in the src/server/data-access folder? In other words, if the client functionality is a

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Milimetric added a comment. In T212189#4838090, @Addshore wrote: In T212189#4835359, @Milimetric wrote: Now, I started looking through the code and it looks like there's an effort to keep server and client logic as common as possible, with factories and interfaces and nice patterns, but there

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Addshore added a comment. There sure has been a fair amount of discussion on this ticket! So I have created an updated interacting diagram showing off a few more details of the overall flow (and updated the description) F27687680: Wikidata Termbox SSR Sequence-SSR.png This highlights the various

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

daniel added a comment. You first need to render the page on the server before you know whether the client supports JS/SW or not, so it will need to be rendered on the server irrespective of the client's capabilities in case where MW calls the service directly before handing out the page. Yes,

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

daniel added a comment. When rendering the page, index.php knows the exact data that needs to be rendered already, correct? I just had a brief chat with @Jakob_WMDE and @Pablo-WMDE about this. For the current use case ("term box"), this would be easy, but for the anticipated generalized use case

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

mobrovac added a comment. In T212189#4837824, @daniel wrote: That works, but defies the purpose. The idea is to present a default rendering to clients that don't have JS enabled (or no sufficiently current JS support). That rendering should be generated by the same vue.js code that does the

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

daniel added a comment. In T212189#4837780, @mobrovac wrote: When rendering the page, index.php knows the exact data that needs to be rendered already, correct? If so, it can send that to the client, and then based on whether JS/SW is available or not, the client either renders it or sends a

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

mobrovac added a comment. In T212189#4837768, @daniel wrote: But for the case at hand, there might be a workaround: the PHP code that renders the (Wikibase Entity) page content already knows what data will be needed for the rendering. It can send it to the rendering service along with the request

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

daniel added a comment. @Milimetric wrote: In the simplest case, this code would be almost identical client and server-side. No matter where it's running, nodejs or the browser, it would request data, receive it, and render it as html. I think you are right, that does make sense; Then, the

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

akosiaris added a comment. In T212189#4833536, @daniel wrote: "We should not introduce a service that is called by MediaWiki, and itself calls MediaWiki." Slightly OT, but a +1000 YES to this. Been there, seen that antipattern, it's a mess to reason about. The coupling of the 2 components make

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Milimetric added a comment. In https://wikitech.wikimedia.org/wiki/WMDE/Wikidata/SSR_Service we see that "There is a server-side and the client-side variant of the code, which are distributions of the same implementation." Looking at the current repository, we see that it shares data models

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

daniel added a comment. Well, I consider calling the MW api from a service called by MediaWiki an antipattern that we should absolutely avoid. Oh, I got that wrong - I thought the service would be public facing and called directly from the client! But apparently, itt's not, it would be hidden

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Joe added a comment. In T212189#4833482, @daniel wrote: I agree with Joe that it would be better to have the service be internal, and be called from MW. It doesn't have to be that way, but it's preferable because: we would not expose a new endpoint we should in general avoid (more) services

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

daniel added a comment. I agree with Joe that it would be better to have the service be internal, and be called from MW. It doesn't have to be that way, but it's preferable because: we would not expose a new endpoint we should in general avoid (more) services calling MediaWiki, because: PHP has

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

daniel added a comment. Using accept-language is not an option, at least not the accept-language from the browser. The relevant list of languages comes from user preferences and the Babel extension. This also means that it typically contains more than one language, and there are many, many

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Joe added a comment. In T212189#4831959, @mobrovac wrote: In T212189#4831314, @daniel wrote: @mobrovac Please note that the term box is shown based on user preferences (languages spoken), the initially served DOM however needs to be the same for all users, so it can be cached. Also note that the

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

mobrovac added a comment. In T212189#4831314, @daniel wrote: @mobrovac Please note that the term box is shown based on user preferences (languages spoken), the initially served DOM however needs to be the same for all users, so it can be cached. Also note that the language specific data that goes

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

daniel added a comment. @mobrovac Please note that the term box is shown based on user preferences (languages spoken), the initially served DOM however needs to be the same for all users, so it can be cached. Also note that the language specific data that goes into the term box has to be loaded

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

mobrovac added a comment. As a further optimisation of both the architecture as well as parsing and load times, MW/Wikibase could populate a (hidden?) tag in the DOM with all the info needed to generate the termbox. Then, if JS is enabled (w/ possibly ServiceWorkers), the client simply generates

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Joe added a comment. Also: it is stated in https://wikitech.wikimedia.org/wiki/WMDE/Wikidata/SSR_Service that "In case of no configured server-side rendering service or a malfunctioning of it, the client-side code will act as a fallback". This is a bit the other way around with respect to what is

[Wikidata-bugs] [Maniphest] [Commented On] T212189: New Service Request: Wikidata Termbox SSR

Joe added a comment. Looking at the attached diagrams, it seems that the flow of a request is as follows: page gets requested to MediaWiki MW sends a request to the rendering service the rendering service sends request(s) to mediawiki via api.php to fetch the data, and sends back the rendered