[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

akosiaris added a comment. In T301471#8806115 , @Michaelcochez wrote: > The testing code is now implemented, and we found two small issues with it. These have now been resolved and the code is simplified further. > > Give this

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Dzahn added a comment. @Michaelcochez Probably it makes sense to merge but also comment on the ticket you linked to and get in touch with thcipriani there. Do your repos show up on that ticket? Are they currently on github? TASK DETAIL https://phabricator.wikimedia.org/T301471 EMAIL

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Michaelcochez added a comment. The testing code is now implemented, and we found two small issues with it. These have now been resolved and the code is simplified further. Give this ticket: https://phabricator.wikimedia.org/T332953 I am uncertain whether it makes sense to merge things

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Michaelcochez added a comment. Hi @akosiaris , apologies for the slow progress. We have been updating several things, but I didn't follow up with the testing code needed for the new parts. I have blocked time in the coming days to proceed with this and then we can merge it in. TASK

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

akosiaris added a comment. Any updates on this one? Per previous comment we were waiting on a merge, has this been done? TASK DETAIL https://phabricator.wikimedia.org/T301471 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: akosiaris Cc:

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

akosiaris added a comment. In T301471#7964353 , @Michaelcochez wrote: > Regarding the option of using a batch of queries to an external database; the issue is that what we are creating is a specialized index specifically for what we

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

akosiaris added a comment. In T301471#7964314 , @Michaelcochez wrote: > An update on the current status, mainly regarding the index file: > > First, I made a mistake in my response above. The size of the file is a lot smaller than

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Michaelcochez added a comment. Regarding the option of using a batch of queries to an external database; the issue is that what we are creating is a specialized index specifically for what we need. What we perform is a tree traversal were at each node a new decision is made. To do

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Michaelcochez added a comment. An update on the current status, mainly regarding the index file: First, I made a mistake in my response above. The size of the file is a lot smaller than what I wrote above. The binary version is currently around 75mb (and not 1.5gb). Progress:

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

akosiaris added a comment. In T301471#7840496 , @Michaelcochez wrote: > I merged the pull request on github now. > > I do not have rights to push to the gerrit repository, it might just be my limited knowledge of how gerrit works.

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Addshore added a comment. > I do not have rights to push to the gerrit repository, it might just be my limited knowledge of how gerrit works. I believe you will still need to submit a patch / patches / merge commit to gerrit, and review this +2ing it in the UI. I imagine we will want

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Michaelcochez added a comment. I merged the pull request on github now. I do not have rights to push to the gerrit repository, it might just be my limited knowledge of how gerrit works. I will look into the helm chart/CI setup soon. > questions around the index file: This

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Addshore added a comment. @Michaelcochez looks like we have a few next steps! - Merge the PR (it has approval) - Push that code to gerrit - Try creating a helm chart for the service https://wikitech.wikimedia.org/wiki/Deployment_pipeline/Migration/Tutorial#Creating_a_Helm_Chart and

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

akosiaris added a comment. In T301471#7837692 , @Addshore wrote: > So a summary comment as promised! > > In T301471#7799393 , @Michaelcochez wrote: > >> @QChris I noticed the

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

ItamarWMDE added a project: User-ItamarWMDE. TASK DETAIL https://phabricator.wikimedia.org/T301471 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: ItamarWMDE Cc: akosiaris, QChris, ItamarWMDE, Joe, Aklapper, Addshore, karapayneWMDE, Martaannaj,

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Addshore added a subscriber: akosiaris. Addshore added a comment. So a summary comment as promised! In T301471#7799393 , @Michaelcochez wrote: > @QChris I noticed the addition of the .gitreview file on gerrit. Is this file needed?

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Michaelcochez added a comment. @Joe We now made the changes to use the bullseye distribution and the provided image with go installed See: https://github.com/martaannaj/RecommenderServer/pull/22/files @QChris We just went ahead and added that file in our repository. This is now

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Michaelcochez added a comment. @Joe : would it be fine if we include the index file into the image for the A/B testing phase? If that is successful, we can see whether it makes sense to move it to a different location. We are fixing the blubber configuration in the coming days. Could

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Michaelcochez added a subscriber: QChris. Michaelcochez added a comment. @QChris I noticed the addition of the .gitreview file on gerrit. Is this file needed? If so, we would merge it into our github repository, so we can keep the active development there and synchronize with gerrit. TASK

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

QChris added a comment. In T301471#7725821 , @Joe wrote: > Ok so a few requirements: > > 1. we need the repository to be on gerrit, [...] [X] Done

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Addshore added a comment. In T301471#7714044 , @Joe wrote: > You also say the service will need to read an index file. How is that generated? This is currently generated using this go code

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Michaelcochez added a comment. We'd be happy to receive a patch or pull request. TASK DETAIL https://phabricator.wikimedia.org/T301471 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Michaelcochez Cc: ItamarWMDE, Joe, Aklapper, Addshore,

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Joe added a comment. In T301471#7726097 , @Michaelcochez wrote: > @Joe for the base image, would you recommend our current approach of starting from an 'empty' image and downloading the latest go distribution ourselves, or should we

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Michaelcochez added a comment. @Joe for the base image, would you recommend our current approach of starting from an 'empty' image and downloading the latest go distribution ourselves, or should we take this one: https://docker-registry.wikimedia.org/golang1.17/tags/ I am not sure how

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Joe added a comment. Ok so a few requirements: 1. we need the repository to be on gerrit, and to include a `.pipeline` directory to be built using blubber/the deployment pipeline. 2. you should probably base your image on debian bullseye and not debian stretch, but that can be done

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

MatthewVernon edited projects, added serviceops; removed SRE. TASK DETAIL https://phabricator.wikimedia.org/T301471 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: MatthewVernon Cc: ItamarWMDE, Joe, Aklapper, Addshore, karapayneWMDE, Martaannaj,

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Michaelcochez added a comment. @Joe we have created files for blubber before, I assume what is needed is very similar to that? I am not sure I know what the `deployment-charts repository` is . The code is under development on https://github.com/martaannaj/RecommenderServer/ also the

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Joe added a comment. Hi, if this service is to be used in the WMF production environment (and given the call graph, it will), it needs to run on kubernetes, and thus we will need to be built using our deployment pipeline first, and use the deployment-charts repository to define the

[Wikidata-bugs] [Maniphest] T301471: New Service Request SchemaTree

Maintenance_bot added a project: Wikidata. Restricted Application added a project: wdwb-tech. TASK DETAIL https://phabricator.wikimedia.org/T301471 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Maintenance_bot Cc: Aklapper, Addshore, karapayneWMDE,