Our Scholia webapp on Toolforge is struggling with the request load at https://scholia.toolforge.org/
According to toolviews (also displayed at https://people.compute.dtu.dk/faan/scholia-page-view-statistics.html) we had yesterday almost 700,000 daily hit. Monitoring the log (tail -f uwsgi.log) I see quite a lot of various "troubling" hits, e.g., to the static files. I do not know what it is but I imagine it is some kind of GenAI wannabees with a Playwright script crawling Scholia. In the current Scholia webapp code, we have embarrassments: 1) Serving static files in the Flask webapp. 2) Blocking requests to Wikidata API or the SPARQL endpoint here and there in the code (most Scholia requests are client-side SPARQL requests though). I have been moving some static file requests to tools-static.wmflabs.org, but still need to do some more that is embedded in Bootstrap. I am thinking about moving from Flask to an async framework. I am gaining some experience with FastAPI for web services. Am I correct that async on Toolforge will buy us a bit of extra performance? Are there other Toolforge users that have struggling webapps and, if yes, then what do you do? If it continue we could do login I suppose. In the Scholia repo we have this issue: https://github.com/WDscholia/scholia/issues/2727 best regards Finn Årup Nielsen https://people.compute.dtu.dk/faan/ _______________________________________________ Cloud mailing list -- [email protected] List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
