On Mon, 2024-11-18 at 10:35 +0100, Linus Nordberg wrote: > Hi all, > > Snapshot is behind Fastly since Sunday Nov 17 2024. I think that's > bad > and would like to change that. It's bad in the short term since we > expose user data to a third party. It's bad in the long term since > the > short term bad won't go away until we learn how to deal with web > traffic.
That's a trade off between the advantages of a CDN and privacy. For me as snapshot user that needs it to build reproducible things in CI systems, the most important aspect is reliability and performance. > > I have not been able to solve the problem with more incoming HTTP > traffic than what the snapshot setup comfortably can deal with. > Partly > because I'm not very knowledgeable in this field and partly because I > have not been given enough access to the cache layer(s). I also had a look at this topic (mostly based on code-review) and identified a couple of problems: 1. apt behaves badly on 429 TooManyRequests. Addressed in [1] 2. Expensive redirects to farm (DB lookup!) are cached too short. Addressed in [2], also affected by [3] 3. Varnish internal redirect to farm not working [4], unfortunately reverted due to not working properly in prod setup [1] https://salsa.debian.org/apt-team/apt/-/merge_requests/383 [2] https://salsa.debian.org/snapshot-team/snapshot/-/merge_requests/23 [3] https://salsa.debian.org/dsa-team/mirror/dsa-puppet/-/commit/63f16e08199040871752135df533f0001fe537fb [4] https://lists.debian.org/debian-snapshot/2024/11/msg00008.html > > DSA have legitimate concerns about exposing user data to people who > do > not need access to it. Would it help if my relation to Debian was > formalised further than the current status of Debian Contributor? I'm just a DM, but I definitely want to help improving the situation. > > More generally, I sometimes find it hard to understand the roles and > responsibilities wrt the snapshot service. This results in me on the > one > hand being overly cautious with asking for some things and on the > other > hand sometimes pestering the wrong people, most probably also in the > wrong way. It would be good to minimise unnecessary frustration and > lost > calendar time. Same! It took me quite some time to get an understanding of the overall architecture of s.d.o which all its layers. Also I don't know who is responsible for the intermediate infrastructure (basically everything between the s.d.o flask app and the DNS entry s.d.o). I further can only guess where exactly the bottlenecks are. These obviously depend on the usage patterns which I (for good reasons) do not have insights into. > > There's a Snapshot service meeting today at 1700Z in #debian-snapshot > for all who are interested in helping out. I'll try to join. Best regards, Felix Moessbauer -- Siemens AG, Technology Linux Expert Center
