i have had this message ready to send to mirror operators for since before the last two mirrors joined the network - another joined this week; and this concern really cant be postponed any longer - the reason why this is important is because new mirrors puts a significant extra load onto the repo server - in 2018, lukeshu told me that we were already pushing the bandwidth limits on 1984's generosity, and the number of mirrors has nearly doubles since then - tiering the mirrors in this way, is a matter of respect, appreciation, and gratitude to 1984
no one on this list needs to do anything; but i thought people may be interested or would like to comment - i also changed the wiki page quite a bit, to be more aligned with the arch article, and added some suggestions, such as seeding the p2p network instead or creating more complete mirrors - the one thing i anticipate that people may have an opinion about, is how i made the requirements more precise and stringent (isacdaavid already did) - anticipating that some may wonder why: the best answer is that the mirror network is very healthy now, healthy than ever - if these are going to be tiered now, then theres no reason not to raise the standard for tier-1 and tier-2, while retaining the original loose requirements for tier-3 https://wiki.parabola.nu/Creating_a_mirror this is the message i sent to all mirror operators: == new tiered mirrors schema == due to an increasing number of mirrors joining the parabola mirror network, the sync schedules are now overlapping in some places - we would like to reorganize them into a more uniform structured schedule for the future - this would reduce the load on our repo server, and also improve the service for parabola users - the increasing number of mirrors is, of course, a good thing - presumably, it indicates that parabola is gaining in popularity; but parabola does not have any funding, so it is putting a strain on our infrastructure currently, all mirrors are pulling directly from parabola; which is pushing the limit of what our extremely generous hosting benefactor can offer for gratis - the plan is such that the relatively modest hosts will be asked to be on "tier-2" (pulling regularly from some relatively faster host, other than parabola.nu); and the higher-capacity hosts will be asked if it is feasible for them to become dedicated "tier-1" mirrors (ones which some number of the tier-2 mirrors will pull from regularly) initially, we would like to have three to six tier-1 mirrors, synchronizing with the parabola tier-0 server, every thirty minutes (5-10 minutes separation); and for each tier-2 mirror to sync once or twice per hour, distributed however the tier-1 mirrors can accommodate them the intended effect is such that delivery times for new packages will be improved for users significantly; with 404 errors being much less likely, due to all mirrors being within a tighter temporal proximity to each other - although a few of the newer mirrors are already on a rapid schedule, most are currently syncing much less frequently - the end-to-end delay is up to two days, for clients of those mirrors; and 404s are common when using the default re-director; because of the loosely-knit federation the accelerated sync rates should have no adverse impact on mirrors - before each sync attempt, the recommended mirroring script downloads a timestamp file, and cancels the complete sync if there are no changes - that is because some megabytes are transferred even when there are no changes - with that guard in place, the total incoming bandwidth for any mirror is essentially the same, regardless of the rate at which the sync script is run - its use was recently added to the wiki as a hard criteria for endorsed mirrors it is, of course, at each mirror operator's discretion to throttle any connections from unknown clients, or those using excessive bandwidth - managing the precise sync schedules of tier-2 mirrors, will be at the discretion of each tier-1 mirror operator - we ask only that tier-1 mirror operators give priority on the rsync channel, to the tier-2 mirrors to which they are feeding, providing that the admins are responsive and agreeable to reasonable terms of use suggestions == mailing list == some of the current mirror operators have suggested a new low-volume mailing list, dedicated for only important communication among the mirror operators, rather than expecting everyone to subscribe to the development mailing list, and/or publishing contact info publicly - it would be extremely rare for parabola to need to send a mass email such as this one; but it could be useful, for occasional coordination of sync schedules, such as now - you should receive an invitation soon - it may be somewhat busy during this initial migration into tiers; but the volume will be close to nil afterward == actions requested == at this time, we have incomplete information on many of the mirrors; so there is no request for action on your part yet - this is the initial attempt to gather all of the information needed to coordinate the network, and subscribe mirror operators to the new mailing list _______________________________________________ Dev mailing list Dev@lists.parabola.nu https://lists.parabola.nu/mailman/listinfo/dev
