Here is the progress on the upload of old packages to archive.org.
I uploaded a few packages to test if my script works:


There is one identifier for each package, and then all versions of the
package + their signatures are contained under this identifier (see "Show
all" on the right).

Now, to finish this, I have a few questions:

- does the devops team have a place to store passwords?  I would like to
  create an "Arch Linux" account, so that I'm not the only one to have
  access.  I also need an email address for the account, maybe something
  like internetarchive at archlinux.org or just the devops mailing list address?

- is that OK to upload ~2 TB from orion?  Is the server on an limited data

- I'm still waiting on a final confirmation from archive.org, whether they
  are OK with this amount of data.

The upload process itself is quite slow latency-wise, it takes 5-10
seconds to upload a file whatever its size.  For packages from 2013 to
2015 there's 250k files to upload, I estimate it will take a few days if I
run 32 upload threads in parallel.

By the way, we could even keep the year/month/day symlink hierarchy on
orion for old packages, and redirect downloads to archive.org.  There is
just a small issue with packages that have "+", "@" or "." in their name,
because that's not allowed as identifiers in archive.org (see the second
and third examples above, where my script replaced the "@" with "_")


Attachment: signature.asc
Description: PGP signature

Reply via email to